132 results about "Q learning algorithm" patented technology

The invention discloses an edge computing resource allocation optimization method and system based on reinforcement learning, and relates to the technical field of edge computing resource management.The method comprises the steps that computing load data and network state data are collected, a historical load sequence is formed, a state space matrix is constructed, the historical load sequence is mapped into a discrete state value, and the discrete state value is obtained; establishing a deep neural network model; inputting the state space matrix into a deep neural network model, and generating a resource allocation strategy set according to the priority and time delay requirements of the calculation tasks; iteratively training the resource allocation strategy set by adopting a double Q learning algorithm, setting a reward function based on task completion time and a resource utilization rate, and selecting an optimal resource allocation strategy; and adjusting computing resource allocation of the edge nodes according to the optimal resource allocation strategy, and meanwhile, feeding back an execution result to the deep neural network model for online learning updating. According to the method, intelligent optimization of edge computing resource allocation is realized by introducing a deep reinforcement learning method.

The invention relates to the technical field of Internet of Things management, in particular to a lightweight Internet of Things management system based on a star flash protocol stack, which comprises a plurality of modules such as a star flash protocol communication module, a dynamic topology management module and a resource virtualization module. The satellite flash protocol communication module realizes low-power-consumption efficient connection of equipment; the dynamic topology management module optimizes the network topology by using an improved multi-agent Q learning algorithm; the resource virtualization module realizes accurate resource allocation through a weighted bipartite graph matching model; the safety protection module adopts an attention mechanism to detect abnormities and guarantee data safety; the edge co-processing module realizes intelligent task unloading by means of deep reinforcement learning; all the modules cooperatively work under the overall planning of the cross-module coordination controller, and the information barrier is broken. According to the invention, the problems of high communication energy consumption, poor resource allocation, weak security protection, low task processing efficiency, insufficient module collaboration and the like of a traditional Internet of Things system are effectively solved, the system communication efficiency, the resource utilization rate and the security are remarkably improved, and the task processing capability is enhanced.

The invention discloses a video transmission method and system based on streaming processing, and relates to the technical field of video transmission, and the method comprises the steps: collecting a video stream, carrying out the preprocessing, extracting a video frame, packaging the video frame into a picture stream, carrying out the data fragmentation, distributing an initialization path for fragmented data, and calculating an initial traffic proportion; and optimizing path selection and adjusting path flow distribution according to network feedback to obtain a path optimization result, and transmitting the fragmented data according to the path optimization result. Through multi-level feature extraction and key frame selection of video frames, fragmented packaging of video data is realized, a dynamic path optimization technology is combined, path selection and flow distribution are dynamically adjusted based on a Q learning algorithm and a luminescence firefly algorithm, the transmission efficiency is improved, video image processing and a dynamic network optimization technology are fully combined, and the video image processing efficiency is improved. And a brand new solution is provided for efficient and stable transmission of high-definition video streams.

The invention discloses an AI model adaptive tuning method based on deep reinforcement learning, and belongs to the technical field of industrial process control, the method comprises the following steps: firstly, constructing a deep reinforcement learning environment comprising a state space, an action space and a reward function, the state space comprising a process variable measurement value, a controller parameter and a performance index; the action space is a parameter adjustment quantity vector, and the reward function comprehensively considers the stability, rapidity and accuracy of the system; then, based on a deep Q learning network of a dual-network structure, training data are stored and sampled by adopting an experience playback mechanism, and network parameters are iteratively optimized through a deep Q learning algorithm; and finally, realizing online adaptive adjustment of controller parameters by using the optimized deep Q network. The method has the advantages of being high in adaptability, good in optimization performance, high in practicability and the like, intelligent optimization control over an industrial process control system can be achieved, and the overall performance of the system is improved.

The invention discloses a building environment dynamic regulation and control method and system based on Internet of Things driving, particularly relates to the field of building environment intelligent regulation and control, and is used for solving the problems of multi-region environment dynamic regulation and control and fine management. The environment data of each sub-region of the building is accurately acquired and synchronized, real-time region division is realized in combination with an adaptive clustering algorithm, and the accurate capture of environment features and the reasonability of space division are ensured. On the basis, the independence and interaction strength of the sub-regions are comprehensively evaluated by calculating an environmental independence index, and a scientific basis is provided for refined and coordinated regulation and control. Based on a rule engine and a decision tree reasoning algorithm, an optimal regulation and control strategy is automatically generated, and continuous optimization is performed in combination with a Q learning algorithm, so that the system can adapt to environmental changes, dynamically adjust regulation and control actions, realize intelligent optimization and self-learning of the regulation and control strategy, and remarkably improve the regulation and control efficiency and accuracy of the building environment.

The invention discloses an AI-based automatic production line scheduling system in an industrial internet, which relates to the technical field of production scheduling and comprises a production plan management module S1, a dynamic scheduling engine module S2, a resource scheduling module S3, a real-time monitoring system module S4, an exception handling center module S5 and a data optimization platform module S6. In the industrial internet, an AI-based automatic production line scheduling system, an X dynamic scheduling engine millisecond response and a multi-agent reinforcement learning engine based on a federated learning architecture realize millisecond response scheduling, each device is used as an autonomous decision-making unit, and dynamic coordination is performed through a distributed Q learning algorithm, so that the vacancy rate of the devices is greatly reduced, and the scheduling efficiency is improved. According to a long-short-term memory network deep analysis model of order delivery cycle compression, emergency order insertion response speed improvement, multi-modal AI quality monitoring, fusion of vibration, thermal imaging and current spectrum, the detection rate is greatly improved compared with a unified sensor, causal reasoning and root cause analysis are performed, a fault causal graph is constructed to position a deep problem, and the average repair time is shortened.

The invention provides a charging pile scheduling method and system, and belongs to the technical field of charging pile energy management, and the method comprises the steps: predicting the second state data of a charging pile group based on the historical state data of the charging pile group and the first state data of the charging pile group, taking the historical state data, the first state data and the second state data as state data of the charging pile group; wherein the first state data is the state data of the charging pile group in the current preset time period; the second state data is predicted state data of the charging pile group in a future target time period; processing the state data of the charging pile group, the demand data of the charging vehicle and the power grid load data based on an ant colony algorithm to obtain an initial charging scheduling strategy; and adjusting the initial charging scheduling strategy based on a Q learning algorithm to obtain a target charging scheduling strategy. According to the charging pile scheduling method and system provided by the invention, the charging efficiency can be improved.

The invention relates to a disturbance-containing mechanical arm trajectory tracking control double-Q reinforcement learning method, and belongs to the technical field of mechanical arm intelligent control. According to the method, a double-Q reinforcement learning method is adopted to solve the problem of trajectory tracking control of the interfered mechanical arm. An interference observer is designed to estimate and counteract the influence of external interference and model inaccuracy, and the control precision and interference suppression capability of the mechanical arm system are improved; a tracking controller based on a double-Q reinforcement learning method is designed, and the control cost is kept while the dynamic and steady-state performance of the system is improved by utilizing the excellent performance of a double-Q learning algorithm; in addition, the problem that the Q value is too high in a traditional Q learning method is solved through the double-Q reinforcement learning method. According to the method, the advantages of the double-Q reinforcement learning method are effectively utilized, under the condition that the system contains disturbance and uncertain factors, the fast tracking performance of the mechanical arm system on the expected trajectory is achieved, and the good steady-state performance is achieved.

The invention discloses an electromagnetic micromirror output feedback tracking control method and device based on adaptive Q learning, and the method comprises the steps: constructing an electromagnetic micromirror model based on a physical structure of an electromagnetic micromirror; combining the electromagnetic micro-mirror model with a reference trajectory to construct an augmentation system so as to convert a trajectory tracking problem into a state adjustment problem; based on the augmented system, designing a Q function Bellman equation to establish a data-driven performance evaluation framework; based on the augmented system, the state of the augmented system is reconstructed through historical input and output data, so that angular velocity measurement is avoided; obtaining an optimal control input and a new Q function Bellman equation based on the reconstructed augmented system state and the Q function Bellman equation; and based on the optimal control input and the new Q function Bellman equation, solving a control problem by using an adaptive Q learning algorithm. The method breaks through the limitation that a traditional model depends on a control method, and achieves optimal tracking control only through historical input and output data.

The invention relates to the technical field of power dispatching optimization, in particular to a multi-source power real-time dispatching optimization method and system based on reinforcement learning. The method comprises the following steps: acquiring required load information data of power generation equipment in different time periods; constructing an optimal scheduling strategy model of hybrid wind-fire distributed power generation; adopting a Monte Carlo method and a double-Q learning algorithm to optimize the optimal scheduling strategy model; and performing multi-source power real-time scheduling optimization by using the optimal scheduling strategy model. The multi-source electric power dispatching method is optimized in real time through the reinforcement learning method, the optimal electric power dispatching strategy is searched, compared with a traditional method, the electric power cost is reduced by 15% or above, meanwhile, automatic dispatching can be achieved, and manual intervention and experience dependence are reduced.

The invention discloses a method and a system for preventing a pressure sore pad based on deep Q learning and an attention mechanism. The method comprises the steps that pressure distribution data and posture data of a patient are collected through a pressure sensor and a posture sensor, normalization processing is carried out on the collected data, and a standardized state data flow is generated; and constructing a contact surface pressure distribution model of the patient and the pressure sore prevention pad according to the standardized state data flow to obtain a contact surface state matrix and the like. According to the method, a self-attention mechanism and a deep neural network are adopted, a Q learning algorithm is combined, predefined air bag pressure adjustment actions are evaluated and optimized, and an optimal air bag adjustment strategy is generated. The local pressure of the pressure sore prevention pad is adjusted according to the strategy. The state of a patient can be intelligently sensed, pressure distribution of the pressure sore prevention pad can be adjusted in a self-adaptive mode, pressure sores are effectively prevented, and nursing quality and patient comfort are improved.

The invention discloses a smart power grid false data injection identification method based on deep reinforcement learning. The intelligent power grid false data injection identification method based on deep reinforcement learning comprises the following steps that a preset node in a power grid is selected as a reference node, a system node state set is obtained, and the system state set comprises a phase angle of a system at a detection moment and a measurement value of a detector at the detection moment; constructing a direct-current power grid system model based on the system node state set, and defining an attack strategy in the direct-current power grid system model; randomly selecting the system node state set as initial sample data, inputting the initial sample data into the direct-current power grid system model, and obtaining a countermeasure sample in combination with a deep Q learning algorithm; and building a multi-layer attack detection algorithm framework to train the adversarial sample to obtain a power grid hostile attack recognition model. According to the method, the problem of low false data attack detection precision is solved through adversarial sample generation and a multi-level attack detection algorithm framework.

The invention provides a low-delay video stream code rate adaptation method and system based on off-line element reinforcement learning. The method comprises the following steps: constructing an off-line expert track data set; constructing a code rate adaptation strategy network, and outputting a downloading code rate selection result and a network throughput correction value by taking a video stream transmission state as input; and adopting a meta-implicit Q learning algorithm to perform double-layer alternate optimization of an inner layer and an outer layer on the policy network, the inner layer updating policy parameters of the policy network based on the offline expert trajectory data set, and the outer layer further adjusting the policy network parameters through meta reinforcement learning to improve the adaptability of the policy network to unseen tasks. According to the method, multiple expert algorithm experiences are integrated through offline reinforcement learning, high generalization and adaptive ability can be realized in multiple target time delays and complex network environments without online trial and error exploration, and finally, high-quality and smooth playing of video streams in a low-time-delay live broadcast scene is supported.

The invention discloses a multi-agent system double-layer Q learning control method and system based on Warisstein distance. The method comprises the following steps: establishing a multi-agent system state space model under fault and unknown distribution interference; constructing an inner-layer fault-tolerant control system and an outer-layer robust control system; designing an internal fault-tolerant control gain based on a Q learning algorithm; designing an outer layer robust control gain and a bias item based on the Warisstein distance and a zero-sum game framework; and designing a distributed consistency protocol in combination with neighborhood state information. According to the method, under the conditions that a system model is unknown, an executor has additive time-varying faults, external disturbance probability distribution is uncertain and only depends on limited samples, the influence of compensation faults and disturbance can be effectively reduced, the H infinity performance constraint is met, meanwhile, asymptotic state synchronization of all agents is achieved, and system stability is ensured.

The invention discloses a data center global temperature control optimization method based on online-offline hybrid reinforcement learning, and the method comprises the steps: constructing an interpretable global state space fusing physical information, and guaranteeing that a state variable has a clear physical meaning; a combined action space is defined, and cooperative control of the cold source side and the tail end side is achieved; designing a multi-objective reward function based on physical model driving, and comprehensively considering energy consumption, temperature stability, carbon emission and physical optimization indexes; obtaining a basic security policy through offline pre-training, and extracting reliable behaviors from historical data by using a conservative Q learning algorithm; strategy optimization is achieved through online safety fine adjustment, and gradual adjustment is conducted under multiple constraints to adapt to real-time changes; and finally, deploying an optimization strategy to realize closed-loop control of the system, and establishing a continuous learning mechanism to cope with long-term drift. According to the method, a complete offline-online mixed learning system is established, and the energy efficiency of the data center temperature control system is improved.

An adaptive virtual synchronous machine control method based on deep reinforcement learning belongs to the field of power grid operation control. It establishes a photovoltaic storage grid-connected model containing photovoltaic systems, battery energy storage and variable loads; uses virtual synchronous generator control to achieve complementary photovoltaic output power for the battery energy storage system; in order to solve the problem of microgrid frequency deviation caused by microgrid load power changes, a deep reinforcement learning algorithm is used to control and adjust the virtual inertia and damping coefficient of the virtual synchronous generator; the double deep Q learning algorithm in deep reinforcement learning is used to control the virtual inertia and damping coefficient; a double-layer evaluation network is constructed, combined with a reward function, and after multiple trainings, the adaptive adjustment of the virtual inertia and damping coefficient is achieved. It uses deep reinforcement learning technology to adaptively control the virtual inertia and damping coefficient, suppress the frequency deviation, accelerate the recovery of the system operating frequency, and improve the stability of the system.

According to the method, a fuzzy subset is set according to the required torque, the current SOC value and the difference value between the engine rotating speed and the motor rotating speed, and the torque of an ISG motor is dynamically adjusted through a fuzzy control algorithm. The fuzzy control adopts a fixed fuzzy rule, so that a Q table is established through a Q learning algorithm, the compensation torque of the ISG motor is obtained, the final output torque of the ISG motor is determined to be the sum of the output torque of the fuzzy controller and the compensation torque output by the Q learning algorithm, and the torque output by the ISG motor is determined. According to the method, the advantage of high response speed of the torque of the motor is utilized, the difference value between the required torque and the actual torque is effectively compensated, the rotating speed fluctuation of the engine caused by overlarge change of the torque of the engine is avoided, and the fuel economy and the driving operation stability are improved.

The invention provides a micro-grid dispatching control method, device and equipment based on strategy optimization and a medium, and belongs to the technical field of power grid dispatching, and the method comprises the steps: collecting the real-time operation data of new energy power generation equipment, power utilization load equipment and an energy storage system; receiving power generation and power utilization plans of a user, extracting multi-resource dynamic response characteristics through an LSTM model in combination with the real-time operation data and the historical operation data, and outputting an adjustment capability evaluation matrix; receiving a power grid dispatching plan, matching an optimal operation mode from the knowledge base based on the adjustment capability evaluation matrix, and generating a dispatching instruction set; issuing a scheduling instruction set to the target device and executing the scheduling instruction set, collecting execution data in real time and calculating operation deviation; when the operation deviation exceeds a threshold value, an online Q learning algorithm is adopted to update the instruction weight, and an optimization strategy is fed back to the knowledge base. Through real-time data acquisition and an intelligent optimization algorithm, fine scheduling of the micro-grid is realized, the resource utilization rate and the operation flexibility are improved, and the operation cost is reduced.

A non-strategy output feedback optimal tracking control method for a time-delay industrial process with unknown interference belongs to the field of advanced control of industrial processes, can overcome the problem that a conventional time-delay process control method excessively depends on a system model, and specifically comprises the following steps: step 1, describing a time-delay space expression form with interference, introducing a state increment and an output error to establish a new augmentation model; 2, introducing a time delay performance index, and designing a control strategy; 3, proposing a Q learning algorithm with fault-tolerant performance under a time delay condition, and solving an optimal control gain through continuous learning; according to the method, the optimal controller gain matrix can be found through continuous learning under the condition that a system accurate model is unknown, and a system output track can well track an expected value and achieve a good control effect under the condition that the system has state time delay and unknown interference.

The invention provides a high-order robust control method for a mobile robot in combination with a weighted double-Q learning algorithm. The high-order robust control method comprises the following steps: step 1, constructing an intelligent trolley motion scene including time-varying friction, variable gradient, random external disturbance and sensor noise complex uncertainty; step 2, designing a weighted double-Q learning adaptive nonlinear expansion state observer; step 3, based on the step 2, constructing a reward function including a trajectory tracking error, a disturbance estimation error and a stability constraint; 4, designing an improved super-spiral sliding mode control law based on the weighted double-Q learning adaptive nonlinear extended state observer; and 5, setting a plurality of groups of working condition simulation comparison schemes, and carrying out simulation experiment verification. According to the invention, real-time adaptive control of the mobile robot on different working conditions is realized, the trajectory tracking precision of the system is improved, and sliding-mode control buffeting is inhibited at the same time.

The invention provides a power transmission network power failure maintenance scheduling optimization method based on Q learning, and the method comprises the steps: generating a Gantt chart through obtaining an original topological file of a power grid, and determining a state judgment rule and a target function of reinforcement learning; integrating the maintenance plans of the subordinate units to a Gantt chart to form an initial maintenance plan; calculating an initial power flow set by using a Matlab built-in power flow solver, and evaluating the operation state of the power grid; in the agent training process, an epsilone-greedy method is adopted to select a maintenance action, update a Gantt chart and a topological file, and update a Q value through a Bellman equation to optimize a maintenance plan; and after training is completed, outputting a Gantt chart of a final power failure maintenance plan. According to the method, the power grid dispatching problem is abstracted into the Q learning problem, power failure dispatching is optimized by utilizing the Q learning algorithm through the reasonable constraint and the objective function, the working intensity of dispatching personnel is relieved, the talent training cost is reduced, the dispatching efficiency and reliability are improved, and the key technical problem of power grid dispatching is solved.

The invention discloses a multi-modal sequence recommendation method and system based on hyperbolic curvature adaptive learning, and the method comprises the steps: exploring the adaptive hyperbolic curvature of a time sequence behavior diagram in different modals based on a reinforcement learning frame through a curvature adaptive learning agent; modeling the multi-modal interest of the user in a hyperbolic space with a specific curvature through a multi-modal interest learning agent; a Nash Q learning algorithm is adopted to cooperatively update a curvature adaptive learning agent and a multi-mode interest learning agent, and when the two agents reach Nash equilibrium, adaptive curvatures in different modes are explored; and learning a user behavior interaction mode under the image mode and the text mode in the hyperbolic space, predicting the score of the user for the next multi-mode item, and carrying out multi-mode item recommendation based on the predicted score. According to the method, the hyperbolic space is used as the embedding space, and the user preference information is mined by using the text and image modal information of the project, so that the overall recommendation performance is improved.

The invention discloses a multi-robot collaborative operation and task allocation system and method, relates to the technical field of robot operation, and is used for solving the problems that in a traditional method, task allocation efficiency is low, resources are wasted, especially in a multi-robot system, tasks cannot be accurately allocated, dynamic adaptability is lacked, task and environment changes cannot be coped, and the task allocation efficiency is low. An optimal decision strategy is obtained step by step; and a single learning algorithm has defects. An auction algorithm is used for solving the task allocation problem of the robot, the work progress and the space matching degree of the robot are considered, a Q learning algorithm is used for determining whether to switch to a Sarsa learning algorithm or not according to the task completion condition of the robot in combination with environment interaction and a continuous optimization strategy, and the task allocation efficiency is improved. The design of the flexible switching strategy makes up for the defects of single reinforcement learning strategy and lack of flexibility and adaptability in the prior art, so that the robot can dynamically adjust the optimization algorithm according to the actual situation, and the task completion quality and efficiency are improved.

The invention discloses a manufacturing method of a packaging substrate of an LED electronic display screen, and relates to the technical field of LED circuit board processing. Through integration of a material demand plan MRP system and CNC cutting equipment and utilization of an intelligent typesetting algorithm based on a genetic algorithm, the utilization of substrate materials is maximized, and waste is reduced; an intelligent drilling adjustment system is constructed by adopting a depth Q learning algorithm, real-time adjustment and optimization of drilling parameters are realized, and the hole position precision and the hole shape consistency are ensured; in the intelligent plating process, electroplating parameters are monitored and adjusted in real time by utilizing a neural network model based on deep learning, the uniformity and adhesive force of an electroplated layer are improved, and finally, an automatic plate grinding machine is adopted for resin grinding in the press-fit brownification and resin grinding stages, so that the flatness of the surface of a substrate is ensured.

The invention discloses an equipment fault detection data analysis method and system for a digital factory. The method comprises the following steps: S1, synchronously acquiring comprehensive maintenance data of equipment by using a high-precision clock synchronization technology; s2, constructing a graph neural network by taking the position data of the devices as graph nodes and taking function association and data interaction data between the devices as edges of a graph, constructing a device association model based on the graph neural network, and taking the state data of the devices as input and output state association factors of the device association model; s3, taking the comprehensive maintenance data and the state correlation factor as state input of a reinforcement learning agent, training the agent by adopting a Q learning algorithm, inputting real-time state data into the agent, outputting a state score, setting an abnormal state threshold value, when the state score is lower than the abnormal state threshold value, marking the corresponding real-time data as abnormal, and when the state score is lower than the abnormal state threshold value, marking the corresponding real-time data as abnormal data. According to the method, unnecessary shutdown and maintenance can be reduced under the condition that the operation state of the equipment is complex and changeable.

The invention discloses a multi-objective optimization method and system for a heating and ventilation system based on reinforcement learning, and relates to the technical field of building energy system optimizing.The method comprises the steps that a state space is constructed based on collected environment parameters, equipment state parameters and historical operation data of the heating and ventilation system; the state space is composed of state vectors generated after normalization and discretization; constructing an action space based on controllable equipment of the heating and ventilation system; the action space is generated by discretizing the operation mode and control parameters of the controllable equipment; constructing a multi-target reward function to evaluate a control effect based on the state vector and the action space; taking the state vector as input, and performing iterative training by using a Double Q learning algorithm until a trained control model is obtained; and inputting the currently acquired state vector into the trained control model, and outputting an optimal action to generate a control instruction for the heating and ventilation system.

The invention discloses a self-adaptive multi-path scheduling method based on deep reinforcement learning in a heterogeneous network, which comprises the following steps: in a link management module, introducing a multi-path transmission gain model for quantifying a trade-off relationship between aggregation bandwidth improvement and end-to-end delay increase of a plurality of links, by screening the link sets with better performance and smaller difference, the negative influence of low-performance links on the transmission performance of the system is effectively reduced; in the deep reinforcement learning module, the adaptability, learning efficiency and decision accuracy of the model to an unstable environment are improved by combining priority experience playback and an asynchronous learning mechanism, and optimal balance between network throughput and reordering delay is realized by dynamically adjusting a data packet distribution strategy and a transmission sequence. According to the method, the asynchronous priority replay double-depth Q learning algorithm is adopted, real-time perception of the link state and decision scheduling of the data packet can be realized in a dynamic network environment, the aggregation throughput of the network is remarkably improved, and the transmission delay is effectively reduced.

The invention discloses a spacecraft many-to-many game confrontation strategy planning and optimizing method, and belongs to the field of multi-satellite task planning. Comprising the following steps that modeling is conducted on many-to-many spacecraft game confrontation, the situation of mutual influence of strategies of two game parties is described, and the situation of mutual influence of the strategies of the two game parties comprises multiple groups of single-round games; in the single-round game, modeling is carried out on the multi-target task planning of the two game parties, and multi-target task planning models of the two game parties are obtained; the multi-target task planning models of the two game parties are solved based on a genetic algorithm, and decision-making schemes of the round of the two parties are obtained; and continuously training the many-to-many spacecraft game model through a Q learning algorithm to obtain a multi-target task weight of the attacker in each game round. According to the method, the problem of unreasonable multi-target optimization weight can be solved, so that relatively high game benefits are finally obtained, and the accuracy and efficiency of multi-satellite task processing are improved.

The present invention provides a model-free control method for a human-in-the-loop multi-agent system under a denial of service attack, belonging to the field of multi-agent system control technology. This control method is based on human-in-the-loop technology and introduces external human experts to supervise the system to improve the security and reliability of the system; at the same time, the DoS attack between the intelligent agents is modeled as a time-varying switching topology, and a fully distributed designated time leader output observer is designed, which effectively reduces the adverse effects of the DoS attack jump on the observation effect and achieves the designated time convergence performance of the observation error; in addition, an optimization performance function is designed and a model-free Q learning algorithm is derived. This control method realizes model-free optimization control of a human-in-the-loop multi-agent system under the influence of DoS attacks, while taking into account the designated time convergence performance of the consistency error, effectively improving the control quality.

The invention belongs to the technical field of job shop scheduling, and particularly relates to a self-learning job shop scheduling method meeting waiting time constraint, which comprises the following steps: S0, constructing a job shop scheduling problem model with waiting time constraint; the method comprises the following steps: S1, acquiring job shop scheduling problem data, a configuration algorithm and operation parameters; s2, constructing chromosome individuals, generating chromosomes and initializing a population; s3, calculating the fitness value of each chromosome individual; s4, forming a new generation of population; s5, combining the fitness information of the current population, dynamically selecting a crossover rate Pc through a Q learning algorithm, and performing crossover operation on the population; s6, dynamically determining a mutation rate Pm in the same parameter combination space by using a Q learning algorithm, and performing mutation operation on the crossover progeny to generate an updated population; and S7, judging whether an iteration termination condition is met or not. According to the method, a feasible and near-optimal scheduling scheme can be efficiently generated on the premise of ensuring that the inter-process waiting time constraint is met.