Flexible production line adaptive scheduling control method fusing multi-agent reinforcement learning

Abstract

The invention provides a flexible production line adaptive scheduling control method fusing multi-agent reinforcement learning, and relates to the technical field of production line scheduling, and the method comprises the steps: obtaining the state of processing equipment and workpiece demand information in real time, constructing a state and action space, mapping the equipment into agents, achieving information sharing, and then selecting a scheduling action and executing the scheduling action. And the state-action value function is updated through the experience playback pool, and the multi-agent model is optimized. The system monitors the equipment state during operation, dynamically updates the state and action space to deal with faults or material shortage, and generates a compensation scheduling scheme. Finally, in combination with a real-time scheduling instruction, adaptive scheduling control of the flexible production line is realized, and the production efficiency and flexibility are improved.

Description

Technical Field

[0001] The present invention relates to production line scheduling technology, and in particular to a flexible production line adaptive scheduling control method integrating multi-agent reinforcement learning. Background Art

[0002] As an important part of modern manufacturing industry, flexible production line has the characteristics of product diversification, small production batches, frequent switching, etc., which puts forward higher requirements on production scheduling control system. Traditional flexible production line scheduling methods mainly include heuristic algorithms, mathematical programming and intelligent optimization algorithms, etc. These methods have realized the automatic scheduling of production process to a certain extent. In recent years, with the development of artificial intelligence technology, reinforcement learning has received extensive attention because of its ability to continuously optimize the decision-making process by interacting with the environment. In particular, multi-agent reinforcement learning technology can better adapt to the characteristics of equipment distribution and task collaboration in flexible production lines.

[0003] However, the existing flexible production line scheduling and control methods still have some obvious shortcomings. First, most traditional scheduling algorithms are designed based on static optimization models and cannot effectively deal with dynamic disturbances in the production process, such as equipment failures, material shortages and other emergencies, resulting in poor stability and robustness of the scheduling scheme. Secondly, when dealing with large-scale production systems, existing reinforcement learning methods often adopt a centralized learning framework with high computational complexity, which makes it difficult to meet real-time decision-making requirements. In addition, it is difficult for a single intelligent agent to fully perceive the state information of a complex production environment, which affects the accuracy of scheduling decisions. Third, the existing technology lacks an effective adaptive mechanism. When the production environment changes, it is impossible to quickly adjust the strategy and generate a reasonable compensation plan, making it difficult to achieve continuous and efficient operation of the production line, especially the weak recovery ability under abnormal conditions.

[0004] By introducing a flexible production line adaptive scheduling control method that integrates multi-agent reinforcement learning, the above problems can be effectively solved, the intelligent adaptive scheduling of the production line in a dynamic and complex environment can be realized, and the overall operation efficiency and anti-interference ability of the system can be improved. Summary of the invention

[0005] The embodiments of the present invention provide a flexible production line adaptive scheduling control method integrating multi-agent reinforcement learning, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention, Provides a flexible production line adaptive scheduling control method integrating multi-agent reinforcement learning, including: Acquire the real-time operating status of multiple processing equipment in the flexible production line and the processing requirement information of multiple workpieces, and construct a state space and an action space according to the real-time operating status and the processing requirement information; Mapping the multiple processing equipment to corresponding intelligent agents, establishing a local communication network between the intelligent agents to achieve information sharing between adjacent intelligent agents, and the intelligent agents selecting and executing scheduling actions based on current state information and local observation information of neighboring intelligent agents; After executing the scheduling action, the system instant reward value is obtained and stored in the experience replay pool, data is sampled from the experience replay pool to update the state-action value function, and an optimized multi-agent reinforcement learning model is obtained through repeated iterative training; The multi-agent reinforcement learning model continuously monitors the operating status of the multiple processing equipment during operation, and when equipment failure or material shortage is identified, dynamically updates the state space and action space of the affected area, and uses the updated state space and action space to recalculate the scheduling strategy to obtain a compensation scheduling solution; The compensation scheduling scheme is combined with the output results of the multi-agent reinforcement learning model to generate a real-time scheduling instruction sequence. The real-time scheduling instruction sequence is decomposed into specific control instructions for each processing equipment and issued to the corresponding processing equipment, thereby realizing adaptive scheduling control of the flexible production line.

[0007] Constructing the state space and the action space according to the real-time operation state and the processing requirement information includes: The real-time operation status of the flexible production line includes the equipment status, task queue and remaining processing time, and the processing demand information includes the processing sequence and processing time of the workpiece; The real-time operating status and the processing requirement information are mapped to a feature vector space, the spatial features in the feature vector space are extracted through a deep neural network, and a state space and an action space are dynamically constructed according to the spatial features, wherein the state space represents the real-time state distribution of the processing equipment and the workpiece, and the action space represents an executable scheduling decision set.

[0008] The plurality of processing equipment are mapped to corresponding agents, a local communication network is established between the agents to realize information sharing between adjacent agents, and the agents select scheduling actions and execute them according to current state information and local observation information of neighboring agents, including: The plurality of processing equipment in the flexible production line are mapped into corresponding intelligent agents, and each of the intelligent agents extracts features of its own state information based on an attention mechanism to obtain a state feature vector reflecting the real-time working characteristics of the processing equipment; Constructing a dynamic topology intelligent agent communication network, modeling the degree of association between the intelligent agents based on the intelligent agent communication network, determining the communication weight by calculating the state similarity and task relevance between the intelligent agents, and adaptively adjusting the communication weight as the state of the intelligent agent and the task allocation change, so as to realize the interaction of differentiated information between the intelligent agents; A hybrid strategy network is constructed based on the state feature vector and the differential information, wherein the hybrid strategy network combines deterministic strategy with random exploration, outputs deterministic action distribution through the hybrid strategy network, and introduces random noise to realize strategy exploration, wherein the training goal of the hybrid strategy network is jointly determined by task completion and collaborative efficiency; A scheduling action is selected according to the output result of the hybrid strategy network, and state transfer information and reward information after executing the scheduling action are stored in the experience pool.

[0009] Constructing a dynamic topology agent communication network, modeling the degree of association between the agents based on the agent communication network, and determining the communication weight by calculating the state similarity and task relevance between the agents includes: Constructing multiple agents into a directed graph network structure, wherein nodes in the directed graph network structure represent agents, and edges in the directed graph network structure represent communication links, determining a corresponding neighborhood node set for each of the agents, obtaining a device operating parameter vector, a task queue state vector, and a resource state vector of each of the agents, and combining the device operating parameter vector, the task queue state vector, and the resource state vector to form a state vector of the agent; Extract features from the state vector to obtain a feature vector, calculate the inner product of the feature vectors corresponding to any two of the agents, divide the inner product by the square root of the feature dimension, and then pass the resultant softmax activation function to obtain the connection strength; The state similarity is obtained by calculating the dot product of any two state vectors divided by the product of the respective vector moduli, the cosine similarity between any two agents is calculated and used as the task relevance, and the state similarity and the task relevance are weighted to obtain the communication weight.

[0010] The multi-agent reinforcement learning model continuously monitors the operating status of the multiple processing equipment during operation. When equipment failure or material shortage is identified, the state space and action space of the affected area are dynamically updated, and the scheduling strategy is recalculated using the updated state space and action space to obtain a compensation scheduling solution, including: Collect vibration data, temperature data and current data of processing equipment to construct a multidimensional sensor data matrix, perform feature extraction on the multidimensional sensor data matrix to obtain spatiotemporal features, perform time series modeling through the multidimensional sensor data matrix based on the spatiotemporal features to obtain a device state feature sequence, obtain material inventory data and material consumption data in a material management system, perform weighted analysis to obtain a material supply forecast result, and generate a system abnormal situation vector based on the device state feature sequence and the material supply forecast result; The system abnormal situation vector uses a directed acyclic graph to describe the propagation path of the abnormal event, calculates the impact probability of the abnormal event on different production units according to the propagation path, compares the impact probability with a preset probability threshold to determine the affected area, and predicts the state evolution trajectory during the abnormal propagation process through probability graph reasoning based on the affected area; Based on the state evolution trajectory, historical compensation plans for similar scenarios are retrieved from the compensation strategy knowledge base, and the historical compensation plans are migrated to the current abnormal scenario using a meta-learning network. The meta-learning network includes a policy network and a value network. The policy network outputs the probability distribution of compensation actions, and the value network evaluates the long-term benefits of compensation actions. The candidate compensation plan with the highest benefit is selected as the final compensation scheduling plan.

[0011] Based on the state evolution trajectory, retrieving historical compensation schemes for similar scenarios from the compensation strategy knowledge base includes: Acquire state data of production equipment, the state data including equipment state parameters, process parameters and constraint condition parameters, and concatenate the state data on a feature dimension to form a dimensional feature vector; Perform time series sampling on the dimensional feature vector to obtain a sampling sequence, arrange the dimensional feature vectors in the sampling sequence in chronological order and splice them on the time dimension to form a time series feature matrix, the number of rows of the time series feature matrix corresponds to the dimension of the dimensional feature vector, the number of columns of the time series feature matrix corresponds to the number of sampling time points, and the time series characteristics of the device state evolution are characterized by the time series feature matrix; A time series decay function is constructed based on the time series feature matrix, wherein the time series decay function uses a negative exponential form to assign different weight coefficients to the dimensional feature vectors at different times, and the time series weighting of the historical state information is achieved through the weight coefficients; The similarity between the time series feature matrix and the historical time series feature matrix stored in the compensation strategy library is calculated, the feature similarity between the feature vectors of the same moment dimension is calculated, and the feature similarity is multiplied by the weight coefficient of the corresponding moment to obtain a trajectory similarity score; the historical compensation strategies in the compensation strategy library are sorted according to the trajectory similarity score.

[0012] Combining the compensation scheduling scheme with the output result of the multi-agent reinforcement learning model to generate a real-time scheduling instruction sequence, the real-time scheduling instruction sequence is decomposed to form specific control instructions for each processing equipment and sent to the corresponding processing equipment, thereby realizing adaptive scheduling control of the flexible production line, including: The difference between the compensation scheduling scheme and the scheduling scheme at the previous moment is squared to obtain a first modulus value, the absolute value of each item is taken after subtracting the multi-agent reinforcement learning model from the scheduling scheme at the previous moment item by item, the absolute value of each item is weightedly calculated to obtain a second modulus value, and the ratio of the first modulus value to the second modulus value is multiplied by a temperature parameter and then subjected to a sigmoid function to obtain a fusion scheduling vector; Constructing a scheduling instruction set based on the fused scheduling vector, calculating a priority score for each scheduling instruction in the scheduling instruction set, and sorting the scheduling instruction set according to the priority score to form a priority queue; Decomposing the scheduling instructions in the priority queue into atomic operation sequences, constructing a device capability matrix, wherein the number of rows of the device capability matrix is ​​the number of devices, the number of columns is the number of atomic operation types, the elements in the device capability matrix represent the capability values ​​of the corresponding devices to perform the corresponding atomic operations, and the mapping probability of each atomic operation to each device is calculated based on the device capability matrix; The atomic operation sequence is assigned to the corresponding device according to the mapping probability, and the atomic operations assigned to the same device are sorted according to the priority score to form a device control instruction sequence. Based on the device control instruction sequence, each device is controlled to perform the corresponding processing operation to realize adaptive scheduling control of the flexible production line.

[0013] According to a second aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0014] According to a third aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0015] The beneficial effects of this application are as follows: By mapping multiple processing equipment into intelligent agents and establishing a local communication network, information sharing and collaborative decision-making among intelligent agents are achieved, which effectively solves the computing bottleneck problem of traditional central control methods in complex production environments and improves decision-making efficiency and system response speed.

[0016] The experience replay pool is used to store system reward values ​​and to update the state-action value function, so that the intelligent agent can learn from historical experience and continuously optimize the decision-making strategy, thereby improving the scheduling quality and system stability and reducing production interruptions and resource waste.

[0017] During operation, the equipment status is monitored in real time and the state space and action space are dynamically updated. It can quickly respond to abnormal situations such as equipment failure or material shortage on the production line, and automatically generate compensation scheduling plans, which significantly enhances the anti-interference ability and adaptability of the production system and ensures the continuity and reliability of the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of a method for adaptive scheduling control of a flexible production line integrating multi-agent reinforcement learning according to an embodiment of the present invention; Figure 2 A schematic diagram of the dynamic communication network topology structure in the technical solution of the embodiment of the present invention; Figure 3 A schematic diagram comparing device resource utilization of different agent scheduling methods according to an embodiment of the present invention; Figure 4 This is a logic block diagram of the multi-agent reinforcement learning abnormal compensation scheduling embodiment of the present invention; Figure 5 This is a schematic diagram of the dynamic change process of the fusion scheduling vector in the technical solution of the embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0021] Figure 1 FIG. 1 is a flow chart of a method for adaptive scheduling control of a flexible production line integrating multi-agent reinforcement learning according to an embodiment of the present invention. Figure 1 As shown, the method includes: Acquire the real-time operating status of multiple processing equipment in the flexible production line and the processing requirement information of multiple workpieces, and construct a state space and an action space according to the real-time operating status and the processing requirement information; Mapping the multiple processing equipment to corresponding intelligent agents, establishing a local communication network between the intelligent agents to achieve information sharing between adjacent intelligent agents, and the intelligent agents selecting and executing scheduling actions based on current state information and local observation information of neighboring intelligent agents; After executing the scheduling action, the system instant reward value is obtained and stored in the experience replay pool, data is sampled from the experience replay pool to update the state-action value function, and an optimized multi-agent reinforcement learning model is obtained through repeated iterative training; The multi-agent reinforcement learning model continuously monitors the operating status of the multiple processing equipment during operation, and when equipment failure or material shortage is identified, dynamically updates the state space and action space of the affected area, and uses the updated state space and action space to recalculate the scheduling strategy to obtain a compensation scheduling solution; The compensation scheduling scheme is combined with the output results of the multi-agent reinforcement learning model to generate a real-time scheduling instruction sequence. The real-time scheduling instruction sequence is decomposed into specific control instructions for each processing equipment and issued to the corresponding processing equipment, thereby realizing adaptive scheduling control of the flexible production line.

[0022] In an optional implementation, constructing a state space and an action space according to the real-time operation state and the processing requirement information includes: The real-time operation status of the flexible production line includes the equipment status, task queue and remaining processing time, and the processing demand information includes the processing sequence and processing time of the workpiece; The real-time operating status and the processing requirement information are mapped to a feature vector space, the spatial features in the feature vector space are extracted through a deep neural network, and a state space and an action space are dynamically constructed according to the spatial features, wherein the state space represents the real-time state distribution of the processing equipment and the workpiece, and the action space represents an executable scheduling decision set.

[0023] The acquisition of real-time operation status is the basis of the system. The real-time operation status includes equipment status, task queue and remaining processing time. Equipment status refers to the current working status of each device, such as whether it is idle, processing or faulty. The task queue is a list of workpieces to be processed, containing the processing requirement information of each workpiece, such as processing sequence and processing time. The remaining processing time refers to the remaining processing time required for the current workpiece on the equipment. This information can be collected in real time through sensors and equipment management systems to form a dynamically updated status information set.

[0024] Mapping the real-time operation status and processing demand information to the feature vector space is a key step in realizing the construction of the state space and action space. The construction of the feature vector space requires converting information such as the device status, task queue, and remaining processing time into a unified feature representation. Specifically, the status of each device can be encoded, for example, the idle state is encoded as 0, the processing is encoded as 1, and the fault state is encoded as 2. At the same time, each workpiece in the task queue can be characterized by its processing sequence and processing time. Ultimately, all this information will be integrated into a feature vector to form a multidimensional feature space.

[0025] The spatial features in the feature vector space are extracted through a deep neural network. The design of a deep neural network should include multiple layers of neurons so that the relationship between complex states can be captured. During the training process, the input feature vector will be processed through each layer of the network, and high-level feature representations will be gradually extracted. These feature representations can effectively reflect the real-time state distribution between the equipment and the workpiece, and provide a basis for the subsequent construction of the state space and action space.

[0026] Dynamically construct state space and action space. The construction of the state space should take into account the real-time state distribution of equipment and workpieces. Specifically, the state space can be divided into multiple subspaces, each of which corresponds to a specific combination of equipment state and workpiece state. For example, the state space can be divided into multiple state combinations such as "equipment is idle and there are workpieces to be processed", "equipment is busy and there are no workpieces to be processed", etc. Such a division can not only clearly characterize the current operating state of the system, but also provide a clear basis for subsequent scheduling decisions.

[0027] The action space is a set of scheduling decisions that can be executed based on the current state. Each scheduling decision should correspond to a specific scheduling action, such as assigning a workpiece to a specific device for processing. The construction of the action space needs to be combined with the current state information to ensure that the generated scheduling decisions are feasible. For example, when the device is idle and has workpieces to be processed, you can choose to assign the first workpiece in the queue to that device for processing. In this way, the action space can dynamically adapt to changes in real-time operating status.

[0028] In order to verify the effectiveness of the above method, we can test it through specific data cases. For example, suppose there is a flexible production line with three devices and five workpieces. The processing time and sequence of each workpiece are known, and the status of the equipment is fed back in real time through sensors. During the operation of the system, the real-time status information is first obtained to form a feature vector. Then, the features are extracted through a deep neural network to construct the state space and action space. Finally, the system selects the optimal scheduling decision based on the current state to complete the processing of the workpiece.

[0029] In an optional implementation, the plurality of processing equipment are mapped to corresponding intelligent agents, a local communication network is established between the intelligent agents to realize information sharing between adjacent intelligent agents, and the intelligent agents select scheduling actions and execute them according to current state information and local observation information of neighboring intelligent agents, including: The plurality of processing equipment in the flexible production line are mapped into corresponding intelligent agents, and each of the intelligent agents extracts features of its own state information based on an attention mechanism to obtain a state feature vector reflecting the real-time working characteristics of the processing equipment; Constructing a dynamic topology intelligent agent communication network, modeling the degree of association between the intelligent agents based on the intelligent agent communication network, determining the communication weight by calculating the state similarity and task relevance between the intelligent agents, and adaptively adjusting the communication weight as the state of the intelligent agent and the task allocation change, so as to realize the interaction of differentiated information between the intelligent agents; A hybrid strategy network is constructed based on the state feature vector and the differential information, wherein the hybrid strategy network combines deterministic strategy with random exploration, outputs deterministic action distribution through the hybrid strategy network, and introduces random noise to realize strategy exploration, wherein the training goal of the hybrid strategy network is jointly determined by task completion and collaborative efficiency; A scheduling action is selected according to the output result of the hybrid strategy network, and state transfer information and reward information after executing the scheduling action are stored in the experience pool.

[0030] Multiple processing equipment are mapped to corresponding agents. Each agent is responsible for monitoring and managing its corresponding processing equipment. The agent obtains its own status information through sensors and data acquisition modules, including the equipment's operating status, load conditions, fault information, etc. In order to extract the features of this status information, the agent adopts an attention mechanism. Specifically, the agent performs weighted processing on the collected status information, highlights important features, and thus generates a state feature vector. This vector can reflect the real-time working characteristics of the equipment, such as the equipment's processing speed, energy consumption, and failure rate.

[0031] A dynamic topology agent communication network is constructed. Each agent establishes communication connections with its neighboring agents to achieve information sharing. The degree of association between agents is quantified by building a model that takes into account the state similarity and task relevance between agents. State similarity is evaluated by comparing the state feature vectors of the agents, while task relevance is evaluated based on the type and priority of the tasks undertaken by the agents. Communication weights are set based on the calculation results of these similarities and relevances, and are adaptively adjusted as the agent states and task assignments change. This dynamic adjustment mechanism ensures that agents can optimize information exchange according to actual conditions and promote collaborative work.

[0032] A hybrid strategy network is constructed using state feature vectors and differentiation information. The network combines deterministic strategies and random exploration mechanisms to improve the flexibility and effectiveness of scheduling decisions. Specifically, the hybrid strategy network first generates a deterministic action distribution based on the current state feature vector to guide the agent to select the optimal scheduling action. At the same time, in order to avoid falling into a local optimal solution, the hybrid strategy network also introduces random noise to encourage the agent to explore other possible scheduling strategies. The training goal of the hybrid strategy network is to optimize based on task completion and collaborative efficiency to ensure that the agent can achieve the best production efficiency when performing tasks.

[0033] Select a specific scheduling action based on the output of the hybrid strategy network. For example, an agent may choose to adjust the operating speed of a processing device or reallocate tasks to other devices. After executing the scheduling action, the agent will record the state transition information and reward information and store it in the experience pool. This information will be used for subsequent learning and optimization to help the agent continuously improve the accuracy and efficiency of scheduling decisions.

[0034] In order to verify the effectiveness of the above technical means, a specific data case can be considered. In a flexible production line, there are five processing equipment, each of which is managed by an agent. Each agent collects real-time data of the equipment through sensors, including processing speed, fault information, etc., and generates a state feature vector. After processing, the communication network between agents is established, and agent A is connected to agents B and C, and agent D is connected to agent E.

[0035] Assume that the state feature vector of agent A shows that its equipment load is too high, while the equipment load of agent B is low. At this time, agent A will request assistance from agent B through the communication network. Agent B decides to transfer part of the task to agent A based on its state feature vector and task relevance. At this time, the dynamic adjustment of communication weights ensures the timely transmission of information and the effective allocation of tasks.

[0036] The hybrid strategy network generates a deterministic action distribution based on the state feature vector, and Agent A finally chooses to reduce the equipment speed by 10%. After execution, Agent A records the state transition information and the rewards obtained (such as increased production efficiency) and stores them in the experience pool for subsequent learning and optimization.

[0037] Figure 2 This is a schematic diagram of the dynamic communication network topology structure in the technical solution of the embodiment of the present invention: This figure shows the state and communication relationship diagram of a collaborative network of processing equipment agents. The figure contains 6 processing equipment agents (AF), and the operating status and load rate of each agent node are marked. Among them, equipment A, B, D and E are in the "operating" state, with load rates of 87%, 92%, 78% and 65% respectively; equipment C is in the "standby" state with a load rate of 23%; equipment F is in the "fault repair" state with a load rate of 0%. The connecting lines between agents represent the communication weights, which are divided into three categories according to the weight size: high communication weight (>0.8) is represented by solid lines, including 0.92 between AB, 0.88 between BD and 0.84 between DE; medium communication weight (0.4-0.8) is represented by dotted lines, including 0.65 between AC, 0.71 between BE and 0.54 between EF; low communication weight (<0.4) is represented by dotted lines, including 0.35 between CE and 0.26 between CF. This network structure reflects the strength of the collaborative relationship between devices. High-load operating devices maintain strong communication connections, while the communication strength between faulty or standby devices and other devices is relatively weak.

[0038] Figure 3 This is a schematic diagram comparing device resource utilization rates of different agent scheduling methods according to an embodiment of the present invention: This figure shows the comparison results of equipment utilization of three different methods on five processing equipment (AE) and their average values. The figure adopts the form of a line graph, with the horizontal axis representing different processing equipment and the vertical axis representing the percentage of equipment utilization. This technical solution performs best, and its utilization curve is always at the top. The utilization rates from equipment A to equipment E are 84%, 80%, 87%, 89% and 82% respectively, and the average utilization rate reaches 84.4%; the traditional multi-agent method is second, with the corresponding utilization data of 70%, 72%, 68%, 65% and 69%, and the average utilization rate is 68.8%; the rule-based method performs the worst, with the utilization rates of each equipment being 55%, 60%, 52%, 47% and 57% respectively, and the average utilization rate is only 54.2%. From the data trend, this technical solution maintains a high and stable utilization rate on all equipment, with the highest reaching 89% and the lowest being 80%, reflecting a significant performance advantage; while the other two methods not only have a low overall utilization level, but also have large fluctuations, especially the utilization rate of the rule-based method on equipment D drops to a low of 47%.

[0039] In an optional implementation, constructing a dynamic topology agent communication network, modeling the degree of association between the agents based on the agent communication network, and determining the communication weight by calculating the state similarity and task relevance between the agents includes: Constructing multiple agents into a directed graph network structure, wherein nodes in the directed graph network structure represent agents, and edges in the directed graph network structure represent communication links, determining a corresponding neighborhood node set for each of the agents, obtaining a device operating parameter vector, a task queue state vector, and a resource state vector of each of the agents, and combining the device operating parameter vector, the task queue state vector, and the resource state vector to form a state vector of the agent; Extract features from the state vector to obtain a feature vector, calculate the inner product of the feature vectors corresponding to any two of the agents, divide the inner product by the square root of the feature dimension, and then pass the resultant softmax activation function to obtain the connection strength; The state similarity is obtained by calculating the dot product of any two state vectors divided by the product of the respective vector moduli, the cosine similarity between any two agents is calculated and used as the task relevance, and the state similarity and the task relevance are weighted to obtain the communication weight.

[0040] In constructing a dynamic topology agent communication network, we first need to construct multiple agents into a directed graph network structure. In this structure, each node represents an agent, and the edge represents the communication link between agents. To achieve this goal, we first need to determine the set of neighborhood nodes of each agent, that is, the agents directly connected to it.

[0041] For each agent, its device working parameter vector, task queue state vector, and resource state vector need to be obtained. These vectors represent the working state of the agent, the current tasks to be processed, and the available resources. These three vectors are combined to form a comprehensive state vector for subsequent analysis.

[0042] In the feature extraction process of the state vector, principal component analysis or other dimensionality reduction techniques can be used to extract key features and form feature vectors. The generation of feature vectors is to better capture the behavior patterns and state characteristics of the intelligent agent.

[0043] For any two agents, the inner product of their corresponding feature vectors is calculated and the result is divided by the square root of the feature dimension. This process can be understood as measuring the similarity between the two agents in the feature space. By performing softmax activation processing on the inner product result, the connection strength can be obtained, which reflects the potential communication ability between the agents.

[0044] In the calculation of state similarity, we first need to perform a dot product operation on the state vectors of any two agents. Then, we divide the dot product result by the product of the modulus lengths of their respective vectors to measure the similarity between them. The level of state similarity directly affects the efficiency of information exchange between agents.

[0045] Calculating the cosine similarity between any two agents can be seen as another way to measure the relevance of tasks between agents. The higher the cosine similarity value, the stronger the relevance of the two agents in task execution.

[0046] The calculated state similarity and task relevance are weighted to obtain the final communication weight. This weight will be used to guide communication decisions between agents to ensure effective information transmission and reasonable allocation of resources.

[0047] Suppose there are three agents A, B, and C. Agent A's device operating parameter vector is [0.8, 0.6, 0.7], the task queue state vector is [1, 0, 1], and the resource state vector is [0.9, 0.5]. The corresponding vectors of agent B are [0.7, 0.5, 0.6], [0, 1, 0], and [0.8, 0.4], and the vectors of agent C are [0.9, 0.7, 0.8], [1, 1, 0], and [0.6, 0.5].

[0048] By combining these vectors, the state vector of agent A is [0.8, 0.6, 0.7, 1, 0, 1, 0.9,0.5], the state vector of agent B is [0.7, 0.5, 0.6, 0, 1, 0, 0.8, 0.4], and the state vector of agent C is [0.9, 0.7, 0.8, 1, 1, 0, 0.6, 0.5].

[0049] After feature extraction, assume that the feature vectors obtained are A', B', and C'. Calculate the inner product of A' and B', and the result is X, then divide X by the square root of the feature dimension to get the connection strength. Perform a dot product operation on the state vectors A and B to get Y, and then divide Y by the product of the modulus lengths of each vector to get the state similarity. Next, calculate the cosine similarity between A and B, and finally weight the state similarity with the task relevance to get the communication weight.

[0050] In an optional implementation, the multi-agent reinforcement learning model continuously monitors the operating status of the plurality of processing equipment during operation, and when equipment failure or material shortage is identified, dynamically updates the state space and action space of the affected area, and uses the updated state space and action space to recalculate the scheduling strategy to obtain a compensation scheduling solution, including: Collect vibration data, temperature data and current data of processing equipment to construct a multidimensional sensor data matrix, perform feature extraction on the multidimensional sensor data matrix to obtain spatiotemporal features, perform time series modeling through the multidimensional sensor data matrix based on the spatiotemporal features to obtain a device state feature sequence, obtain material inventory data and material consumption data in a material management system, perform weighted analysis to obtain a material supply forecast result, and generate a system abnormal situation vector based on the device state feature sequence and the material supply forecast result; The system abnormal situation vector uses a directed acyclic graph to describe the propagation path of the abnormal event, calculates the impact probability of the abnormal event on different production units according to the propagation path, compares the impact probability with a preset probability threshold to determine the affected area, and predicts the state evolution trajectory during the abnormal propagation process through probability graph reasoning based on the affected area; Based on the state evolution trajectory, historical compensation plans for similar scenarios are retrieved from the compensation strategy knowledge base, and the historical compensation plans are migrated to the current abnormal scenario using a meta-learning network. The meta-learning network includes a policy network and a value network. The policy network outputs the probability distribution of compensation actions, and the value network evaluates the long-term benefits of compensation actions. The candidate compensation plan with the highest benefit is selected as the final compensation scheduling plan.

[0051] Through sensors installed on multiple processing equipment, the vibration data, temperature data and current data of the equipment are collected in real time. These data are integrated into a multi-dimensional sensor data matrix, and each dimension of the matrix represents a type of sensor data. Next, the multi-dimensional sensor data matrix is ​​processed using feature extraction technology to extract spatiotemporal features. These spatiotemporal features reflect the operating status of the equipment in different time periods and can effectively identify the normal and abnormal status of the equipment.

[0052] After the spatiotemporal feature extraction is completed, the system uses time series modeling technology to further analyze the multidimensional sensor data matrix and generate a device state feature sequence. This sequence contains the state change information of the equipment within a certain period of time, which can provide basic data for subsequent fault detection and material supply prediction.

[0053] Data is exchanged with the material management system to obtain material inventory data and material consumption data. By weighting these data, the system can generate material supply forecast results. This result provides the necessary material supply background information for subsequent abnormal situation analysis.

[0054] Based on the equipment status feature sequence and material supply forecast results, the system generates a system abnormal situation vector, which is used to describe the abnormal state of the current system and can reflect the severity of problems such as equipment failure and material shortage.

[0055] After the abnormal situation vector is generated, the system uses a directed acyclic graph (DAG) model to describe the propagation path of abnormal events. The model clarifies the possible propagation path of abnormal events in the production process through the relationship between nodes and edges. The system calculates the probability of the impact of abnormal events on different production units based on this propagation path. The calculation of the impact probability is based on a comprehensive analysis of equipment status characteristics and material supply forecasts.

[0056] The calculated impact probability is compared with the preset probability threshold to determine the affected area. The affected area refers to those production units that are greatly affected by the abnormal event, and these areas need to be prioritized for scheduling optimization.

[0057] Based on the affected area, the system predicts the state evolution trajectory during the abnormal propagation process through probabilistic graph reasoning technology. This trajectory reflects the changing trend of the state of each production unit in the affected area after the abnormal event occurs, providing a basis for formulating compensation scheduling plans.

[0058] After obtaining the state evolution trajectory, the system retrieves historical compensation plans similar to the current abnormal scenario from the compensation strategy knowledge base. Historical compensation plans refer to the scheduling optimization measures taken for similar abnormal situations in the past production process. The system uses the meta-learning network to transfer learning of these historical compensation plans to adapt to the current abnormal scenario. The meta-learning network consists of a policy network and a value network. The policy network is responsible for outputting the probability distribution of compensation actions, while the value network evaluates the long-term benefits of compensation actions.

[0059] During the compensation plan generation process, the system will evaluate all candidate compensation plans and select the plan with the highest long-term benefits as the final compensation scheduling plan. This plan can effectively alleviate production problems caused by equipment failure or material shortages and ensure the continuity and efficiency of the production process.

[0060] Suppose a manufacturing company's processing equipment experiences abnormal vibration during operation, and the vibration data collected by the sensor in real time is significantly higher than the normal range. Through feature extraction and time series modeling, the system determines that the equipment may be faulty. At the same time, the material management system shows that the inventory of raw materials required for the equipment is insufficient to meet production needs. The abnormal situation vector generated by the system shows that equipment failure and material shortage exist at the same time, and affect multiple production units.

[0061] Through the directed acyclic graph model, the system identifies the propagation path of abnormal events and calculates the probability of different production units being affected. Assuming that the probability of a certain production unit being affected is higher than the preset threshold, the system marks it as an affected area. Subsequently, the system predicts the state evolution trajectory in the area and finds that if no measures are taken, production efficiency will drop by 30%.

[0062] When searching for historical compensation plans, the system found similar equipment failures and material shortages in the past. The relevant compensation plans included adjusting production plans, temporarily increasing material purchases, and optimizing equipment maintenance time. Through the meta-learning network, the system migrated these historical plans to the current scenario, evaluated them, and selected a plan to adjust the production plan, which is expected to restore production efficiency to normal levels.

[0063] Figure 4 This is a logic block diagram of abnormal compensation scheduling for multi-agent reinforcement learning according to an embodiment of the present invention: This figure shows a complete system workflow diagram, which mainly includes two parallel data collection and processing branches, as well as subsequent analysis and decision-making processes. The left branch starts with equipment sensor data collection, including parameters such as vibration, temperature and current. After multi-dimensional sensor data matrix feature extraction, time series modeling is performed to obtain the equipment state feature sequence. The right branch starts with material data collection, including inventory and consumption data, and material supply forecasting is performed through weighted material data processing. The outputs of these two branches are jointly fed into the system abnormal situation vector generation link. After that, the process is divided into three parallel paths: the first is to analyze the abnormal propagation path through directed acyclic graph modeling; the second is to calculate the impact probability to determine the change impact area, and output the probability graph reasoning result for state evolution trajectory prediction; the third is connected to the compensation strategy knowledge base for historical compensation plan retrieval, and the compensation action probability distribution calculation is performed through the meta-learning network (including the strategy network and the value network). After compensation action evaluation and long-term benefit calculation, the optimal compensation scheduling plan is finally generated and executed. This flowchart reflects the complete closed-loop control process from data collection, feature extraction, state modeling to decision execution, and realizes the intelligent operation and management of the system through multi-dimensional data fusion and multi-level analysis.

[0064] In an optional implementation, based on the state evolution trajectory, retrieving historical compensation schemes for similar scenarios from a compensation strategy knowledge base includes: Acquire state data of production equipment, the state data including equipment state parameters, process parameters and constraint condition parameters, and concatenate the state data on a feature dimension to form a dimensional feature vector; Perform time series sampling on the dimensional feature vector to obtain a sampling sequence, arrange the dimensional feature vectors in the sampling sequence in chronological order and splice them on the time dimension to form a time series feature matrix, the number of rows of the time series feature matrix corresponds to the dimension of the dimensional feature vector, the number of columns of the time series feature matrix corresponds to the number of sampling time points, and the time series characteristics of the device state evolution are characterized by the time series feature matrix; A time series decay function is constructed based on the time series feature matrix, wherein the time series decay function uses a negative exponential form to assign different weight coefficients to the dimensional feature vectors at different times, and the time series weighting of the historical state information is achieved through the weight coefficients; The similarity between the time series feature matrix and the historical time series feature matrix stored in the compensation strategy library is calculated, the feature similarity between the feature vectors of the same moment dimension is calculated, and the feature similarity is multiplied by the weight coefficient of the corresponding moment to obtain a trajectory similarity score; the historical compensation strategies in the compensation strategy library are sorted according to the trajectory similarity score.

[0065] In modern manufacturing, the state monitoring of production equipment and the optimization of compensation strategies are the key to improving production efficiency and reducing failure rates. The following is a specific implementation method for retrieving historical compensation schemes for similar scenarios from the compensation strategy knowledge base based on the state evolution trajectory.

[0066] Collect the status data of production equipment. The status data includes equipment status parameters (such as temperature, pressure, speed, etc.), process parameters (such as feed rate, processing time, etc.) and constraint parameters (such as maximum load, ambient temperature, etc.). These data are collected in real time through sensors and stored in the database.

[0067] Process the collected status data. First, different types of status data need to be spliced ​​in the feature dimension to form a dimensional feature vector. Each element of the feature vector represents a specific state parameter, and the spliced ​​feature vector can fully reflect the current state of the device. For example, assuming that the temperature of a device is 70°C, the pressure is 1.2MPa, and the speed is 1500rpm, the corresponding feature vector can be expressed as [70, 1.2, 1500].

[0068] After obtaining the dimensional feature vector, time series sampling is performed. The process of time series sampling is to arrange the feature vectors in chronological order to form a time series feature matrix. The number of rows of this matrix corresponds to the dimension of the feature vector, and the number of columns corresponds to the number of sampling time points. The time series feature matrix can clearly show the evolution of the device status over time. For example, suppose the feature vectors collected at 5 different time points are [70, 1.2, 1500], [72, 1.3, 1480], [71, 1.25,1490], [73, 1.4, 1470], [75, 1.5, 1460].

[0069] Construct a time series decay function. The purpose of the time series decay function is to assign different weight coefficients to the dimensional feature vectors at different times, so as to better reflect the impact of historical state information on the current state. Specifically, a closer time point will be assigned a higher weight, while a farther time point will be assigned a lower weight. This weight distribution can be achieved by setting a decay factor, so that the farther the state is, the less impact it has on the current decision.

[0070] After completing the construction of the time series decay function, perform similarity calculation. Compare the current time series feature matrix with the historical time series feature matrix stored in the compensation strategy library. The similarity calculation process is to compare the dimensional feature vectors at the same time and evaluate the feature similarity between them. This process can be achieved by calculating the distance or similarity index between feature vectors. The calculated feature similarity needs to be multiplied by the weight coefficient of the corresponding time to obtain the trajectory similarity score.

[0071] The historical compensation strategies in the compensation strategy library are sorted according to the trajectory similarity score. Strategies with higher scores are more similar to the current device state and are therefore more likely to be effective in the current situation. In this way, the most suitable strategy for the current device state can be quickly screened out from the historical compensation strategies, thereby achieving efficient compensation decisions.

[0072] In an optional implementation, the compensation scheduling scheme is combined with the output result of the multi-agent reinforcement learning model to generate a real-time scheduling instruction sequence, and the real-time scheduling instruction sequence is decomposed to form specific control instructions for each processing equipment and sent to the corresponding processing equipment, thereby realizing adaptive scheduling control of the flexible production line, including: The difference between the compensation scheduling scheme and the scheduling scheme at the previous moment is squared to obtain a first modulus value, the absolute value of each item is taken after subtracting the multi-agent reinforcement learning model from the scheduling scheme at the previous moment item by item, the absolute value of each item is weightedly calculated to obtain a second modulus value, and the ratio of the first modulus value to the second modulus value is multiplied by a temperature parameter and then subjected to a sigmoid function to obtain a fusion scheduling vector; Constructing a scheduling instruction set based on the fused scheduling vector, calculating a priority score for each scheduling instruction in the scheduling instruction set, and sorting the scheduling instruction set according to the priority score to form a priority queue; Decomposing the scheduling instructions in the priority queue into atomic operation sequences, constructing a device capability matrix, wherein the number of rows of the device capability matrix is ​​the number of devices, the number of columns is the number of atomic operation types, the elements in the device capability matrix represent the capability values ​​of the corresponding devices to perform the corresponding atomic operations, and the mapping probability of each atomic operation to each device is calculated based on the device capability matrix; The atomic operation sequence is assigned to the corresponding device according to the mapping probability, and the atomic operations assigned to the same device are sorted according to the priority score to form a device control instruction sequence. Based on the device control instruction sequence, each device is controlled to perform the corresponding processing operation to realize adaptive scheduling control of the flexible production line.

[0073] Get the current compensation scheduling plan and the previous scheduling plan. Square the difference between the two to get the first modulus. This modulus reflects the degree of change between the current scheduling plan and the previous plan, and provides basic data for adjusting the scheduling strategy.

[0074] Get the output of the multi-agent reinforcement learning model and subtract it from the previous scheduling plan item by item to calculate the absolute value of each item. These absolute values ​​represent the deviation of the current scheduling plan from the previous plan in various aspects. Perform a weighted calculation on these absolute values ​​to obtain the second modulus, which is used to measure the stability and reliability of the current scheduling plan relative to the historical plan.

[0075] The ratio of the first modulus to the second modulus is calculated and multiplied by a temperature parameter, which is then processed by the sigmoid function to generate a fusion scheduling vector. This vector combines the output results of the compensation scheduling scheme and the multi-agent reinforcement learning model, providing a balanced scheduling decision basis.

[0076] Based on the fused scheduling vector, a scheduling instruction set is constructed. A priority score is calculated for each instruction in the scheduling instruction set. The priority score can be based on a variety of factors, such as task urgency, resource availability, device status, etc. By sorting the priority scores, a priority queue is formed to ensure that high-priority scheduling instructions can be executed first.

[0077] Decompose the scheduling instructions in the priority queue into a sequence of atomic operations. Each atomic operation represents a basic operation step in the production process. Construct a device capability matrix, where the number of rows corresponds to the number of devices and the number of columns corresponds to the number of atomic operation types. Each element in the matrix represents the capability value of the corresponding device to perform a specific atomic operation. By evaluating the device capabilities, the efficiency and adaptability of each device in performing different operations can be determined.

[0078] After constructing the device capability matrix, the mapping probability of each atomic operation to each device is calculated. The mapping probability reflects the possibility of a specific atomic operation being executed on a specific device, taking into account factors such as the current load of the device, historical execution efficiency, and health status of the device.

[0079] According to the calculated mapping probability, the atomic operation sequence is assigned to the corresponding device. For the atomic operations assigned to the same device, they are sorted according to the priority score to form a device control instruction sequence. This sequence clearly indicates the order of operations that each device needs to perform, thereby achieving efficient production scheduling.

[0080] Based on the equipment control instruction sequence, each device is controlled to perform the corresponding processing operation. By real-time monitoring of the execution status of the equipment, the flexibility and adaptability of the production line are ensured. The entire scheduling control process can dynamically respond to changes in the production environment, optimize resource allocation, and improve production efficiency.

[0081] Assume that a flexible production line has three devices, namely device A, device B and device C, which can perform five atomic operations. Assume that the current compensation scheduling scheme is [2, 3, 1] and the previous scheduling scheme is [1, 2, 2], then the first modulus is (2-1)² + (3-2)² + (1-2)² = 1 + 1 + 1 = 3. Assume that the output of the multi-agent reinforcement learning model is [3, 1, 2], then the second modulus is |3-1| + |1-2| + |2-2| = 2 + 1 + 0 = 3. The fused scheduling vector obtained after processing will be a decision-making basis that comprehensively considers current and historical data.

[0082] Figure 5 This is a schematic diagram of the dynamic change process of the fusion scheduling vector in the technical solution of the embodiment of the present invention: The figure shows the dynamic changes of the three indicators of the system, namely, the compensation strategy weight, the reinforcement learning weight and the temperature parameter, within a time range of 90 seconds, including two key time points: the fault occurred at 20 seconds and the fault was resolved at 50 seconds. Specifically, the temperature parameter (diamond mark) remained at around 0.75 before the fault occurred, rose rapidly to 0.98 after the fault occurred, then slowly decreased, and continued to drop to 0.75 after the fault was resolved; the compensation strategy weight (circle mark) was maintained at 0.40 before the fault occurred, and rose significantly to 0.90 after the fault occurred, but gradually dropped to 0.65 during the fault period, continued to drop after the fault was resolved and finally stabilized at around 0.40; the reinforcement learning weight (square mark) remained at a high level of 0.85 before the fault occurred, dropped to 0.35 briefly after the fault occurred, then gradually rose to 0.65 during the fault period, continued to rise after the fault was resolved and finally stabilized at around 0.85. The changing trends of these three curves clearly reflect the dynamic adjustment strategies of various control parameters of the system during the occurrence, processing and recovery of faults, and reflect the adaptive adjustment ability of the system.

[0083] According to a second aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0084] According to a third aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0085] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. The adaptive scheduling control method for flexible production lines integrating multi-agent reinforcement learning is characterized by: include: Acquire the real-time operating status of multiple processing equipment in the flexible production line and the processing requirement information of multiple workpieces, and construct a state space and an action space according to the real-time operating status and the processing requirement information; Mapping the multiple processing equipment to corresponding intelligent agents, establishing a local communication network between the intelligent agents to achieve information sharing between adjacent intelligent agents, and the intelligent agents selecting and executing scheduling actions based on current state information and local observation information of neighboring intelligent agents; After executing the scheduling action, the system instant reward value is obtained and stored in the experience replay pool, data is sampled from the experience replay pool to update the state-action value function, and an optimized multi-agent reinforcement learning model is obtained through repeated iterative training; The multi-agent reinforcement learning model continuously monitors the operating status of the multiple processing equipment during operation, and when equipment failure or material shortage is identified, dynamically updates the state space and action space of the affected area, and uses the updated state space and action space to recalculate the scheduling strategy to obtain a compensation scheduling solution; The compensation scheduling scheme is combined with the output results of the multi-agent reinforcement learning model to generate a real-time scheduling instruction sequence. The real-time scheduling instruction sequence is decomposed into specific control instructions for each processing equipment and issued to the corresponding processing equipment, thereby realizing adaptive scheduling control of the flexible production line.

2. The method according to claim 1, characterized in that Constructing the state space and the action space according to the real-time operation state and the processing requirement information includes: The real-time operation status of the flexible production line includes the equipment status, task queue and remaining processing time, and the processing demand information includes the processing sequence and processing time of the workpiece; The real-time operating status and the processing requirement information are mapped to a feature vector space, the spatial features in the feature vector space are extracted through a deep neural network, and a state space and an action space are dynamically constructed according to the spatial features, wherein the state space represents the real-time state distribution of the processing equipment and the workpiece, and the action space represents an executable scheduling decision set.

3. The method according to claim 1, characterized in that The plurality of processing equipment are mapped to corresponding agents, a local communication network is established between the agents to realize information sharing between adjacent agents, and the agents select scheduling actions and execute them according to current state information and local observation information of neighboring agents, including: The plurality of processing equipment in the flexible production line are mapped into corresponding intelligent agents, and each of the intelligent agents extracts features of its own state information based on an attention mechanism to obtain a state feature vector reflecting the real-time working characteristics of the processing equipment; Constructing a dynamic topology intelligent agent communication network, modeling the degree of association between the intelligent agents based on the intelligent agent communication network, determining the communication weight by calculating the state similarity and task relevance between the intelligent agents, and adaptively adjusting the communication weight as the state of the intelligent agent and the task allocation change, so as to realize the interaction of differentiated information between the intelligent agents; A hybrid strategy network is constructed based on the state feature vector and the differential information, wherein the hybrid strategy network combines deterministic strategy with random exploration, outputs deterministic action distribution through the hybrid strategy network, and introduces random noise to realize strategy exploration, wherein the training goal of the hybrid strategy network is jointly determined by task completion and collaborative efficiency; A scheduling action is selected according to the output result of the hybrid strategy network, and state transfer information and reward information after executing the scheduling action are stored in the experience pool.

4. The method according to claim 3, characterized in that Constructing a dynamic topology agent communication network, modeling the degree of association between the agents based on the agent communication network, and determining the communication weight by calculating the state similarity and task relevance between the agents includes: Constructing multiple agents into a directed graph network structure, wherein nodes in the directed graph network structure represent agents, and edges in the directed graph network structure represent communication links, determining a corresponding neighborhood node set for each of the agents, obtaining a device operating parameter vector, a task queue state vector, and a resource state vector of each of the agents, and combining the device operating parameter vector, the task queue state vector, and the resource state vector to form a state vector of the agent; Extract features from the state vector to obtain a feature vector, calculate the inner product of the feature vectors corresponding to any two of the agents, divide the inner product by the square root of the feature dimension, and then pass the resultant softmax activation function to obtain the connection strength; The state similarity is obtained by calculating the dot product of any two state vectors divided by the product of the respective vector moduli, the cosine similarity between any two agents is calculated and used as the task relevance, and the state similarity and the task relevance are weighted to obtain the communication weight.

5. The method according to claim 1, characterized in that The multi-agent reinforcement learning model continuously monitors the operating status of the multiple processing equipment during operation. When equipment failure or material shortage is identified, the state space and action space of the affected area are dynamically updated, and the scheduling strategy is recalculated using the updated state space and action space to obtain a compensation scheduling solution, including: Collect vibration data, temperature data and current data of processing equipment to construct a multidimensional sensor data matrix, perform feature extraction on the multidimensional sensor data matrix to obtain spatiotemporal features, perform time series modeling through the multidimensional sensor data matrix based on the spatiotemporal features to obtain a device state feature sequence, obtain material inventory data and material consumption data in a material management system, perform weighted analysis to obtain a material supply forecast result, and generate a system abnormal situation vector based on the device state feature sequence and the material supply forecast result; The system abnormal situation vector uses a directed acyclic graph to describe the propagation path of the abnormal event, calculates the impact probability of the abnormal event on different production units according to the propagation path, compares the impact probability with a preset probability threshold to determine the affected area, and predicts the state evolution trajectory during the abnormal propagation process through probability graph reasoning based on the affected area; Based on the state evolution trajectory, historical compensation plans for similar scenarios are retrieved from the compensation strategy knowledge base, and the historical compensation plans are migrated to the current abnormal scenario using a meta-learning network. The meta-learning network includes a policy network and a value network. The policy network outputs the probability distribution of compensation actions, and the value network evaluates the long-term benefits of compensation actions. The candidate compensation plan with the highest benefit is selected as the final compensation scheduling plan.

6. The method according to claim 5, characterized in that Based on the state evolution trajectory, retrieving historical compensation schemes for similar scenarios from the compensation strategy knowledge base includes: Acquire state data of production equipment, the state data including equipment state parameters, process parameters and constraint condition parameters, and concatenate the state data on a feature dimension to form a dimensional feature vector; Perform time series sampling on the dimensional feature vector to obtain a sampling sequence, arrange the dimensional feature vectors in the sampling sequence in chronological order and splice them on the time dimension to form a time series feature matrix, the number of rows of the time series feature matrix corresponds to the dimension of the dimensional feature vector, the number of columns of the time series feature matrix corresponds to the number of sampling time points, and the time series characteristics of the device state evolution are characterized by the time series feature matrix; A time series decay function is constructed based on the time series feature matrix, wherein the time series decay function uses a negative exponential form to assign different weight coefficients to the dimensional feature vectors at different times, and the time series weighting of the historical state information is achieved through the weight coefficients; The similarity between the time series feature matrix and the historical time series feature matrix stored in the compensation strategy library is calculated, the feature similarity between the feature vectors of the same moment dimension is calculated, and the feature similarity is multiplied by the weight coefficient of the corresponding moment to obtain a trajectory similarity score; the historical compensation strategies in the compensation strategy library are sorted according to the trajectory similarity score.

7. The method according to claim 1, characterized in that Combining the compensation scheduling scheme with the output result of the multi-agent reinforcement learning model to generate a real-time scheduling instruction sequence, the real-time scheduling instruction sequence is decomposed to form specific control instructions for each processing equipment and sent to the corresponding processing equipment, thereby realizing adaptive scheduling control of the flexible production line, including: The difference between the compensation scheduling scheme and the scheduling scheme at the previous moment is squared to obtain a first modulus value, the absolute value of each item is taken after subtracting the multi-agent reinforcement learning model from the scheduling scheme at the previous moment item by item, the absolute value of each item is weightedly calculated to obtain a second modulus value, and the ratio of the first modulus value to the second modulus value is multiplied by a temperature parameter and then subjected to a sigmoid function to obtain a fusion scheduling vector; Constructing a scheduling instruction set based on the fused scheduling vector, calculating a priority score for each scheduling instruction in the scheduling instruction set, and sorting the scheduling instruction set according to the priority score to form a priority queue; Decomposing the scheduling instructions in the priority queue into atomic operation sequences, constructing a device capability matrix, wherein the number of rows of the device capability matrix is ​​the number of devices, the number of columns is the number of atomic operation types, the elements in the device capability matrix represent the capability values ​​of the corresponding devices to perform the corresponding atomic operations, and the mapping probability of each atomic operation to each device is calculated based on the device capability matrix; The atomic operation sequence is assigned to the corresponding device according to the mapping probability, and the atomic operations assigned to the same device are sorted according to the priority score to form a device control instruction sequence. Based on the device control instruction sequence, each device is controlled to perform the corresponding processing operation to realize adaptive scheduling control of the flexible production line.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images