Workshop scheduling optimization method, system and equipment for multi-agent collaborative scheduling and storage medium

Abstract

The invention provides a multi-agent collaborative scheduling workshop scheduling optimization method, system and device, and a storage medium, and the method comprises the steps: obtaining the full-process data of workshop production, integrating the real-time states and constraint relationships of materials, devices, processes and orders based on the full-process data, and constructing a shared state space; constructing and training a material distribution agent and a process scheduling agent based on the shared state space, and performing cooperative training on the material distribution agent and the process scheduling agent to obtain dual agents after cooperative training; obtaining a workshop dynamic event monitored in real time, and updating the shared state space; and triggering the double agents to perform linkage adjustment according to the updated shared state space, and generating and outputting an optimal scheduling scheme. According to the invention, global dynamic optimization of resource utilization and order delivery is realized, and the production efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of profile production scheduling and management, and in particular to a workshop scheduling optimization method, system, equipment and medium for multi-agent collaborative scheduling. Background Technology

[0002] In current discrete manufacturing workshop production scheduling, a hierarchical management framework based on Enterprise Resource Planning (ERP) and Manufacturing Execution System (MES) is commonly used. Under this framework, material allocation and process scheduling are usually designed as two independent and sequentially processed modules. The material management department prepares and allocates materials based on inventory and forecasts, while the production planning department formulates process-level work schedules based on established material availability assumptions, with equipment utilization and order delivery time as optimization objectives.

[0003] However, this separate design leads to significant problems of "information silos" and "decision lag." Because material status (such as real-time availability) and production execution status (such as equipment load and process progress) fail to achieve dynamic coordination and closed-loop feedback, the scheduling system often makes decisions based on static or outdated information. In actual production, this easily leads to two typical failures: first, the process scheduling plan cannot be executed due to incomplete material availability, resulting in idle equipment and production interruptions; second, the material allocation plan fails to flexibly adapt to emergency changes in the process (such as equipment failure or order insertion), leading to inventory backlog or production line waiting for materials. These two factors mutually hinder each other, resulting in overall low efficiency of the production system, weak response to emergencies, and difficulty in achieving global optimization of resource utilization and on-time order delivery rate. Summary of the Invention

[0004] This invention provides a method, system, equipment, and medium for multi-agent collaborative scheduling of workshop scheduling optimization to solve the problems existing in related technologies. The technical solution is as follows:

[0005] In a first aspect, embodiments of the present invention provide a workshop scheduling optimization method for multi-agent collaborative scheduling, comprising: Acquire full-process data of workshop production, including material data, equipment data, process data, and order data; based on the full-process data, integrate the real-time status and constraints of materials, equipment, processes, and orders to build a shared state space; Based on a shared state space, a material allocation agent and a process scheduling agent are constructed and trained. The material allocation agent and the process scheduling agent are then trained collaboratively to obtain a dual agent after collaborative training. During the training process, the material allocation agent adjusts its allocation strategy based on the process scheduling feedback, and the process scheduling agent adjusts its scheduling strategy based on the material allocation results. The system acquires real-time monitored workshop dynamic events and updates the shared state space; it then triggers the two intelligent agents to make coordinated adjustments based on the updated shared state space, generating and outputting an optimized scheduling scheme.

[0006] In one implementation, the method for constructing and training a material allocation agent includes: High-dimensional state data is extracted based on the shared state space, and a first low-dimensional feature vector is generated by the first encoder. The first low-dimensional feature vector is input into the first Actor network, and the first Actor network outputs multiple first candidate allocation strategies that include a three-dimensional action space containing materials, processes, and allocation quantities. Multiple first candidate allocation strategies and a shared state space are input into the first Critic network. The first Critic network evaluates the expected long-term reward of each first candidate allocation strategy and calculates the first advantage function based on the evaluation results. By executing the first candidate allocation strategy in the training environment, a first immediate reward generated by the first multi-objective reward function is obtained, wherein the first multi-objective reward function is used to simultaneously optimize inventory cost, material availability rate and material turnover rate indicators. Based on the first advantage function and the first immediate reward, the parameters of the first Actor network are updated using the PPO-Clip mechanism with the first preset clipping coefficient, and the parameters of the first Critic network are updated based on the first immediate reward, thus obtaining the material allocation agent.

[0007] In one implementation, the first multi-objective reward function adopts a dual constraint mechanism of weighted fusion and KL divergence penalty. The weighted fusion mechanism includes at least three sub-reward items: inventory cost, material availability rate, and material turnover rate. The weight of each sub-reward item is dynamically adjusted according to the real-time production scenario. Furthermore, the first multi-objective reward function includes a penalty item based on KL divergence to constrain the difference between the old and new strategies during the training process.

[0008] In one implementation, the method for constructing and training a process scheduling agent includes: Based on the shared state space and the material allocation results from the material allocation agent, a second low-dimensional feature vector is generated through a dual-attention graph neural network encoder. The second low-dimensional feature vector is input into the second Actor network, and the second Actor network outputs multiple second candidate process scheduling strategies. Multiple second candidate process scheduling strategies and a shared state space are input into the second Critic network. The second Critic network evaluates the expected long-term reward of each second candidate process scheduling strategy and calculates the second dominance function. By executing the second candidate process scheduling strategy in the training environment, the second immediate reward generated by the second multi-objective reward function is obtained; Based on the second advantage function and the second immediate reward, the parameters of the second Actor network are updated using the PPO-Clip mechanism with the second preset pruning coefficient, and the parameters of the second Critic network are updated based on the second immediate reward, finally obtaining the process scheduling agent.

[0009] In one implementation, the second multi-objective reward function includes at least a first sub-objective reward item for optimizing the similarity of the same production line process type, a second sub-objective reward item for optimizing the total completion time of the process, a third sub-objective reward item for optimizing the work-in-process turnover efficiency, and a fourth sub-objective reward item for ensuring that the scheduling plan conforms to the material allocation results.

[0010] In one implementation, the material allocation agent and the process scheduling agent are trained collaboratively to obtain a dual-agent system after collaborative training, comprising: The material allocation agent outputs a material allocation matrix based on real-time data from a shared state space through its first Actor network; The material allocation matrix is ​​input into the process scheduling agent and transformed into constraints in its action space; The process scheduling agent outputs the process scheduling matrix based on constraints through its second Actor network; The process scheduling matrix is ​​used as state feedback and updated to the decision interaction subspace in the shared state space, providing a reference for the material allocation agent to make decisions in the next round.

[0011] In one implementation, the shared state space integrates at least the state data and constraints of materials, equipment, processes, and orders, and is stored and updated in a matrix format.

[0012] Secondly, embodiments of the present invention provide a multi-agent collaborative scheduling workshop scheduling optimization system, which executes the multi-agent collaborative scheduling workshop scheduling optimization method described above; the system includes: The data integration module is used to acquire full-process data of workshop production, including material data, equipment data, process data, and order data; based on the full-process data, it integrates the real-time status and constraint relationships of materials, equipment, processes, and orders to build a shared state space; The agent training module is used to construct and train material allocation agents and process scheduling agents based on the shared state space, and to perform collaborative training on the material allocation agents and process scheduling agents to obtain a dual agent after collaborative training. The linkage scheduling module is used to obtain the dynamic events in the workshop monitored in real time, update the shared state space, trigger the two intelligent agents to make linkage adjustments according to the updated shared state space, and generate and output an optimized scheduling plan.

[0013] Thirdly, an embodiment of the present invention provides an electronic device, which includes a memory and a processor. Among them, the memory and the processor communicate with each other through an internal connection path. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory. When the processor executes the instructions stored in the memory, the processor is caused to execute the method in any one of the above aspects.

[0014] Fourthly, an embodiment of the present invention provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the computer program runs on a computer, the method in any one of the above aspects is executed.

[0015] The advantages or beneficial effects in the above technical solutions at least include: Based on the full-process data, the present invention constructs a shared state space including four dimensions of materials, equipment, processes, and orders, and co-trains the material allocation intelligent agent and the process scheduling intelligent agent. The co-training enables the two intelligent agents of material allocation and process scheduling to share experiences and align goals, and can make real-time linkage decisions based on the global state, breaking the information barrier of traditional serial scheduling, solving the problems of equipment waiting for materials or material idleness caused by plan disconnection, realizing the global dynamic optimization of resource utilization and order delivery, and improving production efficiency.

[0016] The above summary is only for the purpose of the specification and is not intended to be limiting in any way. In addition to the above-described illustrative aspects, embodiments, and features, by referring to the accompanying drawings and the following detailed description, further aspects, embodiments, and features of the present invention will be readily understood. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In the drawings, unless otherwise specified, the same reference numerals throughout the several views represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in accordance with the present invention and should not be regarded as limiting the scope of the present invention.

[0018] Figure 1 It is a schematic flowchart of the workshop scheduling optimization method for multi-agent collaborative scheduling of the present invention; Figure 2 It is a schematic block diagram of the modules of the workshop scheduling optimization system for multi-agent collaborative scheduling of the present invention; Figure 3 It is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0019] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0020] Example 1 This embodiment provides a multi-agent collaborative scheduling method for optimizing workshop scheduling, such as... Figure 1 As shown, the method mainly includes the following steps: Step S1. Data Acquisition and Preprocessing: Acquire full-process data of workshop production, including material data, equipment data, process data, and order data.

[0021] This embodiment comprehensively collects real-time and historical data of the entire workshop production process from three dimensions: material data, equipment data, and process and order data.

[0022] It should be explained that material data includes, but is not limited to, the real-time inventory quantity of each material, the planned and actual arrival time, the variety and specifications information, and the material availability status used to determine whether production can be started (i.e., whether all materials required for a certain order or process have been prepared).

[0023] Equipment data includes, but is not limited to, the real-time operating status of each piece of equipment (such as running, standby, fault), equipment model and capacity parameters, current load rate, and the time spent on each mold change or color change operation.

[0024] Process and order data include, but are not limited to, the final delivery deadline for each order, the type of processes contained in the order (such as cutting, welding, assembly), the process priority and logical dependencies between processes, and the similarity requirements of the process characteristics of the preceding and following processes when producing continuously on the same production line (in order to reduce changeover time).

[0025] The aforementioned end-to-end data is acquired synchronously through two main methods: real-time acquisition at the physical layer and data access at the system layer, ensuring the timeliness and completeness of the information. Real-time acquisition at the physical layer refers to directly collecting real-time status signals such as material flow and equipment operation through IoT devices deployed on the workshop floor (such as RFID readers, status sensors, AGV navigation systems, etc.). Data access at the system layer primarily involves synchronously connecting to the enterprise's existing management information systems via application programming interfaces (APIs) or middleware. For example, it obtains macro-level resource information such as orders, material requirements planning, and inventory master data from the Enterprise Resource Planning (ERP) system; and it obtains workshop-level real-time execution information such as production instructions, work-in-process status, and equipment work logs from the Manufacturing Execution System (MES).

[0026] Furthermore, the collected data from the entire process is preprocessed to eliminate noise and standardize the format, providing reliable input for model training. This preprocessing includes data cleaning and data standardization.

[0027] Data cleaning primarily involves identifying and removing outliers and invalid data caused by sensor communication interruptions, signal interference, or system input errors. For example, it involves marking equipment load values ​​exceeding reasonable physical limits and impossible future timestamp records as invalid and then repairing or removing them.

[0028] Data standardization, on the other hand, transforms data from different sources and with varying units into a unified format and standard. For example, it unifies all time units into "minutes," converts the load rates of different devices into percentages, and standardizes the coding of material specifications.

[0029] The collected data from the entire process, after the above preprocessing, ultimately forms a four-dimensional standardized dataset of "materials-equipment-process-order".

[0030] Step S2. Construct a shared state space for two intelligent agents: Based on the real-time status and constraint relationships of materials, equipment, processes, and orders, a shared state space is constructed.

[0031] This embodiment integrates the real-time status and constraint relationships of materials, equipment, processes, and orders based on the standardized dataset output in step S1, and constructs a shared state space shared by the material allocation agent and the process scheduling agent.

[0032] The shared state space is a collection of all dynamic information in the workshop, equivalent to a shared information database for the material allocation agent and the process scheduling agent. The shared state space includes the real-time status of materials (e.g., 50 units of material A remain, with 30 units expected to arrive in 2 hours), the real-time status of equipment (e.g., equipment 1 currently has a 60% occupancy rate, with remaining capacity to handle 3 processes), the process execution status (e.g., process A has the highest priority and needs to be completed within 8 hours), and the order fulfillment status (e.g., order B has a remaining delivery time of 4 hours, and the current process completion rate is 30%).

[0033] To ensure that the agent's decisions conform to actual production rules, this embodiment defines and integrates the following three types of core production constraints in the shared state space: Material constraints: Define the mandatory association between a process and the required production materials. For example, the start of process C requires two conditions to be met simultaneously: material M1 has been allocated, and the special mold D3 has been assembled. Among them, the availability of material M1 is a necessary prerequisite for triggering this process.

[0034] Equipment constraints: Define the capabilities and limitations of the equipment. For example, equipment 2 is only authorized to process tasks of the "cutting" process type, and requires a fixed mold change preparation time of at least 15 minutes when switching between different tasks.

[0035] Process constraints: Define the logical relationships and technological requirements between processes. For example, process E can only begin after process F is completed (process sequence constraint); on the same production line, the similarity of the technological attributes of adjacent processes that are processed consecutively must reach more than 80% to reduce changeover losses (process similarity constraint).

[0036] This embodiment uses a cleaned and standardized "material-equipment-process-order" dataset as the sole input source. Through information extraction, correlation analysis, and structured encapsulation, a machine-readable and semantically clear structured shared state space is ultimately formed. This shared state space provides a unified, accurate, and fully constrained foundation for the decision-making environment perception of the two subsequent intelligent agents.

[0037] Step S3. Dual-Agent Model Construction and Collaborative Training: Construct and train a material allocation agent and a process scheduling agent based on a shared state space, and perform collaborative training on the material allocation agent and the process scheduling agent to obtain the collaboratively trained dual agents.

[0038] In this embodiment, a reinforcement learning model based on the Proximal Policy Optimization (PPO) algorithm is constructed as the core of the material allocation agent. This model adopts an Actor-Critic architecture, wherein: The first Actor network (policy network) is responsible for outputting the probability distribution in the three-dimensional action space of "material-process-allocation quantity" based on the current state, that is, the tendency to choose various allocation schemes.

[0039] The first Critic network (value network) is responsible for evaluating the long-term expected value of the current state, which guides the policy updates of the Actor network.

[0040] Specifically, the training process for the material allocation agent in S31 includes: Step S311. State Encoding and Feature Extraction: Extract high-dimensional state data based on the shared state space. This high-dimensional state data includes, but is not limited to: real-time inventory levels of all materials, expected material arrival information within the next 2 hours, material requirements lists for all pending processes, and remaining delivery times and priorities for each order. Encode the high-dimensional state data into a first low-dimensional feature vector using the first encoder of the PPO. This process effectively filters out noise and retains the key features required for decision-making.

[0041] Step S312. Strategy Generation: Input the first low-dimensional feature vector into the first Actor network. The first Actor network outputs a probability distribution defined in the three-dimensional action space of "material-process-allocation quantity". Through sampling, 3 to 5 first candidate allocation strategies are generated. For example, strategy A1 may suggest allocating 80% of the scarce material X to the highest priority order O1; strategy A2 may suggest a more even allocation, allocating 60% to O1 and 20% to the order O2 that is about to expire.

[0042] Step S313. Policy Evaluation: Input multiple first candidate allocation policies and global state information of the shared state space into a centralized first Critic network. The first Critic network evaluates the expected long-term reward of each first candidate allocation policy and calculates the first advantage function based on the evaluation results.

[0043] In this context, the first Critic network assigns strategy A to each first candidate. k Calculate a scalar value V(S) t A k ), where S t This represents global state information, with the scalar value indicating the expected long-term cumulative reward for executing this policy in this state. Subsequently, the system calculates the mean value μ of this set of first-candidate allocation policies. v and standard deviation σ v Based on this, the advantage function Adv(A) is calculated for each first candidate allocation strategy. k ) = (V(S t A k ) -μ v ) / σv This intra-group advantage normalization technique eliminates absolute differences in reward scales, making training more stable.

[0044] Step S314. Strategy Execution and Reward Feedback: By executing the first candidate allocation strategy in the training environment, i.e., performing a material allocation, for example, after executing strategy A1, the simulation environment updates the inventory based on the allocation result, advances the process, and may complete some processes due to material availability. Subsequently, the system calls the first multi-objective reward function to calculate the first immediate reward.

[0045] The first multi-objective reward function employs a dual-constraint mechanism of weighted fusion and KL divergence penalty to simultaneously optimize inventory cost, material availability rate, and material turnover rate. The weighted fusion mechanism includes at least three sub-reward items: inventory cost, material availability rate, and material turnover rate, with the weights of each sub-reward item dynamically adjusted based on real-time production scenarios. Furthermore, the first multi-objective reward function includes a penalty term based on KL divergence to constrain the difference between the old and new strategies during training. The expression for the first multi-objective reward function is as follows: First multi-objective reward function = α Inventory cost bonus + beta Complete Set Rate Bonus + γ Turnover rate bonus +λ KL divergence penalty term.

[0046] In the formula, α represents the inventory cost weight; β represents the kitting rate weight; γ represents the turnover rate weight; and λ is a coefficient, λ=0.01. The inventory cost incentive aims to balance inventory levels and avoid material shortages and excessive backlog. Its calculation is based on a comparison between real-time inventory and preset safety thresholds and upper limits. Positive incentives are given when inventory is within a reasonable range, while penalties are imposed based on the degree of deviation when inventory is below the safety threshold or above the upper limit. The kitting rate incentive is directly related to production smoothness. It rewards the number of processes that can be started due to material allocation, while imposing a higher weight penalty on new processes waiting for materials due to improper allocation, thereby minimizing production interruptions. The turnover rate incentive focuses on material flow efficiency, encouraging increased material consumption speed and inventory turnover to reduce stagnant materials. It is typically based on the positive difference between the actual turnover efficiency and the benchmark value. Furthermore, the function includes a KL divergence penalty term, which is for training stability rather than a business indicator. It constrains the magnitude of each strategy update by quantifying the difference between the old and new strategies, preventing drastic fluctuations during training and ensuring smooth training.

[0047] If the system detects that the overall delivery pressure of the current order exceeds the threshold, it automatically increases the kitting rate weight β from 0.3 to 0.5, while decreasing the inventory cost weight α, driving the agent to prioritize ensuring production continuity. In a typical scenario, α=0.4, β=0.3, and γ=0.3.

[0048] Step S315. Strategy optimization and model update: Based on the first advantage function and the first immediate reward, the parameters of the first Actor network are updated using the PPO-Clip mechanism with the first preset pruning coefficient, and the parameters of the first Critic network are updated based on the first immediate reward, finally obtaining the material allocation agent, which is used to receive real-time shared state space data and output the material allocation strategy.

[0049] In this embodiment, after collecting the first advantage functions and corresponding first immediate rewards of all first candidate allocation strategies, the model parameters of the first Actor network and the first Critic network are updated. Specifically: The goal of updating the first Actor network is to guide it to favor actions that yield higher long-term returns. The update process begins by calculating the probability ratio of the new and old policies, which is the current model's evaluation probability for a given first candidate allocation policy divided by the previous version's evaluation probability for that policy. This ratio reflects the direction and extent of policy change.

[0050] To prevent drastic policy jumps during a single update, which could lead to training instability or crashes, this embodiment employs the Clip mechanism of the Proximal Policy Optimization (PPO) algorithm for constraint. The core of this mechanism lies in optimizing the objective function of the first Actor network as follows: ; ; Where θ represents the current parameters of the first Actor network; θ old These are the old parameters from the previous iteration cycle; r1(θ) represents the ratio of the probability of choosing material allocation action a1 in state s under the new and old strategies; Adv1 is the first advantage function provided by the first Critic network, which quantifies the expected additional value of the current action relative to the average level; 1 is the first preset clipping factor (e.g., set to 0.2), which is achieved through the function clip(r1(θ)). 1, 1+ 1) Force the probability ratio r1(θ) to be within the interval [0.8, 1.2] to prevent the update amplitude from being too large; λ1 is the KL divergence penalty coefficient (e.g., 0.01). This is used to measure the difference in distribution between the old and new policies, further constraining the stability of policy updates.

[0051] The entire update process maximizes the objective function J using the stochastic gradient ascent method. Actor1 (θ) is used to fine-tune the internal parameters of the first Actor network, so that the probability of actions that are evaluated as having a higher advantage being selected in the future is moderately increased, while the adjustment range of each update is strictly limited by the pruning mechanism and KL penalty.

[0052] The update objective of the first Critic network is to improve its accuracy in predicting the long-term value of states or state-action pairs, thereby providing a more reliable guiding signal for the first Actor network. During the update, a value target, Target, is first calculated, which is typically based on the immediate actual reward r obtained by the model after performing an action. t And the first Critic network for the next state s t+1 Value estimate V old (s t+1 These factors together constitute an estimate of the total future return, for example, using a time-series difference objective: Target=r t +γ1·V old (s t+1 ), where γ1 is the discount factor.

[0053] Subsequently, the first Critic network previously used this state s t Value prediction V made (s t The first Critic network updates by minimizing this prediction error, specifically using mean squared error as the loss function. This is compared to the calculated target value, and the difference between the two (i.e., the prediction error) is calculated. ; Minimize the loss using stochastic gradient descent. To optimize the parameters of the first Critic network Through repeated iterations, the value predictions of the first Critic network will become increasingly closer to the actual returns, thus enabling the calculation of a more accurate advantage function Adv1, which in turn more effectively guides the policy optimization direction of the first Actor network.

[0054] Step S316. Training Acceleration: To improve training efficiency, this embodiment adopts an asynchronous multi-process architecture, specifically deploying 8 parallel worker processes to asynchronously collect "state-action-reward" data trajectories. A central master process periodically (e.g., every 10 batches of data collected) summarizes all trajectories, performs the aforementioned batch parameter updates, and then synchronizes the updated model parameters to all worker processes, thereby achieving efficient data collection and stable distributed training.

[0055] Finally, through repeated iterations of steps S311 to S316, the material allocation agent gradually learns to balance multiple objectives such as inventory, kitting, and turnover in a complex and dynamic workshop environment, and outputs the globally optimal material allocation decision.

[0056] Furthermore, in this embodiment, the construction of the process scheduling agent mainly adopts a dual-attention mechanism graph neural network as the basic model of the PPO algorithm. By understanding the complex workshop diagram structure, it generates the optimal process scheduling scheme. Specifically, S32. The construction and training process of the process scheduling agent is as follows: Step S321. Joint State Encoding and Feature Extraction Based on Graph Neural Network: Based on the shared state space and the material allocation results (such as material allocation matrix and kitting time) from the material allocation agent, a second low-dimensional feature vector is generated through a dual-attention mechanism graph neural network encoder. Specifically, the input data is modeled as a heterogeneous graph, using real-time updated equipment, process, and order data in the shared state space, as well as the material allocation results output by the material allocation agent. The graph contains two main types of nodes: equipment nodes (attributes include equipment ID, status, load, and capacity list) and process nodes (attributes include process ID, processing time, priority, required material list, and predecessor processes). Edges between nodes represent relationships; for example, a "equipment-process" edge indicates that the equipment can process this process (with adaptability), and a "process-process" edge indicates the process dependency between processes. Subsequently, the heterogeneous graph is input into the dual-attention mechanism graph neural network (GNN) encoder. This encoder performs two key calculations: Node Attention Layer: This layer learns the importance of each node within its neighbor context. For example, it automatically assigns higher attention weights to high-priority process nodes and low-load, capacity-matched device nodes, allowing the model to focus on the elements most critical to the current scheduling decision.

[0057] Edge Attention Layer: This layer learns the importance strength of each edge. For example, for a "device-process" edge, if the device is the optimal or only choice for processing this process, the attention weight of this edge will be enhanced; for a "process-process" edge, if it is a mandatory process sequence constraint, its weight will also be highlighted. This allows the model to accurately quantify the "hardness" of different production constraints.

[0058] After multiple rounds of graph attention message passing and aggregation, the encoder finally outputs a second low-dimensional feature vector that integrates the global state, topology, and constraint strength, providing an information-rich representation for subsequent decision-making.

[0059] Step S322. Generation of candidate scheduling strategies: Input the second low-dimensional feature vector into the second Actor network, and the second Actor network outputs multiple second candidate process scheduling strategies.

[0060] To improve computational efficiency, the action space is represented using a matrix. The output of the second Actor network essentially defines the probability distribution of two key matrices: a device-process allocation matrix (indicating the probability of a process being assigned to each device) and a process start time offset matrix (the adjustment amount to the baseline start time under the condition of satisfying constraints).

[0061] Based on the probability distribution in the three-dimensional action space, K (e.g., 3-5) second candidate process scheduling strategies are generated. Each process scheduling strategy is a specific, executable scheduling scheme that specifies which machine will start processing each process and when.

[0062] Step S323. Evaluation and Advantage Calculation of Scheduling Strategies: The generated scheduling strategies for multiple second candidate processes, along with global information from the shared state space, are input into the second Critic network. The second Critic network calculates the strategy value for each second candidate process scheduling strategy, representing the expected long-term reward that can be obtained by executing this scheduling strategy in the current global state.

[0063] To more stably guide strategy updates, a second dominance function is calculated. This dominance function is achieved by calculating the difference (or further normalization) between the strategy value of each candidate process scheduling strategy and the average strategy value of all candidate process scheduling strategies. This dominance function quantifies the advantage of the current candidate process scheduling strategy relative to the "average" strategy.

[0064] Step S324. Environmental Interaction and Multi-Objective Reward Feedback: The second candidate process scheduling strategy is executed in a training environment simulating the dynamics of a real workshop. The environment simulates the processing procedure based on the equipment allocation and start time in the second candidate process scheduling strategy, handling resource conflicts, material checks, process dependencies, and other logic, and advancing the virtual time. After execution, the second multi-objective reward function is called to calculate the second immediate reward brought by the second candidate process scheduling strategy.

[0065] The second multi-objective reward function includes at least a first sub-objective reward term R used to optimize the similarity of processes within the same production line. 相似 The second sub-objective reward item R, used to optimize the total completion time of the process. 时间The reward function consists of two sub-objectives: a third sub-objective, R_turnover, used to optimize work-in-process turnover efficiency; and a fourth sub-objective, R_constraint satisfaction, used to ensure that the scheduling plan conforms to the material allocation results. These sub-objectives are merged into a single scalar reward value based on preset weights or a dynamic mechanism. The representative formula for the second multi-objective reward function is: Rtotal = δ·Rtotal 相似 + ·R 时间 +ζ·R 周转 +η·R 约束满足 ; Wherein, the first sub-goal reward term R similarity = ; The average similarity (mean) of all consecutive process pairs on the current production line; I is the indicator function, if ≥85%, I=1, additional bonus +0.3, when <85%, I=0, no additional reward.

[0066] R 时间 =max( (-0.4) To estimate the completion time, This refers to the actual completion time. R 周转 = , The baseline work-in-process turnover cycle time; R 约束满足 = , To meet the material kitting constraint on the number of processes, This represents the total number of scheduled processes. When the number of processes to be processed on the same production line is ≥10, the similarity weight δ increases from 0.3 to 0.4; When the urgency of order delivery is ≥90%, the completion time weight ε is increased from 0.4 to 0.5.

[0067] Step S325. Collaborative update and optimization of model parameters: Based on the second advantage function and the second immediate reward, the parameters of the second Actor network are updated using the PPO-Clip mechanism with the second preset pruning coefficient, and the parameters of the second Critic network are updated based on the second immediate reward, finally obtaining the process scheduling agent.

[0068] Specifically, the Clip mechanism of the Proximal Policy Optimization (PPO) algorithm is used to update the second Actor network. For each second candidate action scheduling policy that has been executed in the environment, its selection probability under the policy distribution of the new and old versions of the second Actor network is calculated. The probability ratio of the new and old policies is calculated, which intuitively reflects the degree of change in the preference of the new policy for the action relative to the old policy.

[0069] To prevent drastic and potentially destructive policy jumps during a single update, this embodiment introduces the Clip mechanism of the Proximal Policy Optimization (PPO) algorithm to constrain the second Actor network. This mechanism achieves stable policy updates by optimizing the following objective function: ; ; Where ω represents the current parameters of the second Actor network, ω old These are the old parameters from the previous iteration cycle; r2(ω) represents the ratio of the probability of selecting process scheduling action a2 under the constraints of state s and material allocation matrix M for the new and old strategies; Adv2 is the second advantage function, used to evaluate the quality of scheduling actions relative to the average level; 2 is the second preset clipping factor (typically 0.1 or 0.2), which is achieved through the function clip(r2(ω)). 2, 1+ 2) Limit the probability ratio to the interval [1] 2, 1+ 2] Inside.

[0070] The objective function takes the smaller expected value obtained by multiplying the unpruned probability ratio and the pruned probability ratio by the dominance function, respectively, to construct a conservative policy update "trust region" and prevent policy abrupt changes. The objective function J is maximized using the stochastic gradient ascent method. Actor2 (ω) fine-tunes the parameters of the second Actor network, so that the probability of scheduling actions that bring higher advantages being selected in the future is moderately increased, while ensuring that the update process is smooth and stable.

[0071] The parameter updates of the second Critic network aim to improve its accuracy in predicting the long-term value of states, thereby providing a more reliable advantage estimate (Adv2) for the second Actor network. These updates are achieved by minimizing the following objective function: ; in, 2 represents the parameters of the second Critic network; V 2(s, M) represents the network's value prediction under state s and material constraints M; G2 is the target value that approximates the true value, typically calculated based on the second immediate reward r2 obtained after performing the action and the estimated value for subsequent states, for example, using a temporal difference (TD) objective: G2 = r2 + γ2·V 2,old (s′).

[0072] Where γ2 is the discount factor (0 < γ2 ≤ 1), V 2,old (s′) is the value prediction of the old version of the Critic network for the next state s′ before the parameter update. The objective function J... Critic2 ( 2) Essentially, it is the mean squared error loss function, which measures the value prediction V. The difference between 2(s,M) and the objective value G2 is calculated. This objective function is minimized using stochastic gradient descent to optimize the parameters of the second Critic network. Through repeated iterations, the value prediction of the second Critic network will increasingly accurately reflect the true long-term gains of different scheduling decisions, thus enabling the calculation of a more accurate advantage function Adv2, providing solid guidance for the policy optimization of the second Actor network.

[0073] The updating process of the second Actor network and the second Critic network in steps S321 to S325 is repeated and alternated in the training loop. Each iteration uses the latest interaction data (state, action, reward, new state) to fine-tune the model parameters. As the number of iterations increases, the scheduling strategy generated by the Actor network continues to evolve towards maximizing multi-objective rewards under the value guidance of the Critic network. When the model's performance stabilizes in the validation environment, or when key scheduling indicators (such as average completion time, equipment utilization, and order on-time rate) meet preset requirements, the training process terminates. At this point, the saved second reinforcement learning model with fully optimized parameters constitutes the final, deployable process scheduling agent.

[0074] Step S33. Perform collaborative training on the material allocation agent and the process scheduling agent to obtain a dual agent after collaborative training.

[0075] Specifically, the shared state space is optimized with the goal of constructing a four-dimensional shared state space of "materials-equipment-processes-orders" to provide a unified data source for the two intelligent agents.

[0076] To efficiently manage frequently changing real-time data, a dynamic state subspace is established within the shared state space. This subspace uses a matrix data structure for information storage and organization. For example, the inventory status of materials is stored as an inventory matrix I.m×1 Where m is the number of material types, and each element I[i] represents the real-time available quantity of the i-th material. The overall meaning of m × 1 is that this matrix has m rows and 1 column; the equipment status is stored as the equipment status matrix E. n×p Where n is the number of devices, p is the status dimension (e.g., current load rate, health status code, estimated next idle time); the process progress is stored as a process status matrix O. q×r , where q is the number of processes to be executed, and r is the status dimension (e.g., process priority, assigned device ID, actual start time, percentage of completion).

[0077] Through the data access layer, data streams from sensor networks and business systems are continuously monitored and received. Once a state change is detected (such as material consumption, equipment failure, or process completion), the atomic update of the corresponding matrix element in the dynamic state subspace is immediately triggered, ensuring that the dynamic information in the dynamic state subspace is synchronized with the physical world, providing an accurate decision-making basis for the intelligent agent.

[0078] To enable decision-making collaboration between the two agents, a decision-making interaction subspace is specifically constructed within the shared state space. This subspace stores the material allocation matrix (which defines the mapping relationship between "process-material-allocation quantity") generated by the material allocation agent and the scheduling feedback matrix (which includes the scheduling results of "process-equipment-planned time" and feedback information such as "waiting material process identifier" due to incomplete material sets) generated by the process scheduling agent, thus enabling real-time communication of decision-making data.

[0079] To guide the two agents towards optimal collaborative work at the system level, a global reward subspace is established within the shared state space. This global reward subspace does not store raw data; instead, it calculates joint reward indicators reflecting overall system performance in real time, based on the dynamic state and decision-making interaction subspace. Examples include: on-time order delivery rate calculated based on order delivery deadlines and process progress predictions; global equipment utilization rate calculated by integrating the load and idle time of all equipment; and inventory cost optimization rate calculated by combining inventory levels, turnover rate, and holding cost models. These joint reward indicators (or their weighted combinations) constitute the global reward signal during collaborative training. During the centralized training phase, this signal serves as a standard for evaluating the quality of the joint decisions made by the two agents, driving them not only to optimize their respective local objectives (such as material availability and process similarity) but also to collaboratively optimize the overall efficiency of the entire production system, fundamentally preventing agents from falling into suboptimal solutions due to conflicting objectives.

[0080] The shared state space constructed through the above optimization steps enables end-to-end support from data integration, real-time synchronization, decision interaction to target alignment.

[0081] The material allocation agent and the process scheduling agent are connected to the same training environment and a centralized training paradigm is adopted. The two agents share an experience pool and a global Critic network. The global Critic evaluates the overall value of the joint decision and avoids local optima of a single agent.

[0082] During training, a top-down collaborative decision-making flow is executed, including: Step S331. Generation and Output of Material Allocation Strategy: At the beginning of each training step, the material allocation agent is activated. This agent reads the latest real-time data from the shared state space, including but not limited to the dynamic inventory levels of various materials, the material requirements list for all pending processes, and the priority and urgency of each order. Based on the above state information, the material allocation agent performs inference calculations through its trained first Actor network and outputs a structured material allocation matrix M. This matrix explicitly specifies how many of each type of material should be allocated to which specific pending process or order at the current moment.

[0083] Step S332. Transfer and transformation of material constraints to the process scheduling agent: The material allocation matrix M is input to the process scheduling agent, which parses and transforms it into action space constraints for its own decision model. Specifically, the system determines whether the materials required for each process are "fully equipped" (i.e., all required materials have been allocated in the correct quantities) based on the material allocation matrix M. Only processes marked as "fully equipped" are included in the current set of candidate processes that can be scheduled; conversely, processes with incomplete materials are temporarily blocked and not allowed to be scheduled. This ensures that the process scheduling agent will never assign a task that cannot be started.

[0084] Step S333. Language-based process scheduling decision generation: Under explicit material constraints, the process scheduling agent begins to make scheduling decisions. The process scheduling agent combines information such as equipment status and process attributes read from the shared state space, as well as material constraints received from the material allocation agent, and makes decisions through its trained second Actor network. The second Actor network outputs the decision result, namely a process scheduling matrix. This matrix defines in detail which equipment should be assigned to each (material-complete) process, as well as its planned start and end times, forming a complete production schedule.

[0085] Step S334. Feedback and Closed-Loop Formation of Scheduling Results: The process scheduling matrix is ​​used as a key state feedback and updated in the "decision interaction subspace" of the shared state space. This feedback information not only includes the scheduling scheme itself, but can also generate key execution insights, such as: due to scheduling being advanced or delayed, the actual time point of material demand for some processes has changed; or, the scheduling results reveal certain bottleneck processes whose material consumption rate may differ from expectations.

[0086] In the next training decision cycle, before making a new round of decisions, the material allocation agent reads the latest scheduling feedback from the shared state space. By analyzing the feedback information, the material allocation agent can dynamically adjust its strategy. For example, if the feedback shows that a high-priority process is delayed due to incomplete material sets, the material allocation agent may prioritize ensuring the material supply for that process in subsequent allocations. This constitutes a complete decision-making closed loop of "material allocation → driving scheduling → scheduling feedback → guiding material allocation".

[0087] Step S335. Collaborative Optimization Based on a Global Objective Function: The empirical data generated during the centralized training phase of the above collaborative decision-making process ultimately serves a unified system-level optimization objective. This embodiment minimizes a global objective function J. Global To drive the collaborative training and parameter updates of the two agents, this function integrates the policy optimization objectives and value learning objectives of each agent and introduces global value guidance:

[0088] Among them: J Actor1 (θ) and J Actor2 (ω) represents the policy objective function of the material allocation and process scheduling agent network, respectively, which aims to improve the long-term returns of their respective policies; J Critic1 ( 1) with J Critic2 ( 2) These are the value prediction loss functions (such as mean squared error) for the corresponding Critic networks of the two networks, which aim to improve the accuracy of value estimation; V global (s) is derived from the global Critic network (parameter is...). global The output is a prediction of the overall value of the global state s; G global The actual global return target is calculated based on system-level joint indicators (such as order on-time delivery rate and equipment utilization rate); α3, β3, and γ3 are the weighting coefficients for balancing the various optimization objectives.

[0089] By iteratively repeating the above collaborative decision-making steps (S331-S334), and minimizing the global objective function J based on the collected empirical data, Global (Step S335) The material allocation agent learns to anticipate scheduling feasibility during allocation, and the process scheduling agent learns to optimize production rhythm under constraints. Guided by the value of the global Critic network, their strategy updates not only consider their own rewards but also the impact on the overall system indicators, thus gradually converging to a highly collaborative equilibrium state and achieving integrated collaborative optimization of production resources.

[0090] Step S4. Layered collaborative scheduling execution: Deploy the trained dual agents to the actual production scenario and execute integrated scheduling based on the real-time updated shared state space.

[0091] The collaboratively trained dual agents are deployed to the actual production environment; the material allocation agent outputs the material allocation matrix and material settling time prediction based on the real-time shared state space; the process scheduling agent uses the material allocation matrix as a constraint and the material settling time as the basis for calculating the earliest start time of the process. The process scheduling agent quickly outputs the process scheduling plan through matrix operations, including "the process sequence allocated to each equipment", "the start time and end time of each process", and "the mold change / color change time arrangement".

[0092] Step S5. Dynamic Event Response and Real-time Optimization: Monitor workshop dynamic events such as material events, equipment events, and order events in real time, and update the shared state space to ensure that the decision-making basis information on which the two agents rely is synchronized with the real world.

[0093] Updates to the shared state space immediately trigger a re-decision process between the two agents, which then adjust their actions in tandem based on the latest state. The rapid response of the material allocation agent: The material allocation agent re-runs its decision model based on the updated shared state space (such as new material inventory and new order requirements). For example, when faced with an "urgent order insertion", the agent will prioritize calculating and allocating the materials required for the order, and may dynamically reduce the pre-allocation of some low-priority orders; when faced with "material delays", it will recalculate the material availability time for all affected processes and may trigger alternative material allocation strategies.

[0094] Synchronous rescheduling of the process scheduling agent: When the process scheduling agent synchronously detects state changes (such as equipment unavailability or changes in material kitting time), it will quickly adjust the existing process scheduling scheme based on the latest material allocation matrix and equipment status as hard constraints. For example, for equipment failure events, the agent will automatically migrate unstarted processes on the failed equipment to other equipment with processing capacity and allowable load; for material delay events, it will postpone the earliest start time of the affected processes accordingly and re-optimize the sequence of subsequent processes to minimize the overall completion time delay.

[0095] After generating a new scheduling scheme, it undergoes constraint satisfaction verification and multi-objective performance evaluation. Constraint satisfaction verification ensures the new scheme strictly meets all material constraints (materials for scheduled processes must be complete), equipment constraints (processes must be scheduled on capable and available equipment), and process constraints (such as process sequence and similarity requirements). Multi-objective performance evaluation calculates key performance indicators (KPIs) under the new scheme, including estimated on-time order delivery rate, inventory cost changes, equipment utilization rate, and total completion time, and compares these with the scheme before the event. The goal of the adjustment is to achieve the comprehensive optimization of these multi-objective indicators while satisfying the new constraints, thereby efficiently outputting an optimized new scheduling scheme with response times in minutes or even seconds.

[0096] Step S6. Output and visualization of scheduling results: The optimized scheduling scheme is output in a formatted manner and displayed through a visualization interface.

[0097] The final optimized scheduling scheme generated in step S5 is transformed into specific executable, monitorable, and manageable instructions and views. The specific implementation details are as follows: The optimized scheduling scheme is parsed and formatted into a structured data table that can directly guide production, including: Material Allocation Details: This table details the types and precise quantities of materials allocated to each order or process, along with the estimated material availability time based on current inventory and in-transit information. This table serves as the direct basis for warehouse material preparation and issuance operations.

[0098] Work Process Scheduling Plan (Equipment-Level Work Plan): Centered on each equipment, this plan clearly defines the sequence of work processes to be performed on each machine, as well as the planned start and end times for each process, and the reserved preparation time for production such as mold / color changes. This plan can be directly distributed to the workshop floor as a guide for team members.

[0099] Multi-objective Optimization Index Report: Generate a comprehensive performance report to quantitatively display the expected effects of the current scheduling plan. The core indicators should at least include: order on-time delivery rate, inventory holding cost estimation, average equipment utilization rate, critical path completion time, etc., providing data support for managers' decision-making.

[0100] Meanwhile, for the convenience of personnel in different roles to understand and use, it can be selectively visualized through multiple interfaces. For example, on the large screens at key positions in the workshop, dynamically display the global production Gantt chart, the real-time task status of each device, and the highlighted emergency tasks or delay warnings to achieve information transparency; provide an interactive operation interface for managers such as planners and team leaders, allowing them to view, filter, and export various types of data tables in detail, and can make fine-tuning and confirmation within the system rules through dragging and other methods; or, through a standardized application programming interface, automatically synchronize the final determined material allocation details and process scheduling plans to the enterprise's ERP system (update material reservation and demand) and MES system (issue specific production work orders and operation instructions), forming a data closed-loop from planning to execution, directly driving on-site production activities.

[0101] Based on the workshop scheduling optimization method of multi-agent collaborative scheduling in this embodiment, its beneficial effects are as follows: 1. Break information silos and achieve global collaborative optimization: By constructing a unified four-dimensional shared state space of "material - equipment - process - order", the data barrier between the material flow and the production flow is completely broken. The material allocation agent and the process scheduling agent make decisions and联动 based on real-time and consistent global information, avoiding problems such as plan failure (such as process waiting for materials, useless materials) caused by information asynchronization at the system level, and achieving global optimality of resource allocation.

[0102] 2. Form a decision closed-loop and dynamically respond to production disturbances: Adopting a dual-agent collaborative architecture of "centralized training, distributed execution", a real-time decision closed-loop of "material allocation decision → process scheduling constraint → execution status feedback → allocation strategy adjustment" is established. When dynamic events such as material delays, equipment failures, and emergency order insertions occur, the dual agents can perform millisecond-level联动 adjustments based on the updated shared state space, greatly enhancing the robustness and agility of the production system in dealing with uncertainties.

[0103] 3. Take into account multiple objectives and comprehensively improve production efficiency: By designing a multi-objective reward function with a dynamic weight adjustment mechanism for each agent, the system can simultaneously optimize multiple key performance indicators such as inventory cost, material kit rate, process similarity, equipment utilization rate, and order on-time delivery rate. This multi-objective collaborative optimization mechanism makes the scheduling plan not only meet production feasibility but also achieve a balance in multiple dimensions such as cost, efficiency, and stability, comprehensively improving the overall operational efficiency of the workshop.

[0104] 4. Enabling Real-Time Scheduling for Precise and Efficient Decision-Making: The process scheduling agent innovatively integrates a dual-attention graph neural network with matrix operations, enabling precise modeling of complex equipment-process constraints and transforming scheduling logic into efficient GPU parallel computing. This allows the system to quickly search and output optimal or near-optimal scheduling solutions from a vast number of feasible solutions, meeting the stringent requirements of modern flexible manufacturing for real-time and accurate scheduling.

[0105] In summary, this embodiment effectively solves the inherent problems of insufficient coordination, slow response, and single optimization objective in the traditional hierarchical scheduling mode through a series of collaborative, intelligent, and real-time technical designs, providing core technical support for building an efficient, flexible, and adaptive intelligent workshop scheduling system.

[0106] Example 2 This embodiment provides a multi-agent collaborative scheduling workshop scheduling optimization system, which executes the multi-agent collaborative scheduling workshop scheduling optimization method as described in Embodiment 1; Figure 2 As shown, the system includes: The data integration module is used to acquire full-process data of workshop production, including material data, equipment data, process data, and order data; based on the full-process data, it integrates the real-time status and constraint relationships of materials, equipment, processes, and orders to build a shared state space; The agent training module is used to construct and train material allocation agents and process scheduling agents based on a shared state space, and to perform collaborative training on the material allocation agents and process scheduling agents to obtain a dual agent after collaborative training. The linkage scheduling module is used to acquire real-time monitored workshop dynamic events and update the shared state space; it triggers the two intelligent agents to make linkage adjustments based on the updated shared state space, and generates and outputs an optimized scheduling scheme.

[0107] It should be noted that the functions of each module in the system of this embodiment can be found in the corresponding descriptions in the above methods, and will not be repeated here.

[0108] Example 3 This embodiment provides an electronic device. Figure 3 A structural block diagram of an electronic device according to an embodiment of the present invention is shown. Figure 3 As shown, the electronic device includes a memory 100 and a processor 200. The memory 100 stores a computer program that can run on the processor 200. When the processor 200 executes the computer program, it implements the multi-agent cooperative scheduling workshop scheduling optimization method described in the above embodiments. The number of memories 100 and processors 200 can be one or more.

[0109] The electronic device also includes: The communication interface 300 is used to communicate with external devices and perform data exchange and transmission.

[0110] If the memory 100, processor 200, and communication interface 300 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc.

[0111] Optionally, in a specific implementation, if the memory 100, processor 200, and communication interface 300 are integrated on a single chip, then the memory 100, processor 200, and communication interface 300 can communicate with each other through an internal interface.

[0112] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this invention.

[0113] This invention also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device on which the chip is installed to perform the method provided in this invention.

[0114] This invention also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in this invention.

[0115] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0116] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0117] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0118] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0119] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0120] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-agent collaborative scheduling method for optimizing workshop scheduling, characterized in that, include: Acquire full-process data of workshop production, including material data, equipment data, process and order data; Based on the integrated data of the entire process, the real-time status and constraints of materials, equipment, processes, and orders are used to construct a shared state space; Based on the shared state space, a material allocation agent and a process scheduling agent are constructed and trained, and the material allocation agent and the process scheduling agent are collaboratively trained to obtain a dual agent after collaborative training; wherein, during the collaborative training process, the material allocation agent adjusts the allocation strategy according to the process scheduling feedback, and the process scheduling agent adjusts the scheduling strategy according to the material allocation result. The system acquires real-time monitored workshop dynamic events and updates the shared state space; it then triggers the dual agents to make coordinated adjustments based on the updated shared state space, generating and outputting an optimized scheduling scheme.

2. The workshop scheduling optimization method for multi-agent collaborative scheduling according to claim 1, characterized in that, The method for constructing and training the material allocation agent includes: High-dimensional state data is extracted based on the shared state space, and a first low-dimensional feature vector is generated by the first encoder. The first low-dimensional feature vector is input into the first Actor network, and the first Actor network outputs multiple first candidate allocation strategies that include a three-dimensional action space containing materials, processes, and allocation quantities. The plurality of first candidate allocation strategies and the shared state space are input into the first Critic network, which evaluates the expected long-term reward of each first candidate allocation strategy and calculates the first advantage function based on the evaluation results. By executing the first candidate allocation strategy in the training environment, a first immediate reward generated by the first multi-objective reward function is obtained, wherein the first multi-objective reward function is used to simultaneously optimize inventory cost, material availability rate and material turnover rate indicators. Based on the first advantage function and the first immediate reward, the parameters of the first Actor network are updated using a PPO-Clip mechanism with a first preset clipping coefficient, and the parameters of the first Critic network are updated based on the first immediate reward, ultimately obtaining the material allocation agent.

3. The multi-agent collaborative scheduling workshop scheduling optimization method according to claim 2, characterized in that, The first multi-objective reward function adopts a dual constraint mechanism of weighted fusion and KL divergence penalty. The weighted fusion mechanism includes at least three sub-reward items: inventory cost, material availability rate, and material turnover rate. The weight of each sub-reward item is dynamically adjusted according to the real-time production scenario. Furthermore, the first multi-objective reward function includes a penalty item based on KL divergence to constrain the difference between the old and new strategies during the training process.

4. The workshop scheduling optimization method for multi-agent collaborative scheduling according to claim 1, characterized in that, The method for constructing and training the process scheduling agent includes: Based on the shared state space and the material allocation results from the material allocation agent, a second low-dimensional feature vector is generated through a dual-attention graph neural network encoder. The second low-dimensional feature vector is input into the second Actor network, and the second Actor network outputs multiple second candidate process scheduling strategies; The multiple second candidate process scheduling strategies and the shared state space are input into the second Critic network, which evaluates the expected long-term reward of each second candidate process scheduling strategy and calculates the second dominance function. By executing the second candidate process scheduling strategy in the training environment, a second immediate reward generated by the second multi-objective reward function is obtained. Based on the second advantage function and the second immediate reward, the parameters of the second Actor network are updated using a PPO-Clip mechanism with a second preset pruning coefficient, and the parameters of the second Critic network are updated based on the second immediate reward, ultimately obtaining the process scheduling agent.

5. The multi-agent collaborative scheduling workshop scheduling optimization method according to claim 4, characterized in that, The second multi-objective reward function includes at least a first sub-objective reward item for optimizing the similarity of the same production line process type, a second sub-objective reward item for optimizing the total completion time of the process, a third sub-objective reward item for optimizing the work-in-process turnover efficiency, and a fourth sub-objective reward item for ensuring that the scheduling plan conforms to the material allocation results.

6. The workshop scheduling optimization method for multi-agent collaborative scheduling according to claim 1, characterized in that, The material allocation agent and the process scheduling agent are trained collaboratively to obtain a dual-agent system, which includes: The material allocation agent outputs a material allocation matrix through its first Actor network based on real-time data from the shared state space. The material allocation matrix is ​​input into the process scheduling agent and transformed into constraints in its action space; Based on the constraints, the process scheduling agent outputs a process scheduling matrix through its second Actor network. The process scheduling matrix is ​​used as state feedback to update the decision interaction subspace in the shared state space, so that the material allocation agent can make decisions for the next round.

7. The multi-agent collaborative scheduling workshop scheduling optimization method according to claim 1, characterized in that, The shared state space integrates at least the state data and constraints of materials, equipment, processes, and orders, and is stored and updated in a matrix format.

8. A multi-agent collaborative scheduling workshop scheduling optimization system, characterized in that, The system implements the multi-agent cooperative scheduling optimization method for workshop scheduling as described in any one of claims 1 to 7; the system includes: The data integration module is used to acquire full-process data of workshop production, including material data, equipment data, process data, and order data; based on the full-process data, it integrates the real-time status and constraint relationships of materials, equipment, processes, and orders to construct a shared state space; The agent training module is used to construct and train a material allocation agent and a process scheduling agent based on the shared state space, and to perform collaborative training on the material allocation agent and the process scheduling agent to obtain a dual agent after collaborative training. The linkage scheduling module is used to acquire real-time monitored workshop dynamic events, update the shared state space, trigger the dual intelligent agents to make linkage adjustments based on the updated shared state space, and generate and output an optimized scheduling scheme.

9. An electronic device, characterized in that, include: A processor and a memory, wherein the memory stores instructions that are loaded and executed by the processor to implement the multi-agent collaborative scheduling workshop scheduling optimization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the multi-agent cooperative scheduling workshop scheduling optimization method as described in any one of claims 1 to 7.

Images