A Distributed Scheduling Method for Stamping Resources in Vehicle Manufacturing under Cloud-Edge-Device Collaboration

Abstract

This invention provides a distributed scheduling method for stamping resources in vehicle manufacturing under cloud-edge-device collaboration, belonging to the technical field of artificial intelligence and intelligent manufacturing. The method is characterized by first deeply analyzing the characteristics of stamping resource scheduling in vehicle manufacturing under cloud-edge-device collaboration, designing a corresponding adaptive distributed scheduling framework for stamping resources, constructing a distributed Markov decision process model, and integrating multi-agent reinforcement learning and federated learning mechanisms for distributed scheduling optimization. Simultaneously, considering the uneven distribution of stamping resource scale and total task volume under cloud-edge-device collaboration, a federated aggregation mechanism based on resource quantity is constructed, and a greedy strategy is introduced to improve the convergence performance of the agents. This invention can be widely applied in vehicle manufacturing stamping production enterprises. The proposed model and method can significantly improve scheduling efficiency and system robustness, while effectively ensuring data privacy and security.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and intelligent manufacturing technology, specifically relating to a distributed scheduling method for stamping resources in vehicle manufacturing under cloud-edge-device collaboration. Background Technology

[0002] In recent years, with the rapid development of information technology, stamping production in vehicle manufacturing has gradually shifted from a cloud manufacturing paradigm to a collaborative "cloud-edge-device" manufacturing paradigm. This model, by integrating powerful computing resources and data storage capabilities, has demonstrated significant advantages in dynamic scheduling and has already been implemented in the operational optimization of energy-intensive industries. As a typical highly flexible discrete manufacturing scenario, stamping production, driven by mixed-flow production, still struggles to meet personalized consumer demands, efficiently utilize massive distributed resources, and effectively ensure the security of production privacy data. Under the cloud-edge-device architecture, the cloud platform, through deep collaboration with edge computing centers, can fully leverage its hierarchical autonomy: relying on distributed computing to achieve task decomposition, improving decision-making efficiency through self-learning optimization, and enhancing system robustness through autonomous control, thereby effectively overcoming the portability and iterability bottlenecks of scheduling services in traditional cloud manufacturing models. It is worth noting that the research on scheduling problems under cloud-edge-device collaboration differs from that on production process monitoring and control. Its collaborative architecture design focuses more on lightweight access of equipment-level software applications, requiring less incremental investment in cloud-edge-device infrastructure, which is beneficial for the technological upgrading of small and medium-sized manufacturing enterprises. Currently, solving scheduling models based on deep reinforcement learning theory has become the mainstream approach for addressing tightly coupled scheduling problems. However, this method requires a large number of historical order and resource samples for training to achieve good results, and the training samples for scheduling problems typically require a long accumulation period. Furthermore, existing optimization algorithms usually focus on improving computational efficiency by modifying mechanisms or strategies; how to further maximize algorithm efficiency by simultaneously using distributed multi-source computing power is often overlooked in current research. In view of these issues, federated deep reinforcement learning can provide a better solution by collaboratively scheduling distributed multi-source heterogeneous computing power to achieve dynamic optimization of global computing resource allocation and improve the training efficiency of scheduling models. Summary of the Invention

[0003] To address the problems of existing technologies, namely the difficulty in fully utilizing distributed multi-source computing power under cloud-edge-device collaboration and effectively protecting the privacy and security of stamping production information, this invention provides a distributed scheduling method for stamping resources in vehicle manufacturing under cloud-edge-device collaboration, thereby improving the training efficiency and privacy security of the scheduling model.

[0004] The technical solution of this invention is as follows:

[0005] Step S10: Construct a distributed scheduling model based on the characteristics of stamping resource scheduling in vehicle manufacturing under cloud-edge-device collaboration, and input the relevant resource and task data;

[0006] Step S20: Construct multiple distributed Markov decision models and determine the representation of agent state, action, and reward function;

[0007] Step S30: Initialize algorithm parameters, including total aggregation rounds K, local training rounds M, number of edge devices Z, and training environment of each device;

[0008] Step S40: Initialize the local model parameters and empty global parameter set of each edge device, and perform M local training rounds on each edge device;

[0009] Step S50: Collect the model parameters trained on all edge devices into the corresponding global set for federated aggregation, and redistribute them to each edge device. Repeat steps S40 to S50 until K rounds of aggregation are completed.

[0010] Step S60: At each edge device, the trained model is used to perform distributed optimization scheduling of the vehicle manufacturing stamping resources.

[0011] Furthermore, in step S20, the determined agent state, action, and reward function are represented as follows:

[0012] 1) Agent State

[0013] Based on the characteristics of stamping resource scheduling in vehicle manufacturing, the state of the intelligent agent is designed using a set of element features. Assuming an intelligent agent The number of resources that need to be scheduled is And in the first In each time step, the resource status is represented as follows:

[0014]

[0015] in, The production factor information representing resource m is constructed based on the scheduling model;

[0016] 2) Agent Actions

[0017] Cloud-edge-device collaboration enables intelligent agents to schedule stamping resources in vehicle manufacturing. This refers to the set of scheduling decisions that can be executed under different resource scheduling environment conditions. Stamping resource scheduling decisions are made by updating and traversing the set of ready subtasks, through actions... Different resources are selected to complete the task. The strategies adopted for action generation include: minimum service type switching times, maximum completion speed, highest processing quality, minimum processing cost, minimum logistics time, and minimum load.

[0018] 3) Agent rewards

[0019] No. During each time step, multiple resource selections and actions may be performed. The reward function obtained afterwards :

[0020]

[0021] in, For resources Completion rate of completed tasks; For resources The number of type switching attempts required to complete the task; For resources The objective function value; To perform the action Unfinished subtasks within the selected task; furthermore, to optimize the final scheduling scheme and workflow, the model considers the agent... The sequence of actions is used to evaluate the final scheduling strategy to complete the scheduling process. During the evaluation, the total reward is calculated by accumulating individual steps. :

[0022]

[0023] in, It is an intelligent agent The scheduling of resources completes the main body's action sequence. . It is an action sequence The corresponding set of environmental states, .

[0024] Furthermore, in step S40, the local training performed by each edge device is as follows:

[0025] Step S401: Initialize the time step counter t=0 and the experience replay buffer D;

[0026] Step S402: Generate an initial set of manufacturing resources and a set of ready subtasks;

[0027] Step S403: Initialize the scheduling environment and observe the initial state. ;

[0028] Step S404: Based on the current policy Select Action ;

[0029] Step S405: Perform the action Select subtasks of the chosen task and update the scheduling scheme;

[0030] Step S406: Remove the scheduled subtask from the ready task set, add its successor task, and update the manufacturing resource set;

[0031] Step S407: Observation Reward and the next state ;

[0032] Step S408: Store the experience tuple ;

[0033] Step S409: Increase time step t by 1;

[0034] Step S410: If the ready subtask set is empty, terminate the loop; otherwise, execute steps S403-S410.

[0035] Step S411: Perform local training update on edge device z ;

[0036] Step S412: If the number of training rounds reaches M, terminate; otherwise, execute steps S403-S412.

[0037] Furthermore, in step S50, the federated aggregation mechanism is as follows:

[0038] In each round of aggregation, the number of distributed agents of the same type included in the aggregation edge layer is N, which can be represented as a set. Each distributed intelligent agent The number of samples that all have schedulable resources is Based on the amount of resources involved in different intelligent agents, corresponding parameter aggregation weights are assigned. Then, through model broadcasting, local model training, local model encryption, edge model uploading, and global aggregation, cloud-edge-device federated aggregation is performed in sequence to generate a new globally optimized model.

[0039]

[0040] in, and These represent the network parameters before and after aggregation, respectively; the improved algorithm introduces a greedy strategy to adjust the final network parameters received at the edge. Adjustments will be made:

[0041]

[0042] in, Indicates intelligent agent Data samples for the network The reward value results of the test. Attached Figure Description

[0043] Figure 1 It is a distributed scheduling framework for stamping resources in vehicle manufacturing under cloud-edge-device collaboration;

[0044] Figure 2 This is a comparison of the training performance of the federated deep reinforcement learning mechanism across different algorithms;

[0045] Figure 3 This is a comparison of the model performance changes and mechanisms for each aggregation when the aggregation frequency is 10 times;

[0046] Figure 4 This is a comparison of the model performance changes and mechanisms for each aggregation when the aggregation frequency is 15 times. Detailed Implementation

[0047] The present invention will now be described in further detail with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0048] Example 1: This invention proposes a distributed scheduling method for stamping resources in vehicle manufacturing under cloud-edge-device collaboration. This method integrates multi-source heterogeneous computing resources to achieve distributed scheduling optimization of stamping resources in vehicle manufacturing. The resource scheduling framework involved is as follows: Figure 1 As shown. Within this framework, the specific steps of the method include:

[0049] Step S10: Construct a distributed scheduling model based on the characteristics of stamping resource scheduling in vehicle manufacturing under cloud-edge-device collaboration; wherein, the order project modeling, manufacturing enterprise modeling, and manufacturing process logistics model involved in the scheduling elements can refer to the technical implementation method of existing patent number CN 2023112990251, and the specific distributed scheduling model is as follows:

[0050] (1)

[0051] St

[0052] (2)

[0053] (3)

[0054] (4)

[0055] (5)

[0056] (6)

[0057] (7)

[0058] (8)

[0059] (9)

[0060] (10)

[0061] (11)

[0062] (12)

[0063] (13)

[0064] in, These represent the theoretical maximum time, quality, cost, and number of switching operations, respectively. For resources Maximum completion time; For subtasks Completion time; For subtasks The start time; For resources The quality of task completion; For resources The cost required to complete the task; For resources The number of type switching attempts required to complete the task; For resources The type switching identifier for completing the nth task indicates the sequence of scheduled tasks; For resources Completion rate of completed tasks; for The ascending order of the completion times of all subtasks; This represents the sequence of scheduled subtasks, arranged in the order of processing; hence, it contains information on scheduled subtasks. Formula (1) is the objective function of the global scheduling model, which maximizes the overall benefit of resource scheduling under the cloud platform; Formula (2) is the objective function of each resource through different weight coefficients. - In a balanced scheduling model, all scheduling objectives have equal importance. All are 0.2. Among them, , , , It was obtained based on the random number method before optimization. Formulas (3)-(5) are used to calculate the maximum completion time of the scheduling scheme after the cloud platform performs scheduling. Formulas (6) and (7) are the average quality and total cost achieved by the resource to complete the scheduling scheme, respectively. Formula (8) is the number of service type switching times for the stamping resource to complete the h-th sub-task. Among them, formula (9) is used to calculate The number of manufacturing types in the sequence; if a task sequence has been scheduled. If the service category of the h-th task is the same as the service category of the (h-1)-th task, then... ,otherwise Formula (10) represents the degree of matching between the completed tasks and the expected task completion order in the project sequence indicator manufacturing resources. Furthermore, the task completion process must also meet the completion time, processing quality, and processing cost requirements of the order project, as shown in formulas (11)-(13). Finally, a case study is conducted using five companies as examples. Each company corresponds to a nearby edge terminal, and the five edge terminals are connected to the cloud center. The number and attributes of resources involved in each company are different, but the resource scale and task scale of each company are similar. Overall, this case involves 40 resources distributed across 20 different locations, and 50 order projects, each containing 4 sub-tasks.

[0065] Step S20: Construct multiple distributed Markov decision models and determine the representation methods for agent states, actions, and reward functions, as follows:

[0066] The stamping scheduling process in vehicle manufacturing is constructed as multiple distributed MDP models, where each agent... Each is responsible for scheduling resources within its respective scenario, selecting stamping manufacturing resources based on task priority sequences and different scheduling rules. The specific modeling is as follows:

[0067] (14)

[0068] Among them, actions Represents intelligent agents The manufacturing rules used to select manufacturing resources; status. Represents intelligent agents The state at time t; Represents intelligent agents In state Use action Selected stamping manufacturing resources; Representing intelligent agents respectively The state space and action space, i.e. ; Represents intelligent agents Parameters of the neural network model used for scheduling decisions.

[0069] Assuming an intelligent agent The number of resources that need to be scheduled is And in the first The agent's state at each time step is:

[0070] (15)

[0071] (16)

[0072] in, To manufacture resources At time step Time-related element attributes; To manufacture resources At time step The identifier for switching service types at that time; It is a manufacturing resource At time step The number of service type switches that have already been performed; To manufacture resources At time step The completion time at that time; It is a resource Up to the time step The degree to which the completed tasks match the expected order of task completion; Indicates manufacturing resources At time step The task load at that time is calculated by formula (17).

[0073] (17)

[0074] in, express At time step Free time before.

[0075] The specific strategy used for generating agent actions is as follows:

[0076] 1) Minimum Service Type Switching Count Strategy: Select Minimal resources;

[0077] 2) Maximum completion speed strategy: Select The greatest resource;

[0078] 3) Highest processing quality strategy: Select The highest resource;

[0079] 4) Minimum processing cost strategy: Select Minimal resources;

[0080] 5) Minimum logistics time strategy: Select Minimal resources;

[0081] 6) Minimum load strategy: Select Minimal resources.

[0082] The neural network of the described agent is based on the Actor-Critic architecture, and each agent includes two types: Policy network. With Value Network The Policy network predicts the next action based on the current state, while the Value network assists in training the Policy network. Assume the current state is... Then the data flow in the Policy network is represented by (18)-(19):

[0083] (18)

[0084] (19)

[0085] in, As an intermediate variable; This is the SoftMax activation function; It is a vector representing the availability of actions, with the same number of bits as the action space; Represents the probability of action selection, based on The resulting probability distribution samples the actions. and The vector size is related to the number of agent action selection strategies. To ensure the generalizability of federated experience, each agent... and The dimensions are the same. The structure of the Value network is similar to that of the Policy network, the only difference being that the Value network is used to evaluate the value gained from the action taken. Therefore, the network is defined as (20)-(21).

[0086] (20)

[0087] ( twenty one )

[0088] To ensure the accuracy of the single-variable output, the last layer was modified to a linear layer.

[0089] Step S30: Initialize algorithm parameters, including total aggregation rounds of 15, local training rounds of 667, number of edge devices of 5, and training environment of each device;

[0090] Step S40: Initialize the local model parameters and empty global parameter set of each edge device, and perform M local training rounds on each edge device. The training process uses six deep reinforcement learning algorithms: SAC, AC, DDPG, DDQN, DQN, and PPO.

[0091] Step S50: Collect the model parameters trained on all edge devices into the corresponding global set for federated aggregation. Repeat steps S40 to S50 until K rounds of aggregation are completed. To verify the superiority of the proposed aggregation mechanism in model training, six deep reinforcement learning algorithms—SAC, AC, DDPG, DDQN, DQN, and PPO—are compared. The training process of each edge node is as follows: Figure 2 As shown, each algorithm can provide a more robust solution, with better exploration capabilities and faster, more stable convergence.

[0092] Step S60: The trained model is used to optimize the scheduling of vehicle manufacturing painting resources. To evaluate the scheduling performance of the algorithm of this invention, the dynamic process is simulated by changing the element information of the resource and task models. It is assumed that after the agent completes the generation action, the following perturbations occur to 5 randomly selected resources and 5 sub-tasks:

[0093] 1) Resource processing speed follows , It is the processing speed of the resource since the last scheduling;

[0094] 2) The workload of subtasks follows... , This refers to the workload of the subtasks after the last scheduling.

[0095] 3) If the selected subtask has a 2% chance of order cancellation, the subtask will be removed from the prepared subtask set. ;

[0096] 4) All selected resources have a 2% probability of encountering a resource failure, causing the resource to enter a state of stagnation. In time step Afterwards, the resources will re-enter the system.

[0097] Each algorithm was executed 10 times for each instance to reduce random influence, and the mean and standard deviation of the repeated experiments were recorded, as shown in Table 1. The SAC algorithm exhibited significant overall performance advantages. Its core advantage lies in obtaining higher and more stable reward values ​​in a relatively short solution time. Compared with other algorithms (including AC, DDPG, DQN, DDQN, and PPO), SAC achieved a significant improvement of 16.33% to 40.89%, indicating that it can obtain higher cumulative rewards in decision-making. In terms of stability, the standard deviation of SAC is at a moderate level, slightly inferior to PPO and DDQN, but superior to AC, DDPG, and DQN. In terms of solution efficiency, the average solution time of SAC is 0.518571 seconds, which is better than DDQN, DQN, and PPO (improvement rate 1.51%–27.69%), but slightly slower than AC and DDPG.

[0098] Table 1. Comparison of Reward Values ​​and Solving Times for Scheduling by Different Algorithms

[0099]

[0100] Example 2: This invention proposes a distributed scheduling method for stamping resources in vehicle manufacturing under cloud-edge-device collaboration. It involves a federated reinforcement learning method incorporating a greedy strategy, differing from the technical implementation used in existing patent CN2022114947162. The federated aggregation process considers the number of resources involved in scheduling each edge device and the degradation caused by average aggregation. The SAC algorithm is used for training. The models before and after aggregation of the two mechanisms are tested 10 times, and the average reward value is used as the reference to calculate the model's scheduling performance improvement rate. Comparative experiments of the federated aggregation strategy are conducted in 10 and 15 iterations out of 10,000 training iterations. The experimental results are as follows: Figure 3 , 4 As shown in the figure. The improvement rate is calculated as (average reward after aggregation - average reward before aggregation) / average reward before aggregation. The results show that while the volatility introduced by the average aggregation strategy may effectively improve the algorithm's "exploration" ability, this improvement is uncontrollable in the long run. A greedy strategy can provide a better balance between stability and improved training performance.

Claims

1. A distributed scheduling method for stamping resources in whole vehicle manufacturing under cloud edge-end collaboration, characterized in that, The method steps are as follows: Step S10, based on the characteristics of vehicle manufacturing stamping resource scheduling under cloud edge cooperation, a distributed scheduling model is constructed, and the involved resource and task data are input; Step S20, a plurality of distributed Markov decision models are constructed to determine the representation of the agent state, action and reward function; Step S30, the algorithm parameters are initialized, including the total aggregation round K, the local training round M, the number of edge devices Z and the training environment of each device; Step S40, the local model parameters of each edge device and the empty global parameter set are initialized, and M local training rounds are performed on each edge device; Step S50, collect all the trained model parameters of the edge devices into the corresponding global set for federal aggregation, and reissue to each edge device, repeat steps S40 to S50 until K rounds of aggregation are completed; Step S60, in each edge device, the trained model is used to perform distributed optimization scheduling on the vehicle manufacturing stamping resource; Further, the representation of the agent state, action and reward function determined in step S20 is as follows: 1) Agent state The agent state is designed according to the stamping resource scheduling characteristics of the whole vehicle manufacturing , and the agent state is designed according to the stamping resource scheduling characteristics of the whole vehicle manufacturing , the number of resources to be scheduled is , and in the first time step, the resource state is represented as follows: ； wherein, represents the production factor information of resource m, which is constructed based on a scheduling model; 2) Agent action Vehicle manufacturing stamping resource scheduling agent action space under cloud edge cooperation refers to a set of scheduling decisions that can be executed under different resource scheduling environment states; the stamping resource scheduling decision updates and traverses the ready sub-task set, and selects different resources to complete the task through actions The strategy adopted in action generation includes: minimum service type switching times strategy, maximum completion speed strategy, highest processing quality strategy, minimum processing cost strategy, minimum logistics time strategy, and minimum load degree strategy. 3) Agent reward The first time step is performed in a first resource selection The reward function obtained after performing the action The reward function obtained after performing the action : ； wherein, is a resource the degree of completion of a task that has been completed; is a resource the number of type switches required to complete a task; is an action uncompleted sub-tasks in the selected task; in addition, in order to optimize the final scheduling scheme and workflow, the model evaluates the final scheduling strategy according to the action sequence of the agent to complete the scheduling process; is a resource the objective function value, the calculation formula is as follows: ； wherein, respectively, the theoretically maximum time, quality, cost, and switching times; is the maximum completion time of the resource ; is the quality of the resource ; is the cost of the resource ; is the type switching times of the resource ; - is a weight coefficient, the importance of each scheduling target in each scheduling target scheduling model is the same, i.e. 0.

2. Additionally, in the evaluation, the total reward is computed by accumulating the individual steps : ； wherein, is an agent schedules resources to complete the sequence of actions of the agent, ; is a corresponding set of environment states for the sequence of actions .

2. The method of claim 1, wherein the method further comprises: In step S40, the local training performed by each edge device is as follows: Step S401: initialize the time step counter t=0 and the experience replay buffer D; Step S402: generate an initial manufacturing resource set and a ready subtask set; Step S403: initialize the scheduling environment, observe the initial state ; Step S404: selecting an action based on the current agent ;​ Step S405: performing an action and updating the scheduling scheme; Step S406: remove the scheduled subtask from the ready task set, add its successor task, and update the manufacturing resource set; Step S407: observing the reward and the next state ; Step S408: Store the experience tuple ; Step S409: increase the time step t by 1; Step S410: if the ready subtask set is empty, terminate the loop, otherwise execute steps S403-S410; Step S411: performing a local training update agent on the edge device z ; Step S412: if the training round reaches M, terminate, otherwise execute steps S401-S412.

3. The method of claim 1, wherein the method further comprises: The federal aggregation mechanism described in step S50 is as follows: In each round of aggregation, the number of distributed agents of the same type included in the aggregation edge layer is N, which can be represented as a set ; each distributed agent owns a sample number of schedulable resources , and based on the number of resources involved by different agents, the corresponding parameter aggregation weight is allocated, and the cloud edge end is cooperated through model broadcast, local model training, local model encryption, edge model uploading and global aggregation process in turn to produce a new global optimization model: ； Wherein, With Respectively represent the network parameters before and after polymerization; the improved algorithm introduces a greedy strategy to adjust the network parameters finally received at the edge : ； wherein, represents a reward value result of testing the network with data samples of the intelligent agent .

Images