Intelligent Scheduling Method for Vehicle Stamping Production Based on GNN and PPO

Abstract

This invention discloses a Generative Neural Network (GNN) and PPO-based intelligent scheduling method for vehicle stamping production (GPVSM), belonging to the technical field of artificial intelligence and intelligent manufacturing. Its key features include: constructing a stamping task resource model through a graph neural network; dynamically selecting the optimal scheduling rule using deep reinforcement learning for PPO; achieving global optimization of resource allocation and task scheduling; designing a PPO-driven scheduling rule combination strategy to integrate the advantages of multiple rules to improve efficiency; and employing orthogonal experiments to optimize hyperparameter configuration and enhance model robustness. This invention is widely applicable to vehicle stamping manufacturing enterprises, effectively solving the problem of flexible resource scheduling in vehicle stamping, helping to improve the efficiency and adaptability of scheduling methods, and providing an intelligent solution for stamping production.

Description

Technical Field

[0001] This invention belongs to the field of intelligent manufacturing production scheduling technology, specifically involving an intelligent scheduling method (GPVSM) for vehicle stamping production based on graph neural networks (GNN) and proximal strategy optimization (PPO), which is applicable to the intelligent scheduling of multi-process operations in stamping workshops of the automotive manufacturing industry. Background Technology

[0002] Vehicle stamping production is a crucial link in automobile manufacturing, involving multiple processes such as blanking, cleaning, stamping, and inspection. It is a typical flexible job shop scheduling problem (FJSP). Traditional stamping shop scheduling methods mainly rely on manual experience or simple rule-based scheduling, which presents the following technical problems:

[0003] a. Difficulty in selecting scheduling rules: In stamping production, it is necessary to select a suitable rule from a variety of scheduling rules such as SPT and LPT. It is difficult to dynamically adjust according to the real-time production status. A single scheduling rule is prone to getting stuck in local optima and cannot obtain a globally optimal scheduling scheme.

[0004] b. Poor dynamic adaptability: Traditional scheduling methods struggle to adjust quickly to sudden disruptions such as order changes, equipment malfunctions, and emergency order insertions, lacking proactive scheduling capabilities, leading to decreased production efficiency. Figure 1 As shown.

[0005] c. Weak state representation capability: The complex constraint relationships between workpieces and machines, between processes, and between workpieces in the stamping workshop are difficult to model accurately using traditional methods, making environmental state construction a major challenge.

[0006] d. High computational complexity: Although exact algorithms such as mixed-integer linear programming and branch-and-bound methods can guarantee the optimal solution for small-scale problems, their computational complexity increases exponentially when facing large-scale scheduling problems, resulting in poor real-time performance and difficulty in meeting the dynamic response requirements of production.

[0007] e. Inability to accurately predict delivery capacity: Due to a lack of comprehensive consideration of the coupling of multiple factors in the production process, it is difficult to accurately predict delivery capacity and optimize process production and material supply plans.

[0008] In existing technologies, although metaheuristic algorithms such as genetic algorithms and simulated annealing have been applied to shop floor scheduling, these methods suffer from high computational complexity and slow convergence speed when dealing with large-scale, multi-constraint stamping shop scheduling problems, and cannot respond to shop floor state changes in real time. While deep reinforcement learning has been attempted for scheduling problems, it still has shortcomings in state representation, action space design, and hyperparameter sensitivity, particularly regarding the rigid constraints of processes and resource coupling characteristics unique to stamping production, lacking effective environmental modeling and feature extraction mechanisms. Summary of the Invention

[0009] The purpose of this invention is to provide an intelligent scheduling method for vehicle stamping production based on graph neural networks and near-end strategy optimization, such as... Figure 2 As shown, this invention achieves real-time optimal scheduling in dynamic environments, significantly improving equipment utilization and production efficiency, and solving the aforementioned technical problems existing in traditional scheduling methods. To achieve the above objectives, this invention provides the following technical solution:

[0010] A method for intelligent scheduling of vehicle stamping production based on graph neural networks and near-end strategy optimization, comprising the following steps:

[0011] Step 1: Construct a multi-dimensional state-space model of the stamping workshop. The stamping production system is abstracted as a flexible workshop, and a state space represented by a disjunctive graph is established. This state space is encoded as a three-dimensional tensor. ,in For the number of workpieces, For the number of machines, This represents the number of feature channels.

[0012] The row dimension of the state tensor represents the workpiece, such as... arrive The column dimension represents the machine, such as arrive The vector dimension corresponds to the feature vector of each workpiece-machine pair. All features are normalized to maintain scale consistency, reduce dimensional differences, and accelerate model convergence.

[0013] Step 2: State Feature Extraction Based on Graph Neural Networks. We use graph-structured data as input and employ a graph convolutional network (GCNN) to perform end-to-end feature extraction, forming a high-level abstract representation of the scheduling state.

[0014] a. Treat each workpiece-machine pair as a graph node, with the node characteristics as follows: dimensional vector;

[0015] b. Construct an adjacency matrix to represent process priority constraints and machine exclusive constraints, where process priority edges connect the preceding and following processes of the same workpiece, and machine competition edges connect the processes to be processed on the same machine.

[0016] c. Aggregate neighbor node information through a 3-layer graph convolutional layer;

[0017] d. The graph neural network adopts a weight sharing mechanism, which significantly reduces the number of trainable parameters, enhances the model's generalization ability, and reduces the risk of overfitting.

[0018] Step 3: Design the action space of the hybrid scheduling rules. For the coupled decision-making process of "process sequencing - equipment assignment" in stamping production, a library of composite scheduling rules (CDRs) is constructed as the action space. This overcomes the shortsightedness of a single action space. The action space contains 12 scheduling rules:

[0019] a. SPT: Shortest processing time priority ;

[0020] b. LPT: Longest processing time takes precedence. ;

[0021] c. SPT+SSO: Minimizes the sum of processing time and subsequent operation time. ;

[0022] d. LPT+LSO: The sum of processing time and subsequent operation time is maximized. ;

[0023] e. SPT×TWK: Minimize the product of processing time and total working hours. ;

[0024] f. LPT×TWK: The product of processing time and total working hours is maximized. ;

[0025] g. SROP: Shortest remaining processing time. ;

[0026] h. GROP: Longest remaining processing time. ;

[0027] i. SRPT: Shortest remaining processing time ;

[0028] j. LRPT: Longest remaining processing time ;

[0029] k. SPT×TWKR: Minimize the product of processing time and remaining total time. ;

[0030] l. LPT×TWKR: The product of processing time and remaining total working hours is maximized. ;

[0031] in, This represents the total processing time for subsequent processes. For workpiece The This process is done on the machine. The processing time.

[0032] The agent, based on the characteristics of the current state Select the optimal combination of scheduling rules from the action space, such as Figure 3 As shown, a decision-making mechanism that implements "rule selection" rather than "direct action" reduces exploration complexity and improves decision interpretability.

[0033] Step 4: Construct a near-end policy optimization decision model. The PPO algorithm is used to train the scheduling policy network, and better policies are sought to guide the selection of high-value actions. Thus, a high reward value can be obtained. The PPO algorithm is based on the A2C algorithm and solves the optimal formula through the gradient method, which improves the action policy effect while effectively enhancing the policy space exploration capability. The training process adopts an offline mode: it is pre-trained using historical production data and continuously fine-tuned through actual production feedback. Each training round collects 2048 state transition samples and trains for 10 epochs.

[0034] Step 5: Design a reward function based on equipment utilization, with the primary objective of maximizing machine average utilization while implicitly minimizing the maximum completion time. ;

[0035] Step 6: Adaptive Hyperparameter Optimization Based on Orthogonal Experiments. To address the hyperparameter sensitivity issue of the PPO algorithm, a systematic optimization is performed using orthogonal experimental design:

[0036] a. Determine key hyperparameter factors: optimizer type Discount Factor Learning rate Batch size ;

[0037] b. Construct an orthogonal array: By using an orthogonal array, the number of experiments is reduced to 48 groups, which efficiently obtains effective information on the impact of hyperparameter changes on model performance;

[0038] c. Response metric calculation: based on scheduling reward value and maximum completion time As a response indicator, each group of parameters was tested three times and the average value was taken to reduce the influence of randomness.

[0039] e. Range Analysis: Calculate the range R for each factor and identify the order of main effects. The analysis shows that: pooling size > discount factor > optimizer > learning rate, with the pooling size range R=0.033 being the largest and having the most significant impact on stamping resource scheduling;

[0040] f. Optimal parameter combination: The combination of the maximum cumulative reward level is selected as {Adamax, γ=0.1, lr=0.01, BatchSize=32}. Under this combination, the average reward value of PPO is 0.542, which is 12.4% higher than the default parameters, showing good stability.

[0041] Based on the above scheduling models and algorithm designs, the final intelligent scheduling algorithm flow for vehicle stamping production based on graph neural networks and near-end strategy optimization can be obtained. Attached Figure Description

[0042] Figure 1 Equipment operating efficiency process diagram

[0043] Figure 2 Intelligent scheduling and production planning framework diagram under deep learning

[0044] Figure 3 Three-dimensional tensor structure for defining stamping production state characteristics

[0045] Figure 4 Comparison chart of cumulative values ​​of orthogonal experimental designs

[0046] Figure 5 Orthogonal experimental design reward value results graph

[0047] Figure 6 Range analysis of various factors and parameters using multiple algorithms

[0048] Figure 7 Comparison of Gantt charts for range selection

[0049] Figure 8 Comparison chart of reward value and time results

[0050] Figure 9 Training convergence curves under different instance environments Detailed Implementation

[0051] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection. Unless explicitly stated otherwise, all technical and scientific terms used in this invention have the same meaning as understood by those skilled in the art to which this application pertains. According to an embodiment of the present invention, a method for intelligent scheduling of vehicle stamping production based on graph neural networks and near-end strategy optimization is disclosed, comprising the following steps:

[0052] Step 1: Construct a multi-dimensional state-space model of the stamping workshop:

[0053] 1) Model the current set of tasks to be scheduled as a flexible job shop problem (FJSP), with a total of J=15 jobs;

[0054] 2) Construct a three-dimensional state tensor, where the number of feature channels is... ;

[0055] 3) All numerical features are normalized to ensure consistent input scale.

[0056] Step 2: State feature extraction based on graph neural network:

[0057] 1) Transform the state tensor into a graph structure;

[0058] 2) Construct the adjacency matrix A:

[0059] a. Process priority edge: Within the same workpiece, the node of the preceding process points to the node of the following process;

[0060] b. Machine competition edge: All process nodes to be assigned on the same machine are connected in pairs (undirected);

[0061] 3) A 3-layer graph convolutional network (GCN) is used for feature aggregation.

[0062] Step 3: Design the action space for hybrid scheduling rules:

[0063] 1) Twelve predefined composite scheduling rules (CDRs) constitute the action space. ;

[0064] 2) In each decision step, the agent selects one of the 12 rules as the current action. ;

[0065] 3) The rule execution logic is parsed by the scheduling engine. For example, selecting "SPT+SSO" means executing the rule. Sort the queue to be scheduled.

[0066] Step 4: Build and train the PPO decision model:

[0067] 1) Policy network structure, outputting a 12-dimensional action probability distribution;

[0068] 2) The value network structure is the same, and the output is a scalar state value. .

[0069] Step 5: Calculate the reward function:

[0070] 1) Calculate the average system utilization rate each time a process completion event is triggered;

[0071] 2) Instant rewards are defined as the increase in utilization rate;

[0072] 3) Cumulative rewards are used for policy gradient updates;

[0073] 4) This sparse reward mechanism avoids policy overfitting caused by dense rewards.

[0074] Step 6: Hyperparameter Tuning:

[0075] 1) Using L 16 (4³×6¹) Orthogonal arrays are used to systematically optimize key hyperparameters;

[0076] a. Factor levels: optimizer (5 levels), γ (6 levels), lr (4 levels), BatchSize (4 levels).

[0077] b. Each experiment is run 10 times on the instance, and the average reward value is calculated. As a response indicator;

[0078] 2) Range analysis showed that batch size had the most significant impact (range R=0.033), followed by the discount factor;

[0079] 3) Finally, the optimal combination is selected;

[0080] 4) The model with this combination has an average reward of 0.542, demonstrating excellent stability.

[0081] Example 1: Scheduling of the stamping workshop of three production lines

[0082] This embodiment is applied to the stamping workshop of an automobile OEM, including one stamping production line, equipped with one blanking machine, one cleaning machine, one stamping machine, and one testing instrument, for a total of four pieces of equipment. The production task is 15 sheet metal parts, each of which needs to go through four processes in sequence: blanking, cleaning, stamping, and testing.

[0083] To verify the effectiveness of this invention, hyperparameter examples were conducted. For example... Figure 4 , Figure 5 As shown, the DQN, DDQN, and Dueling DQN algorithms are less sensitive to parameter changes and can complete the scheduling tasks as expected in most cases, with small fluctuations in task completion time. The PPO algorithm is more sensitive to parameter settings, and its performance varies significantly with parameter changes. However, under optimal parameter conditions, its scheduling performance is significantly better than the other three algorithms. In the PPO algorithm, the experimental results obtained from different optimal parameter combinations are basically consistent, indicating that although the algorithm is easily affected by parameters, its results near the optimal parameters are relatively stable.

[0084] The maximum and cumulative reward values ​​for each factor and level were analyzed and calculated, and the results are as follows: Figure 6As shown in the figure, the difference between the maximum and minimum values ​​of the horizontal numbers is the range. The larger the range R, the more significant the impact of this factor on the scheduling of stamping resources in the experiment, that is, the more important this parameter is in the scheduling process.

[0085] Based on the simulation results, a near-optimal combination of hyperparameters was obtained. To verify its effectiveness, comparative experiments were conducted under the same experimental environment using the obtained optimal parameter combination. The scheduling results obtained based on the simulation experiments and selecting the optimal parameters are as follows: Figure 7 , Figure 8 As shown, the PPO algorithm exhibits the best scheduling performance after optimization and demonstrates stable results, but its performance depends on appropriate parameter settings. Experimental results show that the PPO algorithm can effectively improve scheduling performance; therefore, the method proposed in this invention significantly outperforms the comparative methods in terms of convergence. Overall, the three DQN-based algorithms and the PPO algorithm all belong to deep learning-based scheduling methods. PPO possesses more complex computational mechanisms and policy selection capabilities during training, enabling it to better utilize scheduling rules and obtain higher-quality scheduling policies through learning.

[0086] Running the method of this invention in the above environment and comparing it with five baseline methods (exact algorithm, DQN, DuelingDQN, DDQN, PPO), the results are as follows:

[0087] Table 1. Average reward values ​​obtained from experiments under four different environments using the exact algorithm, DQN, Dueling DQN, DDQN, and PPO. ), path length ( ),ratio.

[0088]

[0089] Training results as follows Figure 9 As shown in the diagram, A1 represents the training result of the DQN algorithm in environment A; A2 represents the training result of the DuelingDQN algorithm in environment A; A3 represents the training result of the DDQN algorithm in environment A; and A4 represents the training result of the PPO algorithm in environment A. The PPO algorithm exhibits better convergence during the scheduling process, significantly shortens the policy learning time, and effectively realizes dynamic scheduling of random tasks. Therefore, it is evident that this invention significantly outperforms existing methods in terms of scheduling quality, real-time performance, and robustness.

[0090] This invention solves the challenges of dynamic, high-dimensionality, and real-time performance in vehicle stamping production scheduling by deeply integrating graph neural networks and near-end policy optimization algorithms. Experiments and industrial applications show that this method significantly improves scheduling quality, response speed, and robustness compared to traditional algorithms, providing an efficient and reliable intelligent scheduling solution for smart manufacturing workshops.

Claims

1. A method for intelligent scheduling of vehicle stamping production based on graph neural networks and near-end strategy optimization, characterized in that, Includes the following steps: Step 1: Construct a multi-dimensional state space model of the stamping workshop, abstract the stamping production system into a flexible operation workshop, and establish a state space represented by a three-dimensional tensor; the three-dimensional tensor includes a processing time channel, a scheduling result channel, and a machine utilization channel. Each channel records the standard processing time of the process on the corresponding machine, the process completion time, and the current utilization rate of each machine, and normalizes all features. Step 2: Extract state features based on graph neural networks. Treat each workpiece-machine pair as a graph node, construct an adjacency matrix containing process priority constraint edges and machine competition constraint edges, and aggregate neighbor information through a multi-layer graph convolutional network to generate a low-dimensional state embedding vector. ; Step 3: Design a hybrid scheduling rule action space, which contains 12 composite scheduling rules. The agent selects a scheduling rule from the action space as the scheduling decision action based on the current state. Step 4: Construct a decision model for optimizing PPO based on the near-end policy, using the state embedding vector. As input, output the action probability distribution, and train the policy network; Step 5: Design a sparse reward function based on equipment utilization rate. Calculate the system average utilization rate increment as an immediate reward at the completion of each process, which is used for policy gradient updates. Step 6: Adaptively optimize the key hyperparameters of the PPO algorithm using orthogonal experimental design to determine the optimal hyperparameter combination.

2. The intelligent scheduling method for vehicle stamping production according to claim 1, characterized in that, The three feature channels of the three-dimensional tensor are specifically defined as follows: a. Processing time channel: If the process Available in machine For processing, enter the standard processing time. Otherwise, it is 0; b. Scheduling Result Channel: Records the estimated completion time of each process on the corresponding machine. Initially, it is a zero matrix, which is dynamically updated as scheduling progresses. c. Machine utilization channel: This represents the load rate of each machine at the current moment, with a value range of [0,1].

3. The intelligent scheduling method for vehicle stamping production according to claim 1, characterized in that, The graph neural network uses a 3-layer graph convolutional network with an output dimension of 128 for each layer, an activation function of ReLU, a dropout rate of 0.2, and a weight sharing mechanism to reduce the number of trainable parameters.

4. The intelligent scheduling method for vehicle stamping production according to claim 1, characterized in that, The reward function is defined as follows: ； in For machines Machining workpieces The actual time occupied, for The system's average utilization rate at any given time, coupled with a highly sparsity reward function that triggers only upon completion of a process, avoids policy shifts caused by dense rewards, accelerating convergence. This is achieved at the static initial scheduling time. The reward is calculated only when the process completion event is triggered.

5. The intelligent scheduling method for vehicle stamping production according to claim 1, characterized in that, The orthogonal experimental design uses orthogonal arrays to test combinations of four hyperparameters: optimizer type, discount factor γ, learning rate lr, and batch size. Scheduling reward value and maximum completion time are also considered. As a response indicator, the optimal parameter combination is determined through range analysis.

6. The intelligent scheduling method for vehicle stamping production according to claim 1, characterized in that, include: a) Graph modeling and feature extraction module, used to construct state tensors and call graph neural networks to generate state embeddings; b. PPO policy decision module, used to run the trained policy network and output the optimal scheduling rule; c. Scheduling execution engine, used to parse the selected scheduling rules and generate task sequences.

7. The intelligent scheduling method for vehicle stamping production according to claim 1, characterized in that, The method is applicable to stamping workshops in automobile manufacturing, and each process can be performed on multiple candidate presses, satisfying the constraints of the Flexible Jobs Schedule (FJSP) problem.

Images