Assembly line variable path planning method based on data and knowledge hybrid driving

Abstract

The invention discloses an assembly line variable path planning method based on data and knowledge hybrid driving, which can process more complex assembly line change transmission problems through multi-dimensional change modeling, capture complex relationships in a system and improve accuracy and effectiveness of variable path planning. A multi-agent deep reinforcement learning method of a graph neural network is utilized, and through customized reward function design, the problem that a traditional method is liable to fall into local optimum can be avoided, and a globally optimal solution is searched. By combining expert knowledge and a physical mechanism model with a data driving method, dependence on a large amount of training data can be reduced, the performance of the model under the condition of small data is improved, and the cold start problem is solved. Through hybrid driving of simulation data and experience knowledge, virtual-real mapping rapid modeling in the assembling process can be achieved, the modeling time is shortened, the real-time performance and flexibility of the model are improved, and the method adapts to the rapidly-changing production environment.

Description

Technical Field

[0001] The present invention relates to the technical field of assembly line changes, and in particular to an assembly line variation path planning method based on data and knowledge hybrid drive. Background Art

[0002] As an advanced optimization method, metaheuristic algorithms are widely used in various complex problem-solving scenarios, including engineering design, production scheduling, network optimization, machine learning, and finance. It provides a global search strategy to solve the approximate optimal solution in space by simulating inspirations such as biological behavior or physical phenomena in nature. However, although this method is versatile and flexible, it still has limitations when dealing with high dimensions and complex environments, especially in assembly line change management. It is limited to processing Euclidean space data and it is difficult to fully capture multi-dimensional change relationships, resulting in poor optimization results. In addition, when data is limited, the model is difficult to cold start, which affects the initial performance. Resource allocation is static and lacks dynamic adjustment, which limits efficiency. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an assembly line variant path planning method based on a hybrid drive of data and knowledge.

[0004] To achieve the above purpose, the technical solution provided by the present invention is:

[0005] A method for assembly line variant path planning based on data and knowledge hybrid drive, comprising:

[0006] S1. Carry out multi-dimensional change modeling and establish a directed graph G of the assembly line;

[0007] S2. Based on the directed graph G, a variant path planning model is constructed using graph neural network and deep reinforcement learning;

[0008] S3, optimizing the assembly line change path in the variant path planning model through the reward function;

[0009] S4. Use the deep Q network to train the variant path planning model after the reward function is optimized, and update the Q value through continuous iteration to realize the search for the optimal variant path;

[0010] S5, integrating expert knowledge and physical mechanism models into the variant path planning model that has been trained by the deep Q network;

[0011] S6. According to different optimization objectives, the variant path planning model obtained in step S5 is optimized in both forward and reverse directions. If forward optimization is performed, the final variant path planning model is obtained after forward optimization, and the process proceeds to step S7. If reverse optimization is performed, the process returns to step S1 after reverse optimization.

[0012] S7. Use the final variant path planning model to perform assembly line variant path planning.

[0013] Furthermore, multi-dimensional change modeling is performed, including:

[0014] Data collection and preprocessing:

[0015] Collect relevant data in the assembly line, including equipment information, work-in-progress information, connection relationships between equipment, and production tasks and process data;

[0016] Preprocess the collected data, including data cleaning, standardization and normalization, and store them in a unified format;

[0017] Node feature construction:

[0018] Model each equipment and work-in-progress in the assembly line as a node of a directed graph G;

[0019] Assign characteristics to each node, including the type, status, processing capacity, location, energy consumption of the equipment, and the type, processing stage, size, and weight of the work-in-progress;

[0020] Organize the node feature matrix X to reflect the characteristics of each node in the assembly line;

[0021] Edge feature construction:

[0022] The relationships between equipment in the assembly line and between work-in-process logistics are modeled as edges of a directed graph G.

[0023] Assign characteristics to each edge, including logistics frequency, logistics time, logistics cost, connection strength between devices, accessibility, and transmission delay;

[0024] Organize the edge feature matrix A to reflect the characteristics of each edge in the assembly line;

[0025] Create a graph structure:

[0026] Combine the node feature matrix X and the edge feature matrix A to build the directed graph G of the assembly line.

[0027] Furthermore, a variant path planning model is constructed, including:

[0028] Through the graph neural network, the change transmission relationship is encoded and the features of nodes and edges in the directed graph G are extracted;

[0029] Use deep reinforcement learning algorithm for optimization, select the optimal path for change, and obtain a variant path planning model.

[0030] Furthermore, the formula for extracting node features is:

[0031]

[0032] in, is the feature representation of node v at layer t; N(v) is the set of neighbor nodes of node v; W (t） and are the weight matrices of nodes and edges respectively; Ω is the nonlinear activation function.

[0033] Furthermore, the reward function is a reward function based on time minimization, and the shorter the change propagation time, the higher the reward value;

[0034] The function formula is as follows:

[0035]

[0036] Among them, R(s t ,a t ) is in state s t Next select action a t The reward value of P(a t ) is the selection action a t The path obtained after (u,v) is the change propagation time from node u to node v.

[0037] Furthermore, the Q value update formula is as follows:

[0038] Q(s t ,a t )←Q(s t ,a t )+α[R(s t ,a t )+γ·max α′ Q(s t+1 ,α')-Q(s t ,a t )]

[0039] Among them, Q(s t ,a t ) is in state s t Next select action a t Q value; α is the learning rate; γ is the discount factor s t+1 To perform action a t The next state after max α' Q(s t+1 ,α') is the maximum Q value of the next state.

[0040] Furthermore, the expert knowledge and physical mechanism model are integrated into the variant path planning model that has been trained by the deep Q network. The loss function of the fused variant path planning model is as follows:

[0041]

[0042] in, is the loss function of the fusion model; is the loss function for reinforcement learning; is the loss function of expert knowledge and physical mechanism model; λ is the weight coefficient.

[0043] Furthermore, forward optimization is an indicator prediction optimization, including:

[0044] Take the equipment as the graph node and the WIP logistics as the edge to perform graph regression or node regression;

[0045] Predict node influence and logistics intensity indicators, and evaluate the impact of node and edge characteristics on the overall performance of the assembly line;

[0046] The formula for the regression loss function is as follows:

[0047]

[0048] in, is the regression loss function; y v is the true value; is the predicted value.

[0049] Furthermore, reverse optimization is self-organizing mode improvement optimization, including:

[0050] The equipment and product manufacturing features are used as graph nodes, and the matching processing times between the equipment and manufacturing features are used as the edge weights.

[0051] Perform matrix completion and connection prediction to achieve "node-task" active matching solution recommendation and supplement;

[0052] The formula for the connection prediction loss function is as follows:

[0053]

[0054] in, is the connection prediction loss function; A uv is the actual edge; To predict the edge probability.

[0055] Compared with the prior art, the principles and advantages of this technical solution are as follows:

[0056] 1. Through multi-dimensional change modeling, more complex assembly line change delivery problems can be handled, complex relationships within the system can be captured, and the accuracy and effectiveness of variant path planning can be improved.

[0057] 2. The multi-agent deep reinforcement learning method of graph neural network can avoid the problem of traditional methods easily falling into local optimality through customized reward function design, and find the global optimal solution.

[0058] 3. By combining expert knowledge and physical mechanism models with data-driven methods, we can reduce the dependence on large amounts of training data, improve the performance of the model in small data situations, and solve the cold start problem.

[0059] 4. Through the hybrid drive of simulation data and empirical knowledge, rapid modeling of virtual-reality mapping of the assembly process can be achieved, shortening the modeling time, improving the real-time performance and flexibility of the model, and adapting to the rapidly changing production environment.

[0060] 5. According to the differences in forward and reverse optimization goals, modeling of indicator prediction and self-organizing mode improvement is performed respectively to improve the accuracy of optimization and meet different optimization needs.

[0061] 6. Integrating expert knowledge and physical mechanisms into the variant path planning model that has been trained with a deep Q network can enhance the adaptability and robustness of the model and improve its performance in different environments and working conditions.

[0062] 7. Through matrix completion and connection prediction, it is possible to recommend solutions for active matching of "node-task", improve resource utilization and production efficiency, and make up for the shortcomings of traditional methods in task allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the services required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 The present invention is a principle flow chart of a method for planning assembly line variant paths based on a hybrid drive of data and knowledge;

[0065] Figure 2 It is a schematic diagram of the assembly line diagram structure and its optimization target;

[0066] Figure 3 Schematic diagram of graph neural network perception. DETAILED DESCRIPTION

[0067] The present invention will be further described below in conjunction with specific embodiments:

[0068] like Figures 1 to 3As shown, the assembly line variant path planning method based on data and knowledge hybrid drive described in this embodiment includes the following steps:

[0069] S1. Carry out multi-dimensional change modeling and establish a directed graph G of the assembly line;

[0070] In this step, multi-dimensional change modeling is performed, including:

[0071] Data collection and preprocessing:

[0072] Collect relevant data in the assembly line, including equipment information (equipment type, status, processing capacity, location, energy consumption, etc.), work-in-progress information (type, processing stage, size, weight, etc.), connection relationships between equipment (logistics frequency, logistics time, logistics cost, etc.), and production task and process data;

[0073] Preprocess the collected data, including data cleaning (removing outliers and missing values), standardization and normalization, and store them in a unified format;

[0074] On the assembly line, each device and work-in-progress is modeled as a node set V in the graph G(V,E), and the transfer relationship between devices and work-in-progress logistics is modeled as an edge set E. The mathematical expression of the model is:

[0075] G(V,E)

[0076] V={v1,v2……,v n}

[0077] E={(v i ,v j )|v i ,v j eV}

[0078] Among them, G is the directed graph of the assembly line, V is the node set of equipment and work-in-progress, and E is the directed edge set of the change transfer relationship.

[0079] In the multi-dimensional change modeling step, each device and work-in-progress of the assembly line is modeled as a graph G(V,E), which captures the characteristics of nodes and edges to reflect the complex change transmission relationship in the assembly line. The following is the detailed process:

[0080] Node feature construction:

[0081] In the assembly line, each device and work-in-progress (items being processed) can be regarded as a node. Some features need to be set for each node to describe their properties.

[0082] Node feature example

[0083] Equipment characteristics: type (e.g., robot, conveyor belt), status (normal or faulty), capacity (how much work it can handle), location (where in the assembly line it is), energy consumption (how much electricity it uses).

[0084] Work-in-process characteristics: type (e.g., part A, part B), stage of processing (just beginning, in the middle, almost finished), size, weight.

[0085] Node feature matrix

[0086] Assume there are 4 nodes (devices or WIP) and each node has 3 features. Here is an example feature matrix:

[0087]

[0088] In this matrix:

[0089] The Type column indicates the type of node (1 for equipment, 0 for work in progress).

[0090] The Status column indicates the status of the node (1 for normal, 0 for fault).

[0091] The Work Capacity column indicates work capacity.

[0092] Edge feature construction:

[0093] On the assembly line, the logistics relationship between equipment and work-in-progress can be regarded as edges. It is necessary to set some features for each edge to describe their properties.

[0094] Edge feature example

[0095] Logistics characteristics: logistics frequency, logistics time, logistics cost, etc.

[0096] Connection characteristics: connection strength, reachability, transmission delay, etc. between devices.

[0097] Edge feature matrix

[0098] Assume there are 3 edges and each edge has 3 features. Here is an example feature matrix:

[0099] Edge Set Frequency of information exchange Change delivery efficiency Resource coordination cost Line-1 0.9 5 100 Line-2 0.7 3 80 Line-3 0.8 4 90

[0100] In this matrix:

[0101] The information interaction frequency column indicates the frequency of information exchange between adjacent nodes (equipment and equipment, equipment and work-in-progress), which is particularly important in automated and intelligent assembly lines and may affect the real-time and flexibility of production scheduling.

[0102] The change transmission efficiency column describes the speed at which information or instructions are transmitted on the path when production requirements or product design change, which directly affects the response time of production line adjustment.

[0103] The resource coordination cost column represents the economic cost of reconfiguring and coordinating resources (such as human resources, materials, and equipment resources) between adjacent nodes during the change process, which includes but is not limited to additional adjustment preparation time, temporary shutdown losses, etc.

[0104] Create a graph structure:

[0105] Combine the node feature matrix X and the edge feature matrix A to build the directed graph G of the assembly line.

[0106] S2. Based on the directed graph G, a variant path planning model is constructed using graph neural network and deep reinforcement learning;

[0107] This step specifically includes:

[0108] Through the graph neural network, the change transmission relationship is encoded and the features of nodes and edges in the directed graph G are extracted;

[0109] Use deep reinforcement learning algorithm for optimization, select the optimal path for change, and obtain a variant path planning model.

[0110] The formula for extracting node features is:

[0111]

[0112] in, is the feature representation of node v at layer t; N(v) is the set of neighbor nodes of node v; W (t) and are the weight matrices of nodes and edges respectively; Ω is the nonlinear activation function.

[0113] S3, optimizing the assembly line change path in the variant path planning model through the reward function;

[0114] In this step, the reward function is a reward function based on time minimization. The shorter the change propagation time, the higher the reward value.

[0115] The function formula is as follows:

[0116]

[0117] Among them, R(s t ,a t ) is in state s t Next select action a t The reward value of P(a t ) is the selection action a tThe path obtained after (u,v) is the change propagation time from node u to node v.

[0118] S4. Use the deep Q network to train the variant path planning model after the reward function is optimized, and update the Q value through continuous iteration to realize the search for the optimal variant path;

[0119] In this step, a deep Q network (DQN) is used for training, and the Q value is updated iteratively to search for the optimal variation path.

[0120] During the training process, the model continuously tries different change paths, and adjusts the strategy based on the feedback of the reward function to gradually find the optimal path.

[0121] In DQN, the Q value (Quality Value) represents the sum of the cumulative rewards that can be expected after taking action a in a given state s. The larger the value, the more rewards you can get by taking this action.

[0122] State-Action Pair: The combination of state s and action a is called a state-action pair.

[0123] The update formula of Q value is as follows:

[0124] Q(s t , a t )←Q(s t , a t )+α[R(s t , a t )+γ·max α′ Q(s t+1 ,α′)-Q(s t , a t )]

[0125] Among them, Q(s t ,a t ) is in state s t Next select action a t Q value; α is the learning rate; γ is the discount factor s t+1 To perform action a t The next state after max α' Q(s t+1 ,α') is the maximum Q value of the next state.

[0126] S5. Integrate the expert knowledge and physical mechanism model into the variant path planning model that has been trained by the deep Q network. The loss function of the fused variant path planning model is as follows:

[0127]

[0128] in, is the loss function of the fusion model; is the loss function for reinforcement learning; is the loss function of expert knowledge and physical mechanism model; λ is the weight coefficient.

[0129] S6. According to different optimization objectives, the variant path planning model obtained in step S5 is optimized in both forward and reverse directions. If forward optimization is performed, the final variant path planning model is obtained after forward optimization, and the process proceeds to step S7. If reverse optimization is performed, the process returns to step S1 after reverse optimization.

[0130] In the above, forward optimization is indicator prediction optimization, including:

[0131] Take the equipment as the graph node and the WIP logistics as the edge to perform graph regression or node regression;

[0132] Predict node influence and logistics intensity indicators, and evaluate the impact of node and edge characteristics on the overall performance of the assembly line;

[0133] The formula for the regression loss function is as follows:

[0134]

[0135] in, is the regression loss function; y v is the true value; is the predicted value.

[0136] Reverse optimization is a self-organizing mode improvement optimization, including:

[0137] The equipment and product manufacturing features are used as graph nodes, and the matching processing times between the equipment and manufacturing features are used as the edge weights.

[0138] Perform matrix completion and connection prediction to achieve "node-task" active matching solution recommendation and supplement;

[0139] The formula for the connection prediction loss function is as follows:

[0140]

[0141] in, is the connection prediction loss function; A uv is the actual edge; To predict the edge probability.

[0142] S7. Use the final variant path planning model to perform assembly line variant path planning.

[0143] The embodiments described above are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for assembly line variant path planning based on data and knowledge hybrid drive, characterized in that: include: S1. Perform multi-dimensional change modeling and establish a directed graph G of the assembly line; S2. Based on the directed graph G, a variant path planning model is constructed using graph neural network and deep reinforcement learning; S3, optimizing the assembly line change path in the variant path planning model through the reward function; S4. Use the deep Q network to train the variant path planning model after the reward function is optimized, and update the Q value through continuous iteration to realize the search for the optimal variant path; S5, integrating expert knowledge and physical mechanism models into the variant path planning model that has been trained by the deep Q network; S6. According to different optimization objectives, the variant path planning model obtained in step S5 is optimized in both forward and reverse directions. If forward optimization is performed, the final variant path planning model is obtained after forward optimization, and the process proceeds to step S7. If reverse optimization is performed, then return to step S1 after reverse optimization; S7. Use the final variant path planning model to perform assembly line variant path planning.

2. The assembly line variant path planning method based on data and knowledge hybrid drive according to claim 1 is characterized in that: Conduct multi-dimensional change modeling, including: Data collection and preprocessing: Collect relevant data in the assembly line, including equipment information, work-in-progress information, connection relationships between equipment, and production tasks and process data; Preprocess the collected data, including data cleaning, standardization and normalization, and store them in a unified format; Node feature construction: Model each equipment and work-in-progress in the assembly line as a node of a directed graph G; Assign characteristics to each node, including the type, status, processing capacity, location, energy consumption of the equipment, and the type, processing stage, size, and weight of the work-in-progress; Organize the node feature matrix X to reflect the characteristics of each node in the assembly line; Edge feature construction: The relationships between equipment in the assembly line and between work-in-process logistics are modeled as edges of a directed graph G. Assign characteristics to each edge, including logistics frequency, logistics time, logistics cost, connection strength between devices, accessibility, and transmission delay; Organize the edge feature matrix A to reflect the characteristics of each edge in the assembly line; Create a graph structure: Combine the node feature matrix X and the edge feature matrix A to build the directed graph G of the assembly line.

3. The assembly line variant path planning method based on data and knowledge hybrid drive according to claim 2 is characterized in that: Construct a variant path planning model, including: Through the graph neural network, the change transmission relationship is encoded and the features of nodes and edges in the directed graph G are extracted; Use deep reinforcement learning algorithm for optimization, select the optimal path for change, and obtain a variant path planning model.

4. The assembly line variant path planning method based on data and knowledge hybrid drive according to claim 3 is characterized in that: The formula for extracting node features is: in, is the feature representation of node v at layer t; N(v) is the set of neighbor nodes of node v; W (t) and are the weight matrices of nodes and edges respectively; Ω is the nonlinear activation function.

5. The assembly line variant path planning method based on data and knowledge hybrid drive according to claim 3 or 4, characterized in that: The reward function is a reward function based on time minimization. The shorter the change propagation time, the higher the reward value. The function formula is as follows: Among them, R(s t ,a t ) is in state s t Next select action a t The reward value of P(a t ) is the selection action a t The path obtained after (u,v) is the change propagation time from node u to node v.

6. The assembly line variant path planning method based on data and knowledge hybrid drive according to claim 5 is characterized in that: The update formula of Q value is as follows: Q(s t ,a t )←Q(s t ,a t )+α[R(s t ,a t )+γ·max α' Q(s t+1 ,α')-Q(s t ,a t )] Among them, Q(s t ,a t ) is in state s t Next select action a t Q value; α is the learning rate; γ is the discount factor s t+1 To perform action a t The next state after max α' Q(s t+1 ,α') is the maximum Q value of the next state.

7. The assembly line variant path planning method based on data and knowledge hybrid drive according to claim 6 is characterized in that: The expert knowledge and physical mechanism model are integrated into the variant path planning model that has been trained by the deep Q network. The loss function of the fused variant path planning model is as follows: in, is the loss function of the fusion model; is the loss function for reinforcement learning; is the loss function of expert knowledge and physical mechanism model; λ is the weight coefficient.

8. The assembly line variant path planning method based on data and knowledge hybrid drive according to claim 7 is characterized in that: Forward optimization is indicator prediction optimization, including: Take the equipment as the graph node and the WIP logistics as the edge to perform graph regression or node regression; Predict node influence and logistics intensity indicators, and evaluate the impact of node and edge characteristics on the overall performance of the assembly line; The formula for the regression loss function is as follows: in, is the regression loss function; y v is the true value; is the predicted value.

9. The assembly line variant path planning method based on data and knowledge hybrid drive according to claim 7 is characterized in that: Reverse optimization is self-organizing mode improvement optimization, including: The equipment and product manufacturing features are used as graph nodes, and the matching processing times between the equipment and manufacturing features are used as the edge weights. Perform matrix completion and connection prediction to achieve "node-task" active matching solution recommendation and supplement; The formula for the connection prediction loss function is as follows: in, is the connection prediction loss function; A uv is the actual edge; To predict the edge probability.

Images