Robot assembly planning system and method

Abstract

The invention discloses a robot assembly planning system and method, and belongs to the technical field of robot assembly planning, and the robot assembly planning system comprises a feature processing module and an assembly prediction module. Component nodes, surface nodes and edges among the nodes are provided through a feature processing module to construct an assembly graph with heterogeneous attributes, and data input is provided for subsequent graph representation learning; an assembly prediction module is combined with a graph attention neural network to design a heterogeneous graph representation learning method to predict a next assembly part, and meanwhile, a depth-first method is designed to predict a complete assembly sequence. And the method can also adapt to changes of product structures and new assembly scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of robot assembly planning, and more specifically, proposes a robot assembly sequence planning method based on heterogeneous graph representation learning. Background Art

[0002] With the rise of Industry 4.0, the manufacturing industry is gradually transforming from traditional assembly-line operations to more flexible, intelligent, and responsive production systems. Within the field of intelligent manufacturing, the application of robots is widely considered a key technology driving the future manufacturing revolution. These robots, through their high degree of automation and collaborative capabilities, have the potential to significantly improve production efficiency and reduce labor costs. Robotic assembly sequence planning not only improves production line efficiency and adaptability, but also meets market demand for product personalization and rapid iteration.

[0003] Existing solutions face significant challenges in handling increasingly complex assembly tasks. Traditional approaches rely on complex graph search algorithms and heuristic search strategies, which are often inefficient for large assembly tasks and struggle to adapt to rapid changes in product structure. Furthermore, these methods often require time-consuming feasibility checks during execution, which not only increases computational costs but also limits the responsiveness and flexibility of the production line. Consequently, existing technologies face significant limitations in achieving efficient and reliable automated assembly planning.

[0004] Therefore, in order to solve the above technical problems, it is necessary to provide a robot assembly planning system and method. Summary of the Invention

[0005] The object of the present invention is to provide a robot assembly planning system and method to solve the above-mentioned problems.

[0006] In order to achieve the above-mentioned purpose, the technical solution provided by one embodiment of the present invention is as follows: A robot assembly planning system and method, the assembly planning system comprising: Feature processing module, this module proposes a new modeling method for modeling the overall structure of the assembly parts into a heterogeneous assembly diagram; the heterogeneous assembly diagram is specifically: for a given assembly part , the state Modeled as a graph , which contains two types of nodes: component nodes and surface nodes , and two types of edges: surface-to-surface edges , used to connect all surface nodes; surface to component edges , used to connect each surface node to its corresponding component; Assembly Prediction Module,This module extracts useful information from the assembly diagram based on graph representation learning, and predicts the next action given the current state of an assembly of N parts.

[0007] As a further improvement of the present invention, the component node is responsible for encoding the current state of the assembly. With the eigenvector Related, among them .

[0008] As a further improvement of the present invention, each surface node in the surface node and features Related, among them , both surface type and surface id are vectorized using sinusoidal position encoding.

[0009] As a further improvement of the present invention, the surface-to-surface edge is all surface nodes A fully connected graph is designed to capture the relationships between untouched surfaces. , where 1 means the two surfaces belong to the same part, -1 means the two surfaces are orthogonal, and 0 means they do not belong to the same part.

[0010] As a further improvement of the present invention, the edge from the surface to the component is used to connect each surface and component node. , which represents the surface Belong to the component This type of edge is not associated with any eigenvector.

[0011] As a further improvement of the present invention, the assembly prediction module aims to learn a strategy using heterogeneous graph neural networks , As a parameter, this module outputs , The next assembly The probability of a part.

[0012] As a further improvement of the present invention, the assembly prediction module is composed of three parts: a surface learner, a component learner, and a predictor.

[0013] As a further improvement of the present invention, the component learner and the surface learner are both composed of a graph attention neural network, an instance normalization layer, and a Tanh function. The graph attention neural network can utilize the rich semantics of edge features to update the node features in the graph representation, as shown in the formula: ; in, is the output of the surface learner and the component learner, Represents nodes in a heterogeneous graph (surface nodes , component node ), stands for Graph Attention Neural Network, represents the instance normalization layer, represents the activation layer; Surface learner based on surface nodes and surface-to-surface edges To update node features ; Component learner is based on component nodes , surface nodes , and the edge from the surface to the part To update node features .

[0014] As a further improvement of the present invention, the predictor uses a fully connected layer and a Sigmoid function to make predictions for each component node, as shown in the formula: ; in, is a fully connected layer, is the activation function.

[0015] During training, binary cross entropy is used to minimize the loss between the network output and the ground truth sequence steps from the dataset, and the loss function is applied to each component node separately, which includes an additional regularization term designed to encourage the network not to predict parts that have already been assembled. The loss function is shown in the following formula: ; ; in, is the number of samples in the dataset, It is The number of nodes in the graph, and Respectively expressed as In the figure Model in a sequence of nodes The output score and true value of is the weight, is the value of the assembly flag in the component node.

[0016] As a further improvement of the present invention, a robot assembly planning method includes the following steps: S1. Initialization: From the initial state of the assembly diagram At the beginning, the assembly flags of all component nodes are set to 0. If the current graph The assembly flags of all component nodes in are 1, indicating that all components have been assembled. At this point, the recursion ends and the current sequence list S is returned; S2. Predict the next step: For each component node in the graph, use the assembly prediction module to predict the probability score of the node as the next assembly step If you score Less than the preset threshold , indicating that the possibility of this component being assembled in the next step is low, skip this component and continue to process the next component; S3. Recursion: For scores greater than or equal to the threshold Component nodes, creates a new branch for each such node, and recursively calls the algorithm itself, passing the modified assembly graph ; S4. Output: The algorithm returns a list S containing all possible complete assembly sequences.

[0017] Compared with the prior art, the advantages of the present invention are: 1. This solution designs a graph representation learning method based on heterogeneous graphs to predict assembly sequences. Specifically, the feature processing module proposes component nodes, surface nodes, and edges between nodes to construct an assembly graph with heterogeneous attributes, providing data input for subsequent graph representation learning. The assembly prediction module combines a graph attention neural network to design a heterogeneous graph representation learning method to predict the next component of the assembly. At the same time, a depth-first method is designed to realize the prediction of the complete assembly sequence. This method can not only handle complex assembly tasks, but also adapt to changes in product structure and new assembly scenarios.

[0018] 2. This proposal proposes a model based on graph representation learning that can extract key information from assembly drawings and predict the assembly sequence for each step. It can handle different numbers of input graph nodes and is not restricted by the number of fixed components. This means that this model can effectively transfer knowledge between assembly tasks of different types and scales, improving the adaptability and practicality of the model.

[0019] 3. The model in this scheme uses shared weights on all nodes in the graph, which reduces the complexity burden caused by combinatorial explosion, allowing the model to still maintain efficient memory usage when processing large assembly tasks. An assembly diagram is designed for the assembly process to accurately describe the spatial structure of the assembly parts and the relationship between the parts.

[0020] 4. This proposal designs a method based on heterogeneous graph representation learning, which can not only handle assembly tasks of different types and scales, but also detect and avoid infeasible assembly schemes, thereby significantly improving the efficiency and accuracy of assembly planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of a robot assembly planning system in the present invention; Figure 2 An assembly diagram illustrating the modeling of the feature processing module in the present invention; Figure 3 This is a schematic diagram of the assembly prediction module in the present invention; Figure 4 The figure is a flow chart of a robot assembly planning method in the present invention. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0023] Example 1: See also Figure 1-3 , a robot assembly planning system and method, the assembly planning system comprising: Feature processing module, this module proposes a new modeling method to model the overall structure of assembly parts into heterogeneous assembly drawings; The heterogeneous assembly graph is specifically: for a given assembly component , the state Modeled as a graph , which contains two types of nodes: component nodes and surface nodes , and two types of edges: surface-to-surface edges , used to connect all surface nodes; surface to component edges , used to connect each surface node to its corresponding component; The component node is responsible for encoding the current state of the assembly. Each component node With the eigenvector Related, among them , each surface node in the surface node and features Related, among them , both surface type and surface id are vectorized using sinusoidal position encoding; The surface-to-surface edges are all surface nodes A fully connected graph is designed to capture the relationships between untouched surfaces. , where 1 means the two surfaces belong to the same part, -1 means the two surfaces are orthogonal; 0 means they do not belong to the same part, the edge from the surface to the component: used to connect each surface and component node , which represents the surface Belong to the component This type of edge is not associated with any eigenvector; The assembly prediction module describes the assembly sequence of N parts as a Markov decision process with a discrete state space. and discrete action space In time Status , perform the action Will generate rewards , and according to the transfer function Will state Transfer to state .state Is a binary vector indicating which parts have been placed at the target location (i.e. assembled), where assembled is 1 and unassembled is 0. Represents the next assembly with unplaced parts. For feasible assemblies, there are multiple different sequences that can reach the final state where all N parts are correctly placed. For infeasible assemblies, due to constraints such as part geometry, kinematics and dynamics of the robot system, there will be no such sequence. This module extracts useful information from the assembly diagram based on graph representation learning and predicts the next action given the current state of the assembly of N parts. The goal is to learn a strategy using heterogeneous graph neural networks. , As a parameter, this module outputs , The next assembly The assembly prediction module consists of three parts: surface learner, component learner and predictor.

[0024] The component learner and surface learner are both composed of a graph attention neural network, an instance normalization layer, and a Tanh function. The graph attention neural network can utilize the rich semantics of edge features to update the node features in the graph representation, as shown in the formula: ; in, is the output of the surface learner and the component learner, Represents nodes in a heterogeneous graph (surface nodes , component node ), stands for Graph Attention Neural Network, represents the instance normalization layer, represents the activation layer; Surface learner based on surface nodes and surface-to-surface edges To update node features ; Component learner is based on component nodes , surface nodes , and the edge from the surface to the part To update node features ; The predictor uses a fully connected layer and a Sigmoid function to make predictions for each component node, as shown in the formula: ; in, is a fully connected layer, is the activation function.

[0025] During training, binary cross entropy is used to minimize the loss between the network output and the ground truth sequence steps from the dataset, and the loss function is applied to each component node separately, which includes an additional regularization term designed to encourage the network not to predict parts that have already been assembled. The loss function is shown in the following formula: ; ; in, is the number of samples in the dataset, It is The number of nodes in the graph, and Respectively expressed as In the figure Model in a sequence of nodes The output score and true value of is the weight, is the value of the assembly flag in the component node.

[0026] The above model predicts a set of possible next operations based on the current state of the component. In order to generate a complete sequence (i.e., length N), the present invention designs a depth-first algorithm to achieve the complete assembly sequence prediction based on the predicted state of the assembly drawing, as shown below: A robot assembly planning method comprises the following steps: S1. Initialization: From the initial state of the assembly diagram At the beginning, the assembly flags of all component nodes are set to 0. If the current graph The assembly flags of all component nodes in are 1, indicating that all components have been assembled. At this point, the recursion ends and the current sequence list S is returned; S2. Predict the next step: For each component node in the graph, use the assembly prediction module to predict the probability score of the node as the next assembly step If you score Less than the preset threshold , indicating that the possibility of this part being assembled in the next step is low, skip this part and continue to process the next part; S3. Recursion: For scores greater than or equal to the threshold Component nodes, creates a new branch for each such node, and recursively calls the algorithm itself, passing the modified assembly graph ; S4. Output: The algorithm returns a list S containing all possible complete assembly sequences.

[0027] Notably, the above steps allow the network to consider all possible assembly options at each step and select the best next action based on the current state and existing knowledge. This step-by-step sequence-building approach not only enables it to handle complex assembly tasks but also adapts to changes in product structure and new assembly scenarios.

[0028] The present invention designs a graph representation learning method based on heterogeneous graphs to predict assembly sequences. Specifically, the feature processing module proposes component nodes, surface nodes, and edges between nodes to construct an assembly graph with heterogeneous attributes, providing data input for subsequent graph representation learning; the assembly prediction module combines a graph attention neural network to design a heterogeneous graph representation learning method to predict the next component of the assembly, and at the same time designs a depth-first method to realize the prediction of the complete assembly sequence. This method can not only handle complex assembly tasks, but also adapt to changes in product structure and new assembly scenarios.

[0029] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0030] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A robotic assembly planning system, characterized by: The assembly planning system comprises: Feature processing module, this module proposes a new modeling method to model the overall structure of assembly parts into heterogeneous assembly drawings; The heterogeneous assembly graph is specifically: for a given assembly component , the state Modeled as a graph , which contains two types of nodes: component nodes and surface nodes , and two types of edges: surface-to-surface edges , used to connect all surface nodes; surface to component edges , used to connect each surface node to its corresponding component; Assembly Prediction Module,This module extracts useful information from the assembly diagram based on graph representation learning, and predicts the next action given the current state of an assembly of N parts.

2. A robotic assembly planning system according to claim 1, characterized in that: The component node is responsible for encoding the current state of the assembly. Each component node With the eigenvector Related, among them .

3. A robotic assembly planning system according to claim 1, characterized in that: Each of the surface nodes and features Related, among them , both surface type and surface id are vectorized using sinusoidal position encoding.

4. A robotic assembly planning system according to claim 1, characterized in that: The surface-to-surface edges are all surface nodes A fully connected graph is designed to capture the relationships between untouched surfaces. , where 1 means the two surfaces belong to the same part, -1 means the two surfaces are orthogonal, and 0 means they do not belong to the same part.

5. The robotic assembly planning system according to claim 1, wherein: The surface to component edge: used to connect each surface and component node , which represents the surface Belong to the component ; This type of edge is not associated with any eigenvector.

6. The robotic assembly planning system according to claim 1, wherein: The assembly prediction module aims to learn a strategy using heterogeneous graph neural networks. , As a parameter, this module outputs , The next assembly The probability of a part.

7. A robotic assembly planning system according to claim 6, characterized in that: The assembly prediction module consists of three parts: surface learner, component learner and predictor.

8. A robotic assembly planning system according to claim 7, characterized in that: The component learner and surface learner are both composed of a graph attention neural network, an instance normalization layer, and a Tanh function. The graph attention neural network can utilize the rich semantics of edge features to update the node features in the graph representation, as shown in the formula: ； in, is the output of the surface learner and the component learner, Represents nodes in a heterogeneous graph (surface nodes , component node ), stands for Graph Attention Neural Network, represents the instance normalization layer, represents the activation layer; Surface learner based on surface nodes and surface-to-surface edges To update node features ; Component learner is based on component nodes , surface nodes , and the edge from the surface to the part To update node features .

9. The robotic assembly planning system according to claim 7, wherein: The predictor uses a fully connected layer and a Sigmoid function to make predictions for each component node, as shown in the formula: ； in, is a fully connected layer, is the activation function; During training, binary cross entropy is used to minimize the loss between the network output and the ground truth sequence steps from the dataset, and the loss function is applied to each component node separately, which includes an additional regularization term designed to encourage the network not to predict parts that have already been assembled. The loss function is shown in the following formula: ； ； in, is the number of samples in the dataset, It is The number of nodes in the graph, and Respectively expressed as In the figure Model in a sequence of nodes The output score and true value of is the weight, is the value of the assembly flag in the component node.

10. A robot assembly planning method according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1. Initialization: From the initial state of the assembly diagram At the beginning, the assembly flags of all component nodes are set to 0. If the current graph The assembly flags of all component nodes in are 1, indicating that all components have been assembled. At this point, the recursion ends and the current sequence list S is returned; S2. Predict the next step: For each component node in the graph, use the assembly prediction module to predict the probability score of the node as the next assembly step ; If you score Less than the preset threshold , indicating that the possibility of this part being assembled in the next step is low, skip this part and continue to process the next part; S3. Recursion: For scores greater than or equal to the threshold Component nodes, creates a new branch for each such node, and recursively calls the algorithm itself, passing the modified assembly graph ; S4. Output: The algorithm returns a list S containing all possible complete assembly sequences.

Images