Method and system for adjusting man-machine cooperation assembly station based on graph learning

Abstract

The invention relates to the technical field of mechanical equipment, and discloses a method and a system for adjusting a man-machine cooperation assembly station based on graph learning. The method comprises the following steps: constructing an integrated virtual reconstruction model based on a training method; and the production change information is input into the virtual reconstruction model, and an assembly adjustment strategy is output. The training method comprises the following steps: acquiring an image set of an assembly station, and preprocessing the image set; establishing a graph structure based on an assembly station according to the preprocessed image set, and establishing a graph convolutional network according to the graph structure; the graph convolutional network is optimized, and an optimized graph convolutional network is obtained; and training the optimized graph convolutional network, defining a loss function in the training process to adjust the optimized graph convolutional network, and obtaining a trained graph convolutional network. According to the method, virtual and real synchronization and fusion are realized in a man-machine cooperation assembly system simulation form, so that the consumption of a man-machine cooperation assembly station is greatly reduced, and the production efficiency of an enterprise is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mechanical equipment, and in particular to an adjustment method and system for a human-machine collaborative assembly station based on graph learning. Background Art

[0002] Human-machine collaborative assembly stations are becoming more and more popular in many fields, but they are also facing many challenges. On the one hand, internal process improvements require a lot of manpower, material resources, and financial resources, and they must meet market needs as quickly as possible; on the other hand, market demand changes all the time, requiring companies to dynamically adjust production plans, and assembly systems need to be more flexible. Moreover, customer needs are becoming more and more diverse and personalized, and customization has become an inevitable trend. First of all, the current human-machine collaborative assembly stations have a slow response speed. Internal process improvements require employees to train time and resources, and machines need time to adapt. In the short term, productivity has dropped significantly and cannot respond quickly to market demand. Secondly, the current human-machine collaborative assembly stations are very expensive. Under process improvements or external interference, if no optimization solution is found in the short term, it will cause certain economic pressure. Finally, the current human-machine collaborative assembly stations have poor flexibility, and their efficiency is greatly reduced when dealing with implicit interference and explicit changes. The prior art discloses a human-machine collaborative assembly workstation and method. The human-machine collaborative assembly workstation includes a conveyor roller, a collaborative robot, an assembly positioning sleeve, a stamping mechanism, and a control system; the control system includes a controller and a human-machine interaction interface; a tooling plate is provided on the conveyor roller, the tooling plate is used to carry the gear shaft and the bearing, and the conveyor roller can drive the tooling plate to drive the gear shaft and the bearing to move; the collaborative robot can carry the gear shaft between the assembly positioning sleeve and the stamping mechanism; the stamping mechanism is used to stamp the gear shaft with the bearing; the controller is electrically connected to the machine interaction interface, the conveyor roller, the collaborative robot, and the stamping mechanism respectively. The prior art improves the efficiency of the bearing press-fitting process by using a collaborative robot and manual labor, and uses a collaborative robot to put the gear shaft with the bearing into the stamping mechanism, thereby improving the safety of the assembly process. However, the prior art has poor flexibility. When faced with process improvements or changes in market demand, it cannot respond quickly, resulting in a significant decrease in production efficiency, and training professional operators also requires a lot of time and economic costs. Summary of the invention

[0003] The primary purpose of the present invention is to overcome the problems existing in the prior art and provide a human-machine collaborative assembly method based on graph learning. The present invention greatly reduces the cost of human-machine collaborative assembly stations through the form of human-machine collaborative assembly system simulation, virtual and real synchronization and integration. If the prior art encounters process changes, environmental changes, etc., it will take up a lot of time, space and funds to adjust and adapt. However, this method is used to reconstruct the model, simulate adjustment and optimization, thereby optimizing the production plan offline, shortening its joint debugging and testing cycle, and improving the production efficiency of the enterprise.

[0004] As another object of the present invention, a system adapted to the method based on the aforementioned object is provided.

[0005] As another object of the present invention, a non-volatile storage medium suitable for storing a computer program implemented according to the method described is provided.

[0006] In order to achieve the above object, the present invention provides a training method for a graph convolutional network for an assembly station, the method comprising the following steps: Acquire an image set of an assembly station, and pre-process the image set; Establishing a graph structure based on the assembly station according to the preprocessed image set, and establishing a graph convolutional network according to the graph structure; Optimizing the graph convolutional network to obtain an optimized graph convolutional network; The optimized graph convolutional network is trained, and a loss function is defined during the training process to adjust the optimized graph convolutional network to obtain a trained graph convolutional network.

[0007] Furthermore, the preprocessing includes removing missing or abnormal images in the image set and unifying the formats of the remaining images in the image set to obtain a preprocessed image set.

[0008] Furthermore, establishing a graph convolutional network according to the graph structure specifically includes: Acquiring an adjacency matrix and a node feature matrix based on the graph structure, and normalizing the adjacency matrix to obtain a normalized adjacency matrix; A graph convolutional network is constructed according to the normalized adjacency matrix and the node feature matrix.

[0009] Furthermore, the graph convolutional network is constructed according to the normalized adjacency matrix and the node feature matrix, and the construction method is as follows:

[0010] in For the The node feature matrix of the layer, is the normalized adjacency matrix, For the The weight matrix of the layer, is the activation function, where the normalized adjacency matrix The calculation method is as follows:

[0011] in, is the degree matrix, is the identity matrix, is the adjacency matrix.

[0012] Furthermore, the optimization of the graph convolutional network specifically includes: optimizing the parameters of the graph convolutional network to obtain a first graph convolutional network, the parameters including convolution batch size, convolution kernel size and number of convolution layers; in the first graph convolutional network, using a graph attention pooling method instead of a global average pooling method to calculate node feature weights to obtain an optimized graph convolutional network.

[0013] Furthermore, the loss function includes a first loss function and a second loss function, wherein the first loss function is used to evaluate the performance of the loss-optimized graph convolutional network in predicting the attribute or value of a node, and the second loss function is used to evaluate the performance of the loss-optimized graph convolutional network in predicting whether there is an edge between two nodes, wherein the first loss function is as follows:

[0014] in, is the number of nodes in the graph structure based on the target assembly station, is the true value of the node, is the predicted value of the node; The second loss function is as follows:

[0015] in, is the element of the real adjacency matrix, representing the node and Does the edge between them exist? For the optimized graph convolutional network, the nodes and The predicted probability that there is an edge between them.

[0016] A method for adjusting a human-machine collaborative assembly station based on graph learning, the method being based on a training method for a graph convolutional network for an assembly station, comprising: constructing an integrated virtual reconstruction model based on the training method; inputting production change information into the virtual reconstruction model, and outputting an adjusted assembly strategy.

[0017] Further, the integrated virtual reconstruction model specifically includes: a visual sensor, a soft PLC and its external modules, and simulation software; The visual sensor is used to collect an image set of the target assembly station; The soft PLC and its external modules train a graph convolution network for a target assembly station based on the training method; obtain the trained graph convolution network for the target assembly station, and the trained graph convolution network for the target assembly station receives an image set of the target assembly station to obtain optimization data; The simulation software obtains an optimization model according to the optimization data, and obtains an adjustment assembly strategy based on the optimization model and the original model.

[0018] In order to achieve another object of the present invention, the present invention further provides a human-machine collaborative assembly system based on graph learning, the system is based on the human-machine collaborative assembly method based on graph learning, and includes: Construction module: constructing an integrated virtual reconstruction model based on the training method; Output module: inputs the production change information into the integrated virtual reconstruction model and outputs the adjustment assembly strategy.

[0019] In order to achieve another purpose of the present invention, the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer-stored program is executed by a processor, the adjustment method of a human-machine collaborative assembly station based on graph learning is implemented.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention uses graph convolutional neural networks to analyze and learn nodes, establish an integrated virtual reconstruction model, and can be simulated in the human-machine assembly collaboration system and simulation system to perform iterative optimization of new process production design and execution, effectively improving the rationality of the entire system design and realizing intelligent operation and maintenance of the human-machine assembly collaboration system; 2. It also uses the form of human-machine collaborative assembly system simulation, virtual and real synchronization and integration, which greatly reduces the cost of human-machine collaborative assembly stations. If the existing technology encounters process changes, environmental changes, etc., it will take up a lot of time, space and funds to adjust and adapt. The method of the present invention is used to train and reconstruct the model, simulate adjustment and optimization, thereby optimizing the production plan offline, shortening its joint debugging and testing cycle, and improving the production efficiency of the enterprise; 3. By collecting the movement and production data of the human-machine collaborative assembly station, based on the image data that can be used to record that the physical equipment has not been tampered with, binding with the assembly execution system can achieve product life cycle quality control and anti-counterfeiting, thereby tracking the life cycle of the assembly parts. It greatly reduces the cost of manual verification and workpiece tracking, and provides technical support for transparent monitoring of human-machine collaborative assembly. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a method for adjusting a human-machine collaborative assembly station based on graph learning according to an embodiment of the present invention; Figure 2 It is a block diagram of an adjustment system of a human-machine collaborative assembly station based on graph learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0023] In the description of the present invention, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0024] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] Furthermore, in the description of the present invention, unless otherwise specified, “plurality” means two or more.

[0026] Example 1 like Figure 1 As shown, a method for adjusting a human-machine collaborative assembly station based on graph learning in a preferred embodiment of an embodiment of the present invention includes: S1: Obtain the image set of the assembly station and preprocess the image set; In a feasible embodiment, the preprocessing includes removing missing or abnormal images in the image set and unifying the formats of the remaining images in the image set to facilitate subsequent processing, thereby obtaining a preprocessed image set.

[0027] S2: Establish a graph structure based on the assembly station according to the preprocessed image set, and establish a graph convolutional network according to the graph structure; In a feasible embodiment, S2 specifically includes: S2.1: Build a graph structure based on the assembly station based on the preprocessed image set. Specifically, (1) determine the nodes and edges. The elements of each assembly station production line (such as machines, tools, workpieces, etc.) can be represented as nodes in the graph, and the relationship between nodes (such as workflow, tool use, etc.) can be represented by edges; (2) define the node and edge attributes. Adding attributes to each node and edge can describe the elements of the assembly station in more detail. For machines, its node attributes include type, status, and production capacity; for workpieces, its node attributes include size, weight, and material. Similarly, edges can also contain attributes to represent the characteristics of the relationship. For example, in a certain workflow, the attributes can be the required time, priority, etc.; (3) build a graph structure and use graph data structures (such as adjacency list: use a list to store each node and its adjacent nodes. Adjacency matrix: use a two-dimensional array to represent the connection relationship between nodes. The rows and columns of the matrix correspond to the node information in the graph, and the value of the element indicates the existence or weight of the edge.) to realize the above nodes and edges.

[0028] S2.2: Obtaining an adjacency matrix and a node feature matrix based on the graph structure, normalizing the adjacency matrix to obtain a normalized adjacency matrix; constructing an adjacency matrix according to the graph structure And the node feature matrix, the adjacency matrix A is normalized and recorded as , normalized expression:

[0029] in, is the degree matrix, is the identity matrix; S2.3: Construct a graph convolutional network according to the normalized adjacency matrix and the node feature matrix, and the construction method is as follows:

[0030] in For the The node feature matrix of the layer, is the normalized adjacency matrix, For the The weight matrix of the layer, is the activation function.

[0031] S3: Optimizing the graph convolutional network to obtain an optimized graph convolutional network; In a feasible embodiment, S3 specifically includes: S3.1: Optimize the parameters of the graph convolutional network to obtain the first graph convolutional network, wherein the parameters include the convolution batch size, the convolution kernel size, and the number of convolution layers. Specifically, different kernel parameter settings (such as the stride, padding, kernel size, etc. of the convolution layer) are performed and experiments are conducted in a distributed environment. Here, according to three parameters, the first is the convolution batch size, which is mainly the number of samples used to calculate the gradient at each iteration. Try different batch sizes, such as 32, 64, 128, etc. A smaller batch size can improve the generalization ability of the model, but may increase the training time. The second is the convolution kernel size, which is mainly the size of the filter used in the convolution operation. Select a suitable convolution kernel size according to the characteristics of the input data, such as 3x3, 5x5, etc. The third is the number of convolution layers, which is mainly the number of convolution layers in the network. Observe the impact on model performance by increasing or decreasing convolution layers. Analyze the kernel parameters for transmission and updating between different computing nodes. Different network topologies were set to observe the effect, performance, and efficiency differences of parameter transfer, so as to analyze the impact of different kernel parameters on the transfer parameters, compare multiple groups of data, find the best kernel parameter solution, and finally obtain the first convolutional network.

[0032] S3.2: In the first graph convolutional network, the graph attention pooling method is used instead of the global average pooling method to calculate the node feature weights to obtain an optimized graph convolutional network. Specifically, considering the temporal factors, each assembly feature may change over time. The graph attention pooling method is used to dynamically calculate the weight of each feature before and after the assembly station, and the weighted sum of each feature is performed to obtain a comprehensive feature representation. Attention pooling can dynamically adjust the attention level of the feature before and after the assembly station, thereby improving information utilization; reducing the learning of other parameters, by focusing on the most relevant feature information, the graph attention pooling method helps to improve the accuracy and robustness of the model.

[0033] S4: Train the optimized graph convolutional network, define a loss function during the training process to adjust the optimized graph convolutional network, and obtain a trained graph convolutional network.

[0034] In a feasible embodiment, the loss function includes a first loss function and a second loss function, wherein the first loss function is used to evaluate the performance of the loss-optimized graph convolutional network in predicting the attribute or value of a node, and the second loss function is used to evaluate the performance of the loss-optimized graph convolutional network in predicting whether there is an edge between two nodes, wherein the first loss function is as follows:

[0035] in, is the number of nodes in the graph structure based on the target assembly station, is the true value of the node, is the predicted value of the node; The second loss function is as follows:

[0036] in, is the element of the real adjacency matrix, representing the node and Does the edge between them exist? For the optimized graph convolutional network, the nodes and The predicted probability of the existence of edges between them. Define the loss function, minimize the target value, use back propagation to adjust the parameters of the optimized graph convolutional network, and obtain the trained graph convolutional network.

[0037] S5: construct an integrated virtual reconstruction model based on the training method described in S1-S4; In a feasible embodiment, the integrated virtual reconstruction model specifically includes: a visual sensor, a soft PLC and its external modules, and simulation software; The visual sensor is used to collect the image set of the target assembly station, and transmit the image set to the soft PLC and its external modules through the uplink channel of the C&C uplink and downlink channels. The soft PLC (Software PLC) is a system based on a general computer platform that implements the control function of the traditional hardware PLC (Programmable Logic Controller) through software. It uses computer resources such as CPU, memory, hard disk, etc., as well as specific software tools to simulate the working principle of PLC, thereby realizing the automatic control of industrial equipment; The soft PLC and its external modules train a graph convolution network for a target assembly station based on the training method; obtain a trained graph convolution network for the target assembly station, the trained graph convolution network for the target assembly station receives an image set of the target assembly station, obtains optimization data, and transmits the optimization data to the simulation software through an industrial network; The simulation software obtains an optimization model according to the optimization data, and obtains an adjusted assembly strategy based on the optimization model and the original model. Each production change triggers the online optimization of the simulation model, the simulation model rewriting mechanism, modifies and updates the rule set, reconstructs the configuration diagram and the system controller, and sends the optimization instructions back to the PLC. The PLC then sends instructions from the simulation model translator to the lower-level equipment controller through the downlink channel, thereby adjusting the physical configuration of the assembly node and better assembling and packaging the workpiece. Similarly, each node production change will trigger the simulation model rewriting mechanism, reconstruct the reasoning based on the difference calculation before and after the data, and integrate the reconstructed optimization model. Finally, the global model is incrementally improved according to the latest status parameters of each node, and updated to the local policy database of each node, and then optimization decisions are made to achieve intelligent incremental accumulation and reconstruction mode improvement. S6: Input the production change information into the integrated virtual reconstruction model, and output the adjustment assembly strategy.

[0038] Input production change information, such as product switching, equipment adjustment, and resource allocation, into the integrated virtual reconstruction model to obtain the adjustment assembly strategy.

[0039] Example 2 like Figure 2 As shown, a human-machine collaborative assembly system based on graph learning according to an embodiment of the present invention comprises: Construction module: constructing an integrated virtual reconstruction model based on the training method; Output module: inputs the production change information into the integrated virtual reconstruction model and outputs the adjustment assembly strategy.

[0040] The system of this embodiment is based on a human-machine collaborative assembly method based on graph learning proposed in Embodiment 1. Therefore, the options proposed in Embodiment 1 are also applicable to this embodiment and will not be described in detail here.

[0041] Example 3 An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer-stored program is executed by a processor, the method for adjusting a human-machine collaborative assembly station based on graph learning is implemented.

[0042] The working process of the present invention is: In summary, the embodiment of the present invention provides a human-machine collaborative assembly method, system and storage medium based on graph learning, which analyzes and learns nodes through graph convolutional neural networks, establishes an integrated virtual reconstruction model, can be simulated in the human-machine assembly collaborative system and the simulation system, and performs iterative optimization of new process production design and execution, effectively improving the rationality of the entire system design, and realizing the intelligent operation and maintenance of the human-machine assembly collaborative system; it also synchronizes and integrates virtual and real through the form of human-machine collaborative assembly system simulation, greatly reducing the cost of human-machine collaborative assembly stations. If the existing technology encounters process changes, environmental changes, etc., it will take up a lot of time, space and funds to adjust and adapt, and the method of the present invention is used to train the reconstruction model, simulate the adjustment and optimization, so as to optimize the production plan offline, shorten its joint debugging and testing cycle, and improve the production efficiency of the enterprise; by collecting the movement and production data of the human-machine collaborative assembly station, based on the picture data that can be used to record the physical equipment has not been tampered with, and binding with the assembly execution system, it can realize the life cycle quality control and anti-counterfeiting of the product, thereby tracking the life cycle of the assembly parts. It greatly reduces the cost of manual verification and workpiece tracking, and provides technical support for transparent monitoring of human-machine collaborative assembly.

[0043] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention. These improvements and substitutions should also be regarded as the scope of protection of the present invention.

Claims

1. A training method for a graph convolutional network for an assembly station, characterized in that: The method comprises the following steps: Acquire an image set of an assembly station, and pre-process the image set; Establishing a graph structure based on the assembly station according to the preprocessed image set, and establishing a graph convolutional network according to the graph structure; Optimizing the graph convolutional network to obtain an optimized graph convolutional network; The optimized graph convolutional network is trained, and a loss function is defined during the training process to adjust the optimized graph convolutional network to obtain a trained graph convolutional network.

2. According to claim 1, a training method for a graph convolutional network for an assembly station is characterized in that: The preprocessing includes removing missing or abnormal images in the image set and unifying the formats of the remaining images in the image set to obtain a preprocessed image set.

3. The training method of a graph convolutional network for an assembly station according to claim 1, characterized in that: The step of establishing a graph convolutional network according to the graph structure specifically includes: Acquiring an adjacency matrix and a node feature matrix based on the graph structure, and normalizing the adjacency matrix to obtain a normalized adjacency matrix; A graph convolutional network is constructed according to the normalized adjacency matrix and the node feature matrix.

4. The training method of a graph convolutional network for an assembly station according to claim 3, characterized in that: The graph convolution network is constructed according to the normalized adjacency matrix and the node feature matrix, and the construction method is as follows: in For the The node feature matrix of the layer, is the normalized adjacency matrix, For the The weight matrix of the layer, is the activation function, where the normalized adjacency matrix The calculation method is as follows: in, is the degree matrix, is the identity matrix, is the adjacency matrix.

5. The training method of a graph convolutional network for an assembly station according to claim 1, characterized in that: The optimizing the graph convolutional network specifically includes: optimizing the parameters of the graph convolutional network to obtain a first graph convolutional network, wherein the parameters include a convolution batch size, a convolution kernel size, and a number of convolutional layers; in the first graph convolutional network, using a graph attention pooling method instead of a global average pooling method to calculate node feature weights to obtain an optimized graph convolutional network.

6. The training method of a graph convolutional network for an assembly station according to claim 1, characterized in that: The loss function includes a first loss function and a second loss function, wherein the first loss function is used to evaluate the performance of the loss-optimized graph convolutional network in predicting the attribute or value of a node, and the second loss function is used to evaluate the performance of the loss-optimized graph convolutional network in predicting whether there is an edge between two nodes, wherein the first loss function is as follows: in, is the number of nodes in the graph structure based on the target assembly station, is the true value of the node, is the predicted value of the node; The second loss function is as follows: in, is the element of the real adjacency matrix, representing the node and Does the edge between them exist? For the optimized graph convolutional network, the nodes and The predicted probability that there is an edge between them.

7. A method for adjusting a human-machine collaborative assembly station based on graph learning, the method for adjusting a human-machine collaborative assembly station based on the training method of a graph convolutional network for an assembly station according to any one of claims 1 to 6, characterized in that: include: An integrated virtual reconstruction model is constructed based on the training method; production change information is input into the virtual reconstruction model, and an adjustment assembly strategy is output.

8. The method for adjusting a human-machine collaborative assembly station based on graph learning according to claim 7, characterized in that: The integrated virtual reconstruction model specifically includes: a visual sensor, a soft PLC and its external modules, and simulation software; The visual sensor is used to collect an image set of the target assembly station; The soft PLC and its external modules train a graph convolution network for a target assembly station based on the training method; obtain the trained graph convolution network for the target assembly station, and the trained graph convolution network for the target assembly station receives an image set of the target assembly station to obtain optimization data; The simulation software obtains an optimization model according to the optimization data, and obtains an adjustment assembly strategy based on the optimization model and the original model.

9. A human-machine collaborative assembly system based on graph learning, the system is based on a human-machine collaborative assembly method based on graph learning according to any one of claims 7 to 8, characterized in that: include: Construction module: constructing an integrated virtual reconstruction model based on the training method; Output module: inputs the production change information into the integrated virtual reconstruction model and outputs the adjustment assembly strategy.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer stored program is executed by the processor, the adjustment method of the human-machine collaborative assembly station based on graph learning as described in any one of claims 7 to 8 is implemented.

Images