Robotic arm automated precision assembly method and device

Abstract

The present application discloses a method and device for automated precision assembly of a robot arm, and relates to the technical field of a robot arm. A first assembly strategy for assembling components to be assembled is generated based on object data, category information and posture information based on a reinforcement learning decision model; a second assembly strategy for assembling components to be assembled is generated based on object data, category information, posture information and environmental information based on an MPC algorithm. By weightedly fusing the first assembly strategy and the second assembly strategy, a target assembly strategy for controlling the robot arm is dynamically generated as different assembly environments change, thereby achieving adaptation to complex and changeable assembly environments, controlling the robot arm to perform assembly operations, and improving the accuracy of assembly operations.

Description

Technical Field

[0001] The present application relates to the technical field of robotic arms, and in particular to a method and device for automated precision assembly of robotic arms. Background Art

[0002] Currently, the assembly methods involved in the automated assembly systems used in industrial production processes include: using a robotic arm in conjunction with a visual system to accurately position and assemble components based on preset programs and fixed operating modes. Since the aforementioned methods mainly rely on preset programs and fixed operating modes, although the aforementioned methods can effectively perform assembly tasks with high repeatability and small variations, when applied to complex and changeable assembly environments, the accuracy of assembling components by the aforementioned methods is low. Summary of the invention

[0003] The present application provides a method and device for automated precision assembly by a robotic arm, the main purpose of which is to solve the problem of low accuracy of assembling parts when the related technology is applied in a complex and changeable assembly environment.

[0004] According to a first aspect of the present application, a method for automated precision assembly by a robotic arm is provided, comprising:

[0005] Controlling the 3D scanning device to scan the entire assembly environment to obtain multiple target point cloud data; processing the multiple target point cloud data based on the point cloud registration algorithm to obtain 3D environment data of the entire assembly environment; performing object recognition on the 3D environment data based on a pre-trained spatial target recognition model to obtain object data; the object data includes initial spatial coordinate information of the components to be assembled and spatial coordinate information of the assembly target position;

[0006] Control the industrial camera to take real-time photos of the components to be assembled, and perform real-time analysis of the target image based on the image processing module to obtain the category information and posture information of the components to be assembled;

[0007] Controlling the environmental sensor to collect environmental information of the components to be assembled, wherein the environmental information includes temperature, light intensity, and speed information of the robot arm moving the components to be assembled;

[0008] Generate a first assembly strategy for assembling the to-be-assembled components according to the object data, category information and posture information based on a reinforcement learning decision model;

[0009] Generate a second assembly strategy for assembling the to-be-assembled parts based on the object data, category information, posture information and environmental information based on the MPC algorithm;

[0010] Performing weighted fusion on the second assembly strategy and the first assembly strategy to obtain a target assembly strategy;

[0011] The robot arm is controlled to perform corresponding assembly operations on the components to be assembled according to the target assembly strategy.

[0012] Preferably, the weighted fusion of the second assembly strategy and the first assembly strategy comprises:

[0013] Based on preset evaluation indicators, the first assembly strategy and the second assembly strategy are evaluated respectively to obtain performance scores of the first assembly strategy and the second assembly strategy under specific operating conditions, respectively, where the specific operating conditions are defined by the object data, category information, posture information, and environment information;

[0014] Dynamically assigning a first weight corresponding to the first assembly strategy and a second weight corresponding to the second assembly strategy according to the performance scores of the two under specific operating conditions;

[0015] The first assembly strategy and the second assembly strategy are weighted averaged based on the first weight and the second weight to obtain the target assembly strategy.

[0016] Furthermore, the image processing module is a composite algorithm module of image segmentation and point cloud analysis;

[0017] The composite algorithm module of image segmentation and point cloud analysis performs feature extraction based on the target image to obtain target feature data and determines the category information and posture information of the to-be-assembled component according to the target feature data;

[0018] Estimate the position and posture of the component to be assembled in the world coordinate system based on a preset transformation matrix formula combined with the intrinsic parameter matrix of the industrial camera and the target image to obtain posture information of the component to be assembled, wherein the posture information includes a rotation matrix from the world coordinate system to the camera coordinate system;

[0019] The preset transformation matrix formula is:

[0020] ;

[0021] Wherein, P is the position information of the component to be assembled in the world coordinate system, R is the rotation matrix from the world coordinate system to the camera coordinate system, t is the translation vector from the world coordinate system to the camera coordinate system, K is the intrinsic parameter matrix of the camera, and the intrinsic parameter matrix includes the focal length and the principal point coordinates. T(p) represents the homogeneous coordinates of the projection point p of the component to be assembled on the target image plane, and [R|t] is an augmented matrix, which is formed by combining the rotation matrix R and the translation vector t, and is used to simultaneously represent the rotation and translation transformations.

[0022] Furthermore, the first assembly strategy for assembling the components to be assembled according to the object data, category information and posture information based on the reinforcement learning decision model includes:

[0023] The reinforcement learning decision model adopts a soft actor-critic algorithm; the soft actor-critic algorithm includes an actor network and a critic network;

[0024] A first assembly strategy for assembling the to-be-assembled parts is generated based on the actor network according to the object data, category information and posture information.

[0025] Furthermore, after controlling the robotic arm to perform corresponding assembly operations on the to-be-assembled parts according to the target assembly strategy, the robotic arm automated precision assembly method further includes:

[0026] Performing a reward evaluation on the performance of the assembly operation performed by the robot arm according to the target assembly strategy based on a reward function in a critic network to obtain a reward value;

[0027] Iteratively updating the critic network and the actor network according to the reward value to obtain an updated reinforcement learning decision model including an updated critic network and an updated actor network;

[0028] Controlling subsequent assembly operations performed by the robotic arm based on the updated reinforcement learning decision model;

[0029] The formula of the reward function is:

[0030] ;

[0031] Among them, d represents the target distance between the state of the robot arm after performing the assembly operation and the target state, Δt represents the time required to complete the assembly operation, e represents the energy consumption, the coefficient α is the weight parameter of the target distance, β is the weight parameter of the time required for the assembly operation, γ is the weight parameter of the energy consumption, α, β and γ are used to adjust the influence of each item in the reward function, and R(s,a) represents the reward value obtained after taking action a in state s.

[0032] Further, the critic network and the actor network are iteratively updated according to the reward value, and an updated reinforcement learning decision model including an updated critic network and an updated actor network is obtained, including:

[0033] Performing a value evaluation on the performance of the assembly operation performed by the robot arm according to the target assembly strategy according to the value function in the critic network to obtain a target value;

[0034] defining a loss function for the critic network based on the target value;

[0035] Calculating the target gradient of the critic network based on the loss function, and iteratively updating the network parameters of the critic network according to the target gradient based on a stochastic gradient descent algorithm to obtain the updated critic network;

[0036] Calculating a policy gradient of the actor network according to the target value, and iteratively updating a network parameter of the actor network according to the policy gradient based on a stochastic gradient descent algorithm to obtain the updated actor network;

[0037] replacing the critic network and the actor network based on the updated critic network and the updated actor network to obtain an updated reinforcement learning decision model including the updated critic network and the updated actor network;

[0038] The formula for the stochastic gradient descent algorithm is:

[0039] ;

[0040] Among them, θ represents the network parameters, η is the learning rate, ▽ θ Q(s, a|θ) is the gradient of the objective function Q(s, a|θ), which means adjusting θ to increase the target value obtained after taking action a in state s. The target value depends on the network parameters θ, θ old The network parameters before updating, θ new Indicates the updated network parameters.

[0041] Preferably, generating a second assembly strategy for assembling the to-be-assembled parts based on the MPC algorithm according to the object data, category information, posture information and environment information includes:

[0042] A first cost function based on the MPC algorithm generates a second assembly strategy for assembling the to-be-assembled parts according to the object data, category information, posture information and environmental information;

[0043] The formula for the first cost function is:

[0044] ;

[0045] Where J(x,u) is the total cost of the first cost function, which is a function of the system state sequence x and the control input sequence u, x k represents the system state at time step k, u k represents the control input at time step k, x Nrepresents the system state at any time step in the prediction horizon N. Q and R are weight matrices used to balance the state error and the cost of control actions. The Q matrix is ​​used to weight the deviation of state variables, reflecting the preference for the system state to be close to the desired target. The R matrix is ​​used to weight the control input, reflecting the preference for the economic cost or energy consumption of the control action. P is the terminal weight matrix used to evaluate the difference between the final state and the target state at the end of the prediction horizon. N is the prediction horizon. T represents the transposed matrix and min represents the minimization operation.

[0046] Furthermore, the robotic arm automated precision assembly method comprises:

[0047] Adjusting the operating parameters of the robot arm and / or the attribute parameters of the composite algorithm of image segmentation and point cloud analysis based on a second cost function to ensure the accuracy of the assembly operation, wherein the second cost function is set in a genetic algorithm or a simulated annealing algorithm;

[0048] The formula of the second cost function is:

[0049] ;

[0050] Among them, Ω represents the cost function, θ represents the parameter set, Tc, Ec and Qc represent the costs of cycle time, energy consumption and quality control respectively, λ1, λ2, λ3 are the weights for adjusting the importance of each factor, the optimization goal of cycle time Tc is to reduce the total time in each assembly cycle, the optimization goal of energy consumption Ec is to reduce the average power consumption of system operation, and the optimization of quality control Qc focuses on reducing the output ratio of defective products.

[0051] Furthermore, it also includes:

[0052] Determine whether there is an abnormality in the process of the robot arm performing the assembly operation according to the object data, category information, posture information and environmental information;

[0053] When an abnormality is detected, an alarm is issued and a maintenance plan is output.

[0054] The second aspect of the present application discloses a robotic arm automated precision assembly device, comprising:

[0055] A control unit is used to control the 3D scanning device to scan the entire assembly environment to obtain multiple target point cloud data; to control the industrial camera to take real-time photos of the components to be assembled; and to control the environmental sensor to collect environmental information of the components to be assembled, wherein the environmental information includes temperature, light intensity, and speed information of the mechanical arm moving the components to be assembled;

[0056] A processing unit, used to process the plurality of target point cloud data based on a point cloud registration algorithm to obtain 3D environment data of the entire assembly environment; used to perform object recognition on the 3D environment data based on a pre-trained spatial target recognition model to obtain object data; the object data includes initial spatial coordinate information of the components to be assembled and spatial coordinate information of the assembly target position; used to perform real-time analysis of the target image based on an image processing module to obtain category information and posture information of the components to be assembled;

[0057] A first generating unit, configured to generate a first assembly strategy for assembling the to-be-assembled components according to the object data, category information and posture information based on a reinforcement learning decision model;

[0058] A second generating unit, configured to generate a second assembly strategy for assembling the to-be-assembled components according to the object data, category information, posture information and environment information based on an MPC algorithm;

[0059] A fusion unit, used for performing weighted fusion on the second assembly strategy and the first assembly strategy to obtain a target assembly strategy;

[0060] The assembly unit is used to control the robot arm to perform corresponding assembly operations on the components to be assembled according to the target assembly strategy.

[0061] According to a third aspect of the present application, an electronic device is provided, including:

[0062] at least one processor; and

[0063] a memory communicatively connected to the at least one processor; wherein,

[0064] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the robotic arm automated precision assembly method described in the first aspect above.

[0065] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the robotic arm automated precision assembly method described in the first aspect above.

[0066] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the robotic arm automated precision assembly method as described in the first aspect above.

[0067] The beneficial effects of the present invention are as follows: by utilizing the combination of 3D scanning equipment, industrial cameras and environmental sensors, the collection of relevant information in complex and changeable assembly environments is realized; by utilizing a point cloud registration algorithm, a pre-trained space target recognition model and an image processing module, the initial spatial coordinate information of the components to be assembled in the assembly environment, the spatial coordinate information of the assembly target position, the category information of the components to be assembled and the posture information of the components to be assembled are accurately obtained, thereby providing a guarantee for the subsequent motion trajectory path planning of the robotic arm based on the above data and information; as different assembly environments change, the second assembly strategy and the first assembly strategy are dynamically weighted fused, thereby achieving adaptation to complex and changeable assembly environments; by controlling the robotic arm to perform assembly operations after weighted fusion of the first assembly strategy and the second assembly strategy, the accuracy of the assembly operation is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0069] Figure 1 A schematic flow chart of a method for automated precision assembly by a robotic arm provided in an embodiment of the present application;

[0070] Figure 2 A schematic diagram of a target image recognition process provided by an embodiment of the present application;

[0071] Figure 3 A schematic diagram of an optimization process of a reinforcement learning decision model provided in an embodiment of the present application;

[0072] Figure 4 A composition framework diagram of a robotic arm automated precision assembly device provided in an embodiment of the present application;

[0073] Figure 5 A schematic block diagram of an example electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0074] The following will refer to the attached Figure 1-5 The present application is described in detail in conjunction with the embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0075] The following detailed descriptions are all exemplary descriptions, and are intended to provide further detailed descriptions of the present application. Unless otherwise specified, all technical terms used in the present application have the same meanings as those generally understood by those skilled in the art to which the present application belongs. The terms used in the present application are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application.

[0076] In addition, the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0077] The following describes the robotic arm automated precision assembly method and device of an embodiment of the present application with reference to the accompanying drawings.

[0078] In order to at least solve the problem that the accuracy of assembling parts in the related art is low when the related art is applied in a complex and changeable assembly environment, this embodiment provides a method for automated precision assembly by a robotic arm.

[0079] Figure 1 This is a schematic diagram of a process flow of a robotic arm automated precision assembly method provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0080] Step 101, controlling a 3D scanning device to scan the entire assembly environment to obtain a plurality of target point cloud data; processing the plurality of target point cloud data based on a point cloud registration algorithm to obtain 3D environment data of the entire assembly environment; performing object recognition on the 3D environment data based on a pre-trained spatial target recognition model to obtain object data; the object data includes initial spatial coordinate information of the components to be assembled and spatial coordinate information of the assembly target position;

[0081] Controlling the environmental sensor to collect environmental information of the component to be assembled, wherein the environmental information includes temperature, light intensity, and speed information of the robot arm moving the component to be assembled;

[0082] Step 102, controlling the industrial camera to take real-time photos of the components to be assembled, and performing real-time analysis of the target images based on the image processing module to obtain category information and posture information of the components to be assembled;

[0083] Step 103, generating a first assembly strategy for assembling the to-be-assembled components according to the object data, category information and posture information based on a reinforcement learning decision model;

[0084] Step 104, generating a second assembly strategy for assembling the to-be-assembled components based on the object data, category information, posture information and environment information based on the MPC algorithm;

[0085] Step 105, performing weighted fusion on the second assembly strategy and the first assembly strategy to obtain a target assembly strategy;

[0086] Step 106 : controlling the robot arm to perform corresponding assembly operations on the components to be assembled according to the target assembly strategy.

[0087] The robotic arm automated precision assembly method and device provided in the present application control a 3D scanning device to scan the entire assembly environment to obtain multiple target point cloud data. In practical applications, multiple 3D scanning devices can be combined according to the actual assembly environment to achieve detailed scanning of the entire assembly environment; the multiple target point cloud data are processed based on a point cloud registration algorithm to obtain 3D environmental data of the entire assembly environment; object recognition is performed on the 3D environmental data based on a pre-trained spatial target recognition model to obtain object data; the object data includes initial spatial coordinate information of the components to be assembled and spatial coordinate information of the assembly target position; the industrial camera is controlled to take real-time photos of the components to be assembled, and the target image is analyzed in real time based on the image processing module to obtain category information and posture information of the components to be assembled; in practical applications, multiple industrial cameras can be used for on-site photography according to the actual requirements of the shooting angle; the environmental sensor is controlled to collect environmental information of the components to be assembled, and the environmental information includes temperature, light intensity and the The speed information of the robot arm moving the parts to be assembled, the environmental sensor includes a temperature sensor for collecting the on-site assembly environment temperature, a photosensitive sensor for collecting the on-site assembly environment light and a linear speed sensor for collecting the movement of the parts to be assembled, the linear speed sensor is set on the position area of ​​the robot arm, and the linear speed sensor cannot be set on a certain active joint arm; based on the reinforcement learning decision model, a first assembly strategy for assembling the parts to be assembled is generated according to the object data, category information and posture information; based on the MPC algorithm, a second assembly strategy for assembling the parts to be assembled is generated according to the object data, category information, posture information and environmental information; the second assembly strategy and the first assembly strategy are weightedly fused, and the second assembly strategy and the first assembly strategy are dynamically weighted fused, so as to achieve adaptation to complex and changeable assembly environments. By controlling the robot arm to perform assembly operations after weighted fusion of the first assembly strategy and the second assembly strategy, the accuracy of the assembly operation is improved.

[0088] As a refinement of the embodiment of the present application, before executing step 102, the method further includes: using a real-time digital signal processor (DSP) to preprocess the input image stream, including dynamic range compression (DRC) and Gaussian filtering. These preprocessing steps help to improve the dynamic contrast of the image and reduce noise, providing clearer visual input for subsequent image analysis.

[0089] In some embodiments, the acquisition of the target image includes: acquiring the target image based on an industrial camera capable of outputting at least 4K resolution (3840x2160 pixels), the camera having a frame rate of up to 300fps and an ultra-low exposure time of less than 1ms to capture fast-moving and tiny assembly parts. The camera uses global shutter technology to minimize motion blur and image distortion.

[0090] As a refinement of the embodiment of the present application, when performing the weighted fusion of the second assembly strategy and the first assembly strategy as described in step 105, the following implementation method may also be adopted but is not limited to, for example: the first assembly strategy and the second assembly strategy are evaluated respectively based on preset evaluation indicators, and the performance scores of the first assembly strategy and the second assembly strategy under specific operating conditions are obtained respectively. In this specific embodiment, the time t used for the entire assembly strategy of the first assembly strategy and the second assembly strategy and the angle θ of each joint arm of the robot arm are used as evaluation indicator parameters, and different ranges of time t and angle θ are preset in the system. At the same time, corresponding output evaluation scores are preset for different ranges, so as to obtain evaluation values ​​of corresponding parameters. For example, the assembly time t of the first assembly strategy under specific operating conditions is 1.5min, which falls within the range of 1-2min preset in the system, and 1-2m The evaluation score corresponding to the preset range of in is 5 points, so the performance score of the assembly time of the first assembly strategy is 5 points; the specific operating conditions are defined by the object data, category information, posture information and environmental information; the first weight corresponding to the first assembly strategy and the second weight corresponding to the second assembly strategy are dynamically allocated according to the performance scores of the two under the specific operating conditions, and the dynamic distribution of the corresponding weights of the first assembly strategy and the second assembly strategy can be calculated and allocated based on the entropy weight method; the first assembly strategy and the second assembly strategy are weighted averaged based on the first weight and the second weight, for example, the first weight and the second weight parameters are used to weighted average the assembly time t, and then the weighted assembly time t is finally obtained, and the corresponding indicator parameters of the first assembly strategy and the second assembly strategy are weighted averaged, and the target assembly strategy is finally obtained.

[0091] As a refinement of the embodiment of the present application, in executing step 102, the target image is analyzed in real time based on the image processing module to obtain the category information and posture information of the parts to be assembled, and the following implementation methods may also be adopted but are limited to: the image processing module is a composite algorithm module of image segmentation and point cloud analysis;

[0092] The composite algorithm module of image segmentation and point cloud analysis performs feature extraction based on the target image to obtain target feature data and determines the category information and posture information of the to-be-assembled component according to the target feature data;

[0093] Estimate the position and posture of the component to be assembled in the world coordinate system based on a preset transformation matrix formula combined with the intrinsic parameter matrix of the industrial camera and the target image to obtain posture information of the component to be assembled, wherein the posture information includes a rotation matrix from the world coordinate system to the camera coordinate system;

[0094] The preset transformation matrix formula is:

[0095] ;

[0096] Wherein, P is the position information of the component to be assembled in the world coordinate system, R is the rotation matrix from the world coordinate system to the camera coordinate system, t is the translation vector from the world coordinate system to the camera coordinate system, K is the intrinsic parameter matrix of the camera, and the intrinsic parameter matrix includes the focal length and the principal point coordinates. T(p) represents the homogeneous coordinates of the projection point p of the component to be assembled on the target image plane, and [R|t] is an augmented matrix, which is formed by combining the rotation matrix R and the translation vector t, and is used to simultaneously represent the rotation and translation transformations.

[0097] In some embodiments, extracting features from the target image includes: extracting features from the target image based on an improved deep convolutional neural network (CNN), wherein the network structure includes multiple residual learning modules and batch normalization layers, which optimizes the ability to extract complex component features from the image. The network uses LeakyReLU as an activation function to enhance the performance of the model in nonlinear learning.

[0098] As a refinement of the embodiment of the present application, when executing step 103 of generating a first assembly strategy for assembling the to-be-assembled parts according to the object data, category information and posture information based on the reinforcement learning decision model, the following implementation methods may also be adopted but are limited to: the reinforcement learning decision model adopts a soft actor-critic algorithm; the soft actor-critic algorithm includes an actor network and a critic network;

[0099] A first assembly strategy for assembling the to-be-assembled parts is generated based on the actor network according to the object data, category information and posture information.

[0100] As a refinement of the above embodiment, after executing step 106 to control the robotic arm to perform the corresponding assembly operation on the to-be-assembled parts according to the target assembly strategy, the precision assembly method may also adopt but is not limited to the following implementation methods, including: rewarding and evaluating the performance of the assembly operation performed by the robotic arm according to the target assembly strategy based on the reward function in the critic network to obtain a reward value; iteratively updating the critic network and the actor network according to the reward value to obtain an updated reinforcement learning decision model including an updated critic network and an updated actor network; and controlling the subsequent assembly operation performed by the robotic arm based on the updated reinforcement learning decision model;

[0101] The formula of the reward function is:

[0102] ;

[0103] Wherein, d represents the target distance between the state of the robot arm after performing the assembly operation and the target state, Δt represents the time required to complete the assembly operation, e represents the energy consumption, coefficient α is the weight parameter of the target distance, β is the weight parameter of the time required for the assembly operation, γ is the weight parameter of the energy consumption, α, β and γ are used to adjust the influence of each item in the reward function, and R(s,a) represents the s The reward value obtained after taking action a in the state.

[0104] As a refinement of the above embodiment, when performing the iterative update of the critic network and the actor network according to the reward value to obtain the updated reinforcement learning decision model including the updated critic network and the updated actor network, the method may also adopt but is not limited to the following implementation methods, including: evaluating the performance of the assembly operation performed by the robot arm according to the target assembly strategy according to the value function in the critic network to obtain the target value; defining the loss function of the critic network based on the target value; calculating the target gradient of the critic network based on the loss function, and iteratively updating the network parameters of the critic network according to the target gradient based on the stochastic gradient descent algorithm to obtain the updated critic network; calculating the policy gradient of the actor network according to the target value and iteratively updating the network parameters of the actor network according to the policy gradient based on the stochastic gradient descent algorithm to obtain the updated actor network; replacing the critic network and the actor network based on the updated critic network and the updated actor network to obtain the updated reinforcement learning decision model including the updated critic network and the updated actor network;

[0105] The formula for the stochastic gradient descent algorithm is:

[0106] ；

[0107] Among them, θ represents the network parameters, η is the learning rate, ▽ θ Q(s, a|θ) is the gradient of the objective function Q(s, a|θ), which means adjusting θ to increase the target value obtained after taking action a in state s. The target value depends on the network parameters θ, θ old The network parameters before updating, θ new Indicates the updated network parameters.

[0108] The above method enables the robot arm to self-learn and adjust in a constantly changing assembly environment, thereby improving operation accuracy and efficiency.

[0109] As a refinement of the above embodiment, when executing step 104, the second assembly strategy for assembling the components to be assembled is generated based on the MPC algorithm according to the object data, category information, posture information and environmental information, the precision assembly method can also adopt but is not limited to the following implementation methods, including: generating a second assembly strategy for assembling the components to be assembled based on the MPC algorithm according to the object data, category information, posture information and environmental information; the formula of the first cost function is:

[0110] ；

[0111] Wherein, J(x,u) is the total cost of the first cost function, which is a function of the system state sequence x and the control input sequence u, x k represents the system state at time step k, u k represents the control input at time step k, x N represents the system state at any time step in the prediction horizon N. Q and R are weight matrices used to balance the state error and the cost of control actions. The Q matrix is ​​used to weight the deviation of state variables, reflecting the preference for the system state to be close to the desired target. The R matrix is ​​used to weight the control input, reflecting the preference for the economic cost or energy consumption of the control action. P is the terminal weight matrix used to evaluate the difference between the final state and the target state at the end of the prediction horizon. N is the prediction horizon. T represents the transposed matrix and min represents the minimization operation.

[0112] The above method can realize real-time response to various changes in the assembly process, including positioning errors of the components to be assembled, changes in assembly speed and the influence of environmental factors, thereby ensuring efficient and accurate assembly operations.

[0113] In some embodiments, an event-driven architecture is used to ensure that data exchange between 3D scanners, industrial cameras, and robotic arm control systems is delay-free and efficient. The framework uses a high-speed data bus and priority message queues to manage sensor data and control instructions to ensure real-time response to critical operations.

[0114] In some embodiments, the precision assembly method may also adopt, but is not limited to, the following methods, including: adjusting the operating parameters of the robot arm and / or the attribute parameters of the composite algorithm of image segmentation and point cloud analysis based on a second cost function to ensure the accuracy of the assembly operation, wherein the second cost function is set in a genetic algorithm or a simulated annealing algorithm;

[0115] The formula of the second cost function is:

[0116] ；

[0117] Among them, Ω represents the cost function, θ represents the parameter set, and T c 、E c and Q c Represent the cycle time, energy consumption and quality control costs respectively. λ1, λ2, λ3 are the weights for adjusting the importance of each factor. The cycle time T c The optimization goal is to reduce the total time and energy consumption E in each assembly cycle. c The optimization goal is to reduce the average power consumption of the system operation, and the quality control Q c The optimization focuses on reducing the output proportion of defective products.

[0118] In some embodiments, the precision assembly method may also adopt but is not limited to the following methods, including: judging whether there is any abnormality in the process of the assembly operation performed by the robotic arm based on object data, category information, posture information and environmental information; if it is determined that there is an abnormality, executing an alarm and outputting a maintenance plan.

[0119] In some embodiments, the temperature and light sensors are used as feedback data to adjust the motion parameters of the robot arm and the image processing parameters in the composite algorithm module of image segmentation and point cloud analysis to adapt to changes in the production environment. For example, the light sensor data can be used to adjust the exposure and gain settings of the image to maintain image quality.

[0120] In order to facilitate the understanding of the above embodiments, this embodiment provides an exemplary description, including: raw materials and components are fed into the production line through an automated feeding system. The high degree of automation at this stage ensures that the materials arrive at the assembly starting point accurately at the predetermined time and order, reducing manpower requirements and error rates. High-resolution industrial cameras capture images of components on the assembly line, and image data is transmitted to the processing unit for high-speed processing. The processing steps include image preprocessing, feature extraction, and complex object recognition, all of which are completed through deep learning models to ensure high-precision component detection. Based on the data provided by the composite algorithm module of image segmentation and point cloud analysis, the reinforcement learning algorithm optimizes the action strategy of the robot arm. In this step, the algorithm continuously learns and adjusts by evaluating the effect of each action to achieve more accurate component handling and assembly. The system adjusts operating parameters in real time based on data collected from environmental sensors. Through model predictive control technology (MPC), the target system predicts possible future state changes and adjusts the action strategy of the robot arm accordingly, enhancing the adaptability and flexibility of the target system. All system components are connected through a high-speed data bus to ensure fast and synchronous transmission of data and control commands. Performance optimization is done through regularly running algorithms (genetic algorithms or simulated annealing algorithms), which adjust the target system settings based on historical performance data to optimize overall work efficiency and stability. The assembled parts are moved to the next production stage or quality inspection area through an automated conveyor system. This link is also highly automated to ensure smooth output and quality control of the product. Through the advanced user interface, the operator can monitor the entire assembly process in real time, adjust parameters and quickly respond to any system prompts. The interface design is user-friendly and supports multiple languages ​​to adapt to the needs of different operators and improve the availability and maintenance efficiency of the target system.

[0121] Figure 2A flowchart of a target image recognition process provided for an embodiment of the present application includes: the whole process from image capture to decision support output. This flowchart deeply shows how the image processing module processes and analyzes images captured from the assembly line, including image preprocessing, feature extraction, target recognition and classification, and final data output. Each step uses cutting-edge image processing technology and deep learning algorithms to ensure that the image processing module can accurately and quickly identify and classify various assembly parts, providing precise visual guidance for robot arm operations. The process begins with a high-resolution industrial camera capturing real-time images. The camera is configured to support high dynamic range and global shutter functions to ensure that the image quality remains high even under fast-moving or changing light conditions; then the image is preprocessed by a digital signal processor, mainly with noise suppression and illumination compensation, to optimize the image quality. Specifically, a real-time digital signal processor (DSP) is used to perform preliminary processing on the image, including noise suppression, illumination compensation and color correction, to prepare high-quality image data for subsequent deep processing. Subsequently, the feature extraction stage uses a convolutional neural network to identify key features in the image, providing a basis for target recognition in the deep learning model. Specifically, a convolutional neural network (CNN) is used to extract features from the preprocessed image, and the network structure is optimized to identify and parse complex component shapes and surface features. In the target recognition and classification step, the image processing module uses the trained deep learning model - improved Faster R-CNN, for accurate target recognition and classification. The model can distinguish different assembly parts and their states, and identify abnormal situations that require special treatment. Finally, the image processing results and recognition data are transmitted to the control system through a high-speed data interface, providing real-time visual support for guiding the precise operation and dynamic adjustment of the robot arm, and providing decision support for the operation of the robot arm. This flowchart not only explains the technical details and algorithms used in each step, but also highlights the core role of the image processing module in the entire assembly process, demonstrating its importance in improving assembly accuracy and efficiency.

[0122] Figure 3 A schematic diagram of an optimization process of a reinforcement learning decision model provided in an embodiment of the present application, wherein the process is implemented based on a reinforcement learning algorithm. Figure 3The complete operation cycle from environmental state assessment to state data processing, decision-making of reinforcement learning models, action execution, and performance feedback collection and model adjustment is described in detail. The process highlights the application of the Soft Actor-Critic (SAC) algorithm in optimizing the robot arm's actions, where the decision model is calculated in a GPU-accelerated environment to ensure fast response and high-precision execution. The operation execution accuracy reaches ±0.0005 mm, and the repeatability error is kept below 0.0001 mm, showing the high accuracy and reliability of the system. The flowchart also includes annotations on technical details and performance indicators, such as decision latency, operation accuracy, and environmental adaptability, which are critical to understanding how the target system adjusts and optimizes operations in real time. Through real-time monitoring and data-driven feedback learning mechanisms, the target system can continuously improve its performance and adapt to changing production needs and environmental conditions.

[0123] In summary, the embodiments of the present application can achieve the following effects:

[0124] By combining 3D scanning equipment, industrial cameras and environmental sensors, the collection of relevant information in complex and changeable assembly environments is realized; by using point cloud registration algorithms, pre-trained spatial target recognition models and image processing modules, the initial spatial coordinate information of the parts to be assembled in the assembly environment, the spatial coordinate information of the assembly target position, the category information of the parts to be assembled and the posture information of the parts to be assembled are accurately obtained, which provides a guarantee for the subsequent motion trajectory path of the robotic arm based on the above; as different assembly environments change, the second assembly strategy and the first assembly strategy are dynamically weighted fused to achieve adaptation to complex and changeable assembly environments; by controlling the robotic arm to perform assembly operations after weighted fusion of the first assembly strategy and the second assembly strategy, the accuracy of the assembly operation is improved.

[0125] Corresponding to the above-mentioned robot arm automated precision assembly method, the present application also proposes a robot arm automated precision assembly device. Since the device embodiment of the present application corresponds to the above-mentioned method embodiment, the details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in this application.

[0126] Figure 4 A schematic diagram of a framework of a robotic arm automated precision assembly device provided in an embodiment of the present application, such as Figure 4 As shown, including:

[0127] The control unit 21 is used to control the 3D scanning device to scan the entire assembly environment to obtain multiple target point cloud data; to control the industrial camera to take real-time photos of the components to be assembled; and to control the environmental sensor to collect environmental information of the components to be assembled, wherein the environmental information includes temperature, light intensity, and speed information of the robotic arm moving the components to be assembled.

[0128] The processing unit 22 is used to process the multiple target point cloud data based on the point cloud registration algorithm to obtain 3D environment data of the entire assembly environment; to perform object recognition on the 3D environment data based on a pre-trained spatial target recognition model to obtain object data; the object data includes initial spatial coordinate information of the parts to be assembled and spatial coordinate information of the assembly target position; and to perform real-time analysis of the target image based on the image processing module to obtain category information and posture information of the parts to be assembled.

[0129] The first generating unit 23 is used to generate a first assembly strategy for assembling the to-be-assembled components according to the object data, category information and posture information based on a reinforcement learning decision model.

[0130] The second generating unit 24 is used to generate a second assembly strategy for assembling the to-be-assembled parts based on the MPC algorithm according to the object data, category information, posture information and environment information.

[0131] The fusion unit 25 is used to generate a second assembly strategy for assembling the components to be assembled based on the MPC algorithm according to the object data, category information, posture information and environment information.

[0132] The assembly unit 26 is used to control the robot arm to perform corresponding assembly operations on the components to be assembled according to the target assembly strategy.

[0133] The robotic arm automated precision assembly device provided by the present application controls a 3D scanning device to scan the entire assembly environment to obtain multiple target point cloud data; processes the multiple target point cloud data based on a point cloud registration algorithm to obtain 3D environmental data of the entire assembly environment; performs object recognition on the 3D environmental data based on a pre-trained spatial target recognition model to obtain object data; the object data includes initial spatial coordinate information of the components to be assembled and spatial coordinate information of the assembly target position; controls an industrial camera to take real-time photos of the components to be assembled, performs real-time analysis of the target image based on an image processing module to obtain category information and posture information of the components to be assembled; controls environmental sensors The device collects environmental information of the parts to be assembled, wherein the environmental information includes temperature, light intensity, and speed information of the robot arm moving the parts to be assembled; based on the reinforcement learning decision model, a first assembly strategy for assembling the parts to be assembled is generated according to the object data, category information, and posture information; based on the MPC algorithm, a second assembly strategy for assembling the parts to be assembled is generated according to the object data, category information, posture information, and environmental information; the second assembly strategy and the first assembly strategy are weightedly fused to obtain a target assembly strategy; and according to the target assembly strategy, the robot arm is controlled to perform corresponding assembly operations on the parts to be assembled. Compared with the related art, a first assembly strategy for assembling the parts to be assembled is generated based on the object data, category information and posture information based on the reinforcement learning decision model; and a second assembly strategy for assembling the parts to be assembled is generated based on the object data, category information, posture information and environmental information based on the MPC algorithm, so that the assembly strategy for controlling the robotic arm is dynamically generated as different assembly environments change, thereby achieving adaptation to complex and changeable assembly environments, and controlling the robotic arm to perform assembly operations by weighted fusion of the first assembly strategy and the second assembly strategy, thereby improving the accuracy of the assembly operation.

[0134] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0135] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0136] Figure 5A schematic block diagram of an example electronic device 300 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0137] like Figure 5 As shown, the device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 302 or a computer program loaded from a storage unit 308 to a RAM (Random Access Memory) 303. Various programs and data required for the operation of the device 300 can also be stored in the RAM 303. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An I / O (Input / Output) interface 305 is also connected to the bus 304.

[0138] A number of components in the device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0139] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as a robotic arm automated precision assembly method. For example, in some embodiments, the robotic arm automated precision assembly method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to execute the aforementioned robotic arm automated precision assembly method in any other appropriate manner (eg, by means of firmware).

[0140] Various embodiments of the systems and techniques described above in this article can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0142] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0144] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0145] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0146] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0147] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Claims

1. A method for automated precision assembly of a robotic arm, characterized in that: include: Control the 3D scanning equipment to scan the entire assembly environment and obtain multiple target point cloud data; Processing the plurality of target point cloud data based on a point cloud registration algorithm to obtain 3D environment data of the entire assembly environment; Performing object recognition on the 3D environment data based on a pre-trained spatial target recognition model to obtain object data; the object data includes initial spatial coordinate information of the components to be assembled and spatial coordinate information of the assembly target position; Control the industrial camera to take real-time photos of the components to be assembled, and perform real-time analysis of the target image based on the image processing module to obtain the category information and posture information of the components to be assembled; Controlling the environmental sensor to collect environmental information of the components to be assembled, wherein the environmental information includes temperature, light intensity, and speed information of the robot arm moving the components to be assembled; Generate a first assembly strategy for assembling the to-be-assembled components according to the object data, category information and posture information based on a reinforcement learning decision model; Generate a second assembly strategy for assembling the to-be-assembled parts based on the object data, category information, posture information and environmental information based on the MPC algorithm; Performing weighted fusion on the second assembly strategy and the first assembly strategy to obtain a target assembly strategy; The robot arm is controlled to perform corresponding assembly operations on the components to be assembled according to the target assembly strategy.

2. The method for automated precision assembly by a robotic arm according to claim 1, characterized in that: The weighted fusion of the second assembly strategy and the first assembly strategy includes: Based on preset evaluation indicators, the first assembly strategy and the second assembly strategy are evaluated respectively to obtain performance scores of the first assembly strategy and the second assembly strategy under specific operating conditions, respectively, where the specific operating conditions are defined by the object data, category information, posture information, and environment information; Dynamically assigning a first weight corresponding to the first assembly strategy and a second weight corresponding to the second assembly strategy according to the performance scores of the two under specific operating conditions; The first assembly strategy and the second assembly strategy are weighted averaged based on the first weight and the second weight to obtain the target assembly strategy.

3. The method for automated precision assembly by a robotic arm according to claim 2, characterized in that: The image processing module is a composite algorithm module of image segmentation and point cloud analysis; The composite algorithm module of image segmentation and point cloud analysis performs feature extraction based on the target image to obtain target feature data and determines the category information and posture information of the to-be-assembled component according to the target feature data; Estimate the position and posture of the component to be assembled in the world coordinate system based on a preset transformation matrix formula combined with the intrinsic parameter matrix of the industrial camera and the target image to obtain posture information of the component to be assembled, wherein the posture information includes a rotation matrix from the world coordinate system to the camera coordinate system; The preset transformation matrix formula is: ； Wherein, P is the position information of the component to be assembled in the world coordinate system, R is the rotation matrix from the world coordinate system to the camera coordinate system, t is the translation vector from the world coordinate system to the camera coordinate system, K is the intrinsic parameter matrix of the camera, and the intrinsic parameter matrix includes the focal length and the principal point coordinates. T(p) represents the homogeneous coordinates of the projection point p of the component to be assembled on the target image plane, and [R|t] is an augmented matrix, which is formed by combining the rotation matrix R and the translation vector t, and is used to simultaneously represent the rotation and translation transformations.

4. The method for automated precision assembly by a robotic arm according to claim 3, characterized in that: Generating a first assembly strategy for assembling the to-be-assembled parts based on the object data, category information and posture information based on the reinforcement learning decision model includes: The reinforcement learning decision model adopts a soft actor-critic algorithm; the soft actor-critic algorithm includes an actor network and a critic network; A first assembly strategy for assembling the to-be-assembled parts is generated based on the actor network according to the object data, category information and posture information.

5. The method for automated precision assembly by a robotic arm according to claim 4, characterized in that: After controlling the robotic arm to perform corresponding assembly operations on the to-be-assembled components according to the target assembly strategy, the robotic arm automated precision assembly method further includes: Performing a reward evaluation on the performance of the assembly operation performed by the robot arm according to the target assembly strategy based on a reward function in a critic network to obtain a reward value; Iteratively updating the critic network and the actor network according to the reward value to obtain an updated reinforcement learning decision model including an updated critic network and an updated actor network; Controlling subsequent assembly operations performed by the robotic arm based on the updated reinforcement learning decision model; The formula of the reward function is: ； Where d represents the target distance between the state of the robot arm after performing the assembly operation and the target state, Δt represents the time required to complete the assembly operation, e represents the energy consumption, the coefficient α is the weight parameter of the target distance, β is the weight parameter of the time required for the assembly operation, and γ is the weight parameter of the energy consumption. α, β and γ are used to adjust the influence of each item in the reward function. R(s,a) represents the s The reward value obtained after taking action a in the state.

6. The method for automated precision assembly by a robotic arm according to claim 5, characterized in that: The critic network and the actor network are iteratively updated according to the reward value, and an updated reinforcement learning decision model including an updated critic network and an updated actor network is obtained, comprising: Performing a value evaluation on the performance of the assembly operation performed by the robot arm according to the target assembly strategy according to the value function in the critic network to obtain a target value; defining a loss function for the critic network based on the target value; Calculating the target gradient of the critic network based on the loss function, and iteratively updating the network parameters of the critic network according to the target gradient based on a stochastic gradient descent algorithm to obtain the updated critic network; Calculating a policy gradient of the actor network according to the target value, and iteratively updating a network parameter of the actor network according to the policy gradient based on a stochastic gradient descent algorithm to obtain the updated actor network; replacing the critic network and the actor network based on the updated critic network and the updated actor network to obtain an updated reinforcement learning decision model including the updated critic network and the updated actor network; The formula for the stochastic gradient descent algorithm is: ； Among them, θ represents the network parameters, η is the learning rate, ▽ θ Q(s, a|θ) is the gradient of the objective function Q(s, a|θ), which means adjusting θ to increase the target value obtained after taking action a in state s. The target value depends on the network parameters θ, θ old The network parameters before updating, θ new Indicates the updated network parameters.

7. The method for automated precision assembly by a robotic arm according to any one of claims 1 to 6, characterized in that: Generating a second assembly strategy for assembling the to-be-assembled parts based on the MPC algorithm according to the object data, category information, posture information and environmental information includes: A first cost function based on the MPC algorithm generates a second assembly strategy for assembling the to-be-assembled parts according to the object data, category information, posture information and environmental information; The formula of the first cost function is: ； Where J(x,u) is the total cost of the first cost function, which is a function of the system state sequence x and the control input sequence u, x k represents the system state at time step k, u k represents the control input at time step k, x N represents the system state at any time step in the prediction horizon N. Q and R are weight matrices used to balance the state error and the cost of control actions. The Q matrix is ​​used to weight the deviation of state variables, reflecting the preference for the system state to be close to the desired target. The R matrix is ​​used to weight the control input, reflecting the preference for the economic cost or energy consumption of the control action. P is the terminal weight matrix used to evaluate the difference between the final state and the target state at the end of the prediction horizon. N is the prediction horizon. T represents the transposed matrix and min represents the minimization operation.

8. The method for automated precision assembly by a robotic arm according to claim 7, characterized in that: The robotic arm automated precision assembly method comprises: Adjusting the operating parameters of the robot arm and / or the attribute parameters of the composite algorithm of image segmentation and point cloud analysis based on a second cost function to ensure the accuracy of the assembly operation, wherein the second cost function is set in a genetic algorithm or a simulated annealing algorithm; The formula for the second cost function is: ； Among them, Ω represents the cost function, θ represents the parameter set, and T c 、E c and Q c Represent the cycle time, energy consumption and quality control costs respectively. λ1, λ2, λ3 are the weights for adjusting the importance of each factor. The cycle time T c The optimization goal is to reduce the total time and energy consumption E in each assembly cycle. c The optimization goal is to reduce the average power consumption of the system operation, and the quality control Q c The optimization focuses on reducing the output proportion of defective products.

9. The method for automated precision assembly by a robotic arm according to claim 8, characterized in that: Also includes: Determine whether there is an abnormality in the process of the robot arm performing the assembly operation according to the object data, category information, posture information and environmental information; When an abnormality is detected, an alarm is issued and a maintenance plan is output.

10. A robotic arm automated precision assembly device, characterized in that: include: A control unit is used to control the 3D scanning device to scan the entire assembly environment and obtain multiple target point cloud data; Used to control industrial cameras to take real-time photos of the parts to be assembled; Used to control the environmental sensor to collect environmental information of the components to be assembled, wherein the environmental information includes temperature, light intensity, and speed information of the robot arm moving the components to be assembled; A processing unit, used for processing the plurality of target point cloud data based on a point cloud registration algorithm to obtain 3D environment data of the entire assembly environment; Used to perform object recognition on the 3D environment data based on a pre-trained space target recognition model to obtain object data; the object data includes initial space coordinate information of the components to be assembled and space coordinate information of the assembly target position; Used to perform real-time analysis of target images based on the image processing module to obtain the category information and posture information of the parts to be assembled; A first generating unit, configured to generate a first assembly strategy for assembling the to-be-assembled components according to the object data, category information and posture information based on a reinforcement learning decision model; A second generating unit, configured to generate a second assembly strategy for assembling the to-be-assembled components according to the object data, category information, posture information and environment information based on an MPC algorithm; A fusion unit, used for performing weighted fusion on the second assembly strategy and the first assembly strategy to obtain a target assembly strategy; The assembly unit is used to control the robot arm to perform corresponding assembly operations on the components to be assembled according to the target assembly strategy.

Images