Mechanical arm grabbing control method and system based on digital twinning

Abstract

The invention discloses a mechanical arm grabbing control method and system based on digital twinning, and relates to the technical field of industrial robot intelligent control, and the mechanical arm grabbing control method comprises the steps that target object data and grabbing scene data are collected, and a digital twinning model of a target object and a grabbing scene is constructed; dynamic changes of the target object and the grabbing scene are monitored in real time, the digital twin model is dynamically corrected, and a high-precision digital twin model is generated; based on a high-precision digital twinborn model, extracting characteristics of a target object, and identifying an optimal grabbing point; generating a grabbing strategy based on the optimal grabbing point and the high-precision digital twin model, and verifying the feasibility of the grabbing strategy; and according to a verification result, dynamically executing an intelligent grabbing strategy. According to the method, the grabbing precision and stability of the mechanical arm are improved by correcting the digital twin model in real time and recognizing the optimal grabbing point.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial robot intelligent control, and in particular to a robot arm grasping control method and system based on digital twin. Background Art

[0002] In recent years, with the continuous improvement of industrial automation, robotic arms have been widely used in manufacturing, warehousing and logistics, medical and service robots, and other fields, becoming an important tool for achieving efficient production and intelligent operation. In practical applications, the grasping tasks of robotic arms usually need to be completed in complex dynamic environments, which requires the robotic arms to have high-precision path planning capabilities and real-time dynamic decision-making capabilities. Traditional robotic arm grasping control methods usually rely on predefined simple geometric models or static scene assumptions. However, these methods often show great limitations when facing the dynamic changes of target objects, the complexity of grasping scenes, and the diverse grasping requirements, and it is difficult to meet the needs of practical applications. In recent years, the rapid development of digital twin technology has provided new solutions for the grasping tasks of robotic arms. By constructing a virtual twin model of the target object and the grasping scene, and dynamically updating the twin model using real-time perception technology, the full process digital management and optimization of the robotic arm grasping task can be realized, providing important technical support for intelligent grasping in complex environments.

[0003] Although digital twin technology has shown great potential in the field of robotic arm grasping, the existing related technologies still have many shortcomings. First, when constructing digital twin models, existing technologies often only focus on the geometric shape of the target object, but do not fully consider the material properties of the object (such as friction coefficient, elastic modulus, etc.) and the dynamic changes of the grasping scene, resulting in insufficient accuracy and adaptability of the twin model. Secondly, in the process of generating and verifying grasping strategies, existing methods usually ignore the influence of the physical properties of the grasping point (such as surface roughness, normal distribution, etc.) on the grasping stability, making it difficult to ensure the reliability and safety of the grasping action. Therefore, existing technologies still face great technical challenges in grasping tasks in complex dynamic scenes, and a more efficient and intelligent grasping control method is urgently needed. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a robotic arm grasping control method based on digital twin to solve the problems of insufficient accuracy of twin models and low reliability of grasping strategies in the prior art.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, the present invention provides a robotic arm grasping control method based on digital twins, which includes collecting target object data and grasping scene data, and constructing a digital twin model of the target object and the grasping scene; monitoring the dynamic changes of the target object and the grasping scene in real time, and dynamically correcting the digital twin model to generate a high-precision digital twin model; based on the high-precision digital twin model, extracting the characteristics of the target object and identifying the optimal grasping point; based on the optimal grasping point and the high-precision digital twin model, generating a grasping strategy, and verifying the feasibility of the grasping strategy; according to the verification result, dynamically executing the intelligent grasping strategy.

[0008] As a preferred solution of the digital twin-based robotic arm grasping control method of the present invention, wherein: the target object data includes the position information, three-dimensional point cloud data and material characteristic data of the target object;

[0009] The grabbing scene data includes the spatial distribution and dynamic changes of obstacles.

[0010] As a preferred solution of the digital twin-based robotic arm grasping control method of the present invention, the specific steps of constructing a digital twin model of the target object and the grasping scene are as follows:

[0011] Based on the location information and 3D point cloud data of the target object, a 3D geometric model of the target object is constructed through a triangular mesh reconstruction algorithm;

[0012] Extracting the friction coefficient from the material property data of the target object, and matching the elastic modulus from the material database according to the material type of the target object;

[0013] Integrate the friction coefficient, elastic modulus and three-dimensional geometric model of the target object to generate a complete physical property model;

[0014] Based on the spatial distribution of obstacles in the grasping scene, a two-dimensional environment map is constructed using occupancy grid mapping. Combined with the dynamic change information of obstacles, the two-dimensional environment map is merged with the geometric model of the target object in a unified coordinate system to generate a complete scene layout model.

[0015] The three-dimensional geometric model, physical property model and scene layout model of the target object are integrated to generate a digital twin model of the target object and the grasping scene.

[0016] As a preferred solution of the digital twin-based robotic arm grasping control method of the present invention, the real-time monitoring of the dynamic changes of the target object and the grasping scene, and the dynamic correction of the digital twin model to generate a high-precision digital twin model are as follows:

[0017] Through the depth camera and RGB camera, the position change, posture change and material property change of the target object are monitored in real time;

[0018] The real-time position and dynamic change information of obstacles in the grabbing scene are collected through LiDAR;

[0019] Based on the real-time position information and posture changes of the target object, the spatial position and orientation of the 3D geometric model in the digital twin model are updated;

[0020] According to the changes in the material properties of the target object, the friction coefficient and elastic modulus of the target object are updated, and the physical property model is corrected;

[0021] According to the real-time position and dynamic change information of obstacles in the grasping scene, the position and rotation state of obstacles in the scene layout model are corrected in real time, and the occupation probability and distribution state of obstacles are updated in the occupation grid map;

[0022] A high-precision digital twin model is generated based on the corrected three-dimensional geometric model, physical property model and scene layout model.

[0023] As a preferred solution of the digital twin-based robotic arm grasping control method of the present invention, the specific steps of extracting the characteristics of the target object and identifying the optimal grasping point based on the high-precision digital twin model are as follows:

[0024] Through the three-dimensional geometric model and material property data of the target object, the surface roughness, normal distribution, friction coefficient distribution and material elastic properties of the target object are extracted using the surface property analysis algorithm;

[0025] Filter the smooth surface areas according to the surface roughness and friction coefficient distribution;

[0026] Filter surface uneven areas based on normal distribution;

[0027] Mark the graspable area by filtering the surface area of ​​the target object;

[0028] Based on the edited graspable area, combined with the surface roughness, normal distribution, friction coefficient distribution and material elastic properties of the target object, the grasping stability score of each grasping area is calculated, and the expression is:

[0029]

[0030] Among them, μ f is the friction coefficient distribution value of the grasping point, α is the nonlinear weight coefficient of the friction coefficient, E is the elastic modulus of the target object, A c is the area of ​​contact between the robot and the target object, A tis the total surface area of ​​the graspable area of ​​the target object, Δφ is the normal deviation angle of the grasping point, κ is the nonlinear adjustment coefficient of the normal deviation angle, d is the distance from the grasping point to the center of mass of the target object, λ is the adjustment coefficient of the distance offset, and σ s is the standard deviation of surface roughness in the contact area, γ is the attenuation coefficient of roughness uniformity, η is the nonlinear adjustment coefficient of contact area standardization, and Q is the grasping stability score;

[0031] Select the grasping point with the highest grasping stability score Q in the marked graspable area as the best grasping point P g .

[0032] As a preferred solution of the digital twin-based robotic arm grasping control method of the present invention, the grasping strategy is generated based on the optimal grasping point and the high-precision digital twin model, and the feasibility of the grasping strategy is verified. The specific steps are as follows:

[0033] Based on the high-precision digital twin model of the target object and the optimal grasping point P g Based on the gradient optimization method, the path smoothness, grasping accuracy and obstacle avoidance ability are comprehensively considered to identify the configuration parameters of the optimal grasping path of the robot arm. The expression is:

[0034]

[0035] Among them, q * (t) is the configuration parameter of the optimal grasping path of the robot arm at time t, q 0 is the initial joint angle configuration parameter of the robot, t 0 is the starting time of path planning, t is the current time, p(t') is the spatial position of the end of the robot at time t', Indicates the rate of change of the robot arm joint configuration to the obstacle distance, represents the gradient of the path smoothing term, q(t') is the joint angle configuration parameter of the robot at time t', and t' represents the starting time t of path planning 0 At any time between t and the current time t, ρ is the weight coefficient for dynamically adjusting the attraction of the grasping point, and β is the weight coefficient for dynamically adjusting the obstacle avoidance strength;

[0036] Configuration parameters q of the optimal grasping path of the robot arm * (t), generate the optimal grasping path of the robot arm;

[0037] According to the friction coefficient, contact area, elastic modulus and roughness uniformity of the grasping point, the optimal clamping force distribution of the robot gripper at the grasping point is calculated, and the expression is:

[0038]

[0039] Among them, F c is the optimal gripping force of the robot arm at the grasping point, m is the mass of the target object, g is the gravitational acceleration, σ max Indicates the maximum allowable value of roughness, k e is the influence coefficient of elastic modulus;

[0040] Based on the optimal grasping path of the robot arm and the optimal gripping force F of the robot arm at the grasping point c , generate a crawling strategy;

[0041] Execute the grasping strategy in the high-precision digital twin model, simulate the grasping process, and verify the optimal grasping path and the optimal gripping force F of the robot arm at the grasping point c feasibility;

[0042] In the simulated grasping process, if the target object slips, deforms, or the robot arm collides with an obstacle, the current grasping strategy is considered infeasible;

[0043] In the simulated grasping process, if the target object does not slip or deform, and the robot arm does not collide with obstacles, the current grasping strategy is considered feasible.

[0044] As a preferred solution of the digital twin-based robotic arm grasping control method of the present invention, wherein: the intelligent grasping strategy is dynamically executed according to the verification result, and the specific steps are as follows:

[0045] When the current crawling strategy is feasible, execute the current crawling strategy and record the feedback data during the actual execution process;

[0046] When the current crawling strategy is not feasible, the current crawling strategy is not executed, the feedback data during the simulated crawling process is recorded, and based on the feedback data during the simulated crawling process, the current crawling strategy is optimized and re-verified until the verification result of the current crawling strategy is feasible.

[0047] In the second aspect, the present invention provides a robotic arm grasping control system based on digital twins, including a twin model construction module, a twin model correction module, a grasping point identification module, a strategy generation module and a strategy execution module; the twin model construction module is used to collect target object data and grasping scene data, and construct a digital twin model of the target object and the grasping scene; the twin model correction module is used to monitor the dynamic changes of the target object and the grasping scene in real time, and dynamically correct the digital twin model to generate a high-precision digital twin model; the grasping point identification module is used to extract the characteristics of the target object based on the high-precision digital twin model and identify the optimal grasping point; the strategy generation module is used to generate a grasping strategy based on the optimal grasping point and the high-precision digital twin model, and verify the feasibility of the grasping strategy; the strategy execution module is used to dynamically execute the intelligent grasping strategy according to the verification result.

[0048] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the digital twin-based robotic arm grasping control method as described in the first aspect of the present invention is implemented.

[0049] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the digital twin-based robotic arm grasping control method as described in the first aspect of the present invention.

[0050] The beneficial effects of the present invention are as follows: by real-time correction of the digital twin model and identification of the best grasping point, the accuracy and stability of the robot arm grasping are improved. The real-time correction of the twin model dynamically adapts to scene changes and enhances flexibility and reliability; the best grasping point identification comprehensively evaluates the grasping stability and optimizes the grasping safety and success rate. Combined with the generation and verification of the grasping strategy, the whole process is optimized, the problems of insufficient model accuracy and low grasping reliability are solved, and it is suitable for complex dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0052] Figure 1 This is a flow chart of the robotic arm grasping control method based on digital twin in Example 1.

[0053] Figure 2 Schematic diagram of the robotic arm grasping control system based on digital twin in Example 1. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0055] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0056] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0057] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a robot arm grasping control method based on digital twin, comprising the following steps:

[0058] S1: Collect target object data and grasping scene data, and build a digital twin model of the target object and grasping scene.

[0059] S1.1: The target object data includes the location information, three-dimensional point cloud data and material property data of the target object.

[0060] Furthermore, the position information is used to determine the specific position of the target object in space, the three-dimensional point cloud data is used to construct a geometric model of the target object, and the material property data (such as friction coefficient and elastic modulus) is used to describe the physical properties of the target object to support the accurate identification of the grasping point and the optimization of the grasping strategy.

[0061] S1.2: The captured scene data includes the spatial distribution and dynamic changes of obstacles.

[0062] Furthermore, the spatial distribution of obstacles is used to construct a two-dimensional layout map of the environment to help identify the feasible motion path of the robot arm; the dynamic change information is used to update the position, rotation state and occupancy probability of the obstacles in real time to ensure that the grasping strategy can dynamically avoid obstacles, thereby improving the safety and reliability of grasping tasks in complex scenarios.

[0063] S1.3: Based on the position information and 3D point cloud data of the target object, a 3D geometric model of the target object is constructed by a triangular mesh reconstruction algorithm;

[0064] Specifically, the reference coordinate system of the target object in space is first determined using the position information, and then the collected 3D point cloud data is preprocessed, including denoising and smoothing operations, to improve data quality. Next, a triangular mesh reconstruction algorithm is used to topologically connect the processed point cloud data to generate a continuous surface mesh structure, thereby constructing a 3D geometric model of the target object. This model can accurately describe the external shape of the target object and provide basic support for the subsequent recognition of grasping points and strategy generation.

[0065] S1.4: extracting the friction coefficient from the material property data of the target object, and matching the elastic modulus from the material database according to the material type of the target object;

[0066] Specifically, the friction coefficient of the target object surface is first extracted through the material recognition algorithm. The friction coefficient is used to reflect the contact friction characteristics between the target object surface and the robotic arm. Then, according to the extracted material type information (such as metal, plastic or wood, etc.), it is matched in the material database to obtain the elastic modulus corresponding to the material type of the target object. The elastic modulus is used to describe the deformation characteristics of the target object under force conditions. This process provides key physical parameter support for the mechanical analysis of the grasping point and the optimization of the grasping strategy.

[0067] S1.5: Integrate the friction coefficient, elastic modulus, and three-dimensional geometric model of the target object to generate a complete physical property model;

[0068] Specifically, the geometric morphology model is first used as the basic framework to describe the external shape and spatial distribution of the target object; the friction coefficient is then mapped to the surface characteristics of the geometric model to reflect the friction distribution of different surface areas; finally, the elastic modulus data is combined to supplement the material deformation characteristics of the target object under stress conditions to form a complete physical property model that includes geometric, friction and elastic properties. This model comprehensively describes the mechanical behavior of the target object and provides an accurate physical basis for the selection of grasping points and the generation of grasping strategies.

[0069] S1.6: Based on the spatial distribution of obstacles in the grasping scene, a two-dimensional environment map is constructed using occupancy grid mapping. Combined with the dynamic change information of obstacles, the two-dimensional environment map is merged with the geometric model of the target object in a unified coordinate system to generate a complete scene layout model.

[0070] Furthermore, the position and shape data of obstacles are first collected using a laser radar or a depth camera, and the scene is divided into two-dimensional grid cells with occupation states through the occupancy grid mapping method to construct a two-dimensional environment map reflecting the distribution of obstacles. At the same time, combined with the dynamic change information of the obstacles (such as position changes or rotation states), the occupancy probability and state of each cell in the grid map are updated in real time. Subsequently, the generated two-dimensional environment map is aligned with the three-dimensional geometric model of the target object, and the two are fused through a unified reference coordinate system, so that the target object and the obstacle can be accurately positioned in a unified space, and finally a complete scene layout model is generated, providing a reliable environment description for path planning and grasping strategy generation.

[0071] S1.7: Integrate the three-dimensional geometric model, physical property model and scene layout model of the target object to generate a digital twin model of the target object and the grasping scene.

[0072] Furthermore, the three-dimensional point cloud data of the target object is converted into an accurate three-dimensional geometric model through a triangular mesh reconstruction algorithm to describe its shape and size; secondly, the friction coefficient and elastic modulus are extracted from the material property data and combined with the geometric model to generate a physical property model of the target object to describe its contact characteristics and mechanical properties; then, lidar and occupancy grid mapping technology are used to generate a two-dimensional environmental map of obstacles in the grasping scene, and combined with dynamic change information, a scene layout model is constructed to describe the distribution and dynamic characteristics of obstacles; finally, the three-dimensional geometric model, physical property model and scene layout model of the target object are fused in a unified coordinate system to form a complete digital twin model of the target object and the grasping scene, providing a high-precision virtual environment foundation for the generation and verification of grasping strategies.

[0073] S2: Monitor the dynamic changes of the target object and the grasping scene in real time, and dynamically correct the digital twin model to generate a high-precision digital twin model.

[0074] S2.1: Use the depth camera and RGB camera to monitor the position, posture and material property changes of the target object in real time;

[0075] Furthermore, the position change of the target object includes the real-time position information change of the target object in space, which is used to update its coordinates and reference position in the digital twin model.

[0076] The posture change of the target object includes the rotation angle and direction change of the target object, which is used to adjust the orientation and posture distribution of its three-dimensional geometric model in the scene.

[0077] Changes in the material properties of the target object include dynamic changes in physical properties such as the friction coefficient and elastic modulus of the target object's surface, which are used to modify its physical property model to adapt to actual grasping requirements.

[0078] S2.2: Use LiDAR to collect real-time position and dynamic change information of obstacles in the grasping scene;

[0079] S2.3: Based on the real-time position information and posture changes of the target object, update the spatial position and orientation of the three-dimensional geometric model in the digital twin model;

[0080] For example, when the target object moves or rotates due to external force, the coordinate position and rotation matrix of the three-dimensional geometric model in the digital twin model are adjusted by collecting its position information and posture change data in real time, so that it accurately reflects the current spatial state of the target object.

[0081] S2.4: updating the friction coefficient and elastic modulus of the target object and correcting the physical property model according to the change of the material properties of the target object;

[0082] For example, when the friction coefficient of the surface of the target object decreases due to moisture or contamination, or the elastic modulus of the material changes due to temperature changes, these characteristic changes are detected in real time, the friction coefficient and elastic modulus parameters are updated, and the physical property model is corrected to accurately reflect the mechanical properties of the current object.

[0083] S2.5: According to the real-time position and dynamic change information of obstacles in the grasping scene, the position and rotation state of obstacles in the scene layout model are corrected in real time, and the occupation probability and distribution state of obstacles are updated in the occupation grid map;

[0084] For example, when an obstacle in the grasping scene changes its position or rotation angle due to the movement of mechanical equipment, the lidar collects its position and posture change data in real time, adjusts the coordinates and direction of the obstacle in the scene layout model, and updates the occupancy probability and distribution area of ​​the obstacle in the occupancy grid map to ensure that the digital twin model accurately reflects the current scene status.

[0085] S2.6: Generate a high-precision digital twin model based on the revised three-dimensional geometric model, physical property model and scene layout model.

[0086] Furthermore, based on the real-time monitoring data (such as the position information and posture changes of the target object captured by the depth camera and RGB camera), the spatial position and direction of the three-dimensional geometric morphology model are dynamically updated to ensure that its morphology model accurately reflects the real-time status; secondly, according to the changes in the material properties of the target object (such as changes in the friction coefficient and elastic modulus), the contact characteristics and mechanical characteristics parameters in the physical property model are dynamically corrected to ensure the accuracy of the physical model; then, the position, rotation state and occupancy probability of the obstacles in the scene layout model are updated through the real-time position and dynamic change information of the obstacles collected by the lidar to ensure that the scene layout reflects the latest status of the grasping environment; finally, the corrected three-dimensional geometric morphology model, physical property model and scene layout model are integrated in a unified coordinate system to form a high-precision, dynamically updated digital twin model, which provides real-time and accurate data support for subsequent grasping point identification and strategy generation.

[0087] S3: Based on the high-precision digital twin model, the characteristics of the target object are extracted and the optimal grasping point is identified.

[0088] S3.1: Using the three-dimensional geometric model and material property data of the target object, a surface property analysis algorithm is used to extract the surface roughness, normal distribution, friction coefficient distribution and material elastic properties of the target object;

[0089] Specifically, the surface of the target object is first segmented and meshed to accurately describe its surface morphology. Then, the surface roughness is calculated point by point using the surface characteristic analysis algorithm, and the normal distribution characteristics are extracted by analyzing the distribution of the grid normal vector. At the same time, the local distribution of the friction coefficient is obtained by combining the material characteristic data, and the elastic modulus of the material is calculated according to the material type and database matching. Finally, these characteristic data are mapped to the surface mesh of the target object to form a comprehensive description of its surface physical characteristics, providing key support for grasping point identification and mechanical analysis.

[0090] S3.2: Filter the smooth surface area based on the surface roughness and friction coefficient distribution;

[0091] For example, when the roughness of certain areas on the surface of the target object is low and the friction coefficient is less than a set threshold, these areas are identified as too smooth and unsuitable for grasping and are removed from the graspable area.

[0092] S3.3: Filter surface uneven areas based on normal distribution;

[0093] For example, when the normal vectors of certain areas on the target object’s surface change dramatically (such as edges or sharp protrusions), these areas are identified as uneven and removed to avoid unstable grasping.

[0094] S3.4: Mark the graspable area through the surface area of ​​the filtered target object;

[0095] For example, after the above filtering, the area of ​​the target object with a relatively flat surface and an appropriate friction coefficient is marked as a graspable area, providing a basis for the subsequent grasping point selection.

[0096] S3.5: Based on the edited graspable area, combined with the surface roughness, normal distribution, friction coefficient distribution and material elastic properties of the target object, the grasping stability score of each graspable area is calculated, and the expression is:

[0097]

[0098] Among them, μ f is the friction coefficient distribution value of the grasping point, α is the nonlinear weight coefficient of the friction coefficient, E is the elastic modulus of the target object, A c is the area of ​​contact between the robot and the target object, A t is the total surface area of ​​the graspable area of ​​the target object, Δφ is the normal deviation angle of the grasping point, κ is the nonlinear adjustment coefficient of the normal deviation angle, d is the distance from the grasping point to the center of mass of the target object, λ is the adjustment coefficient of the distance offset, and σ sis the standard deviation of surface roughness in the contact area, γ is the attenuation coefficient of roughness uniformity, η is the nonlinear adjustment coefficient of contact area standardization, and Q is the grasping stability score;

[0099] Select the grasping point with the highest grasping stability score Q in the marked graspable area as the best grasping point P g .

[0100] It should be noted that the grasping stability score Q of each graspable area is calculated based on the surface roughness, normal distribution, friction coefficient distribution and material elastic properties of the graspable area, and the grasping point with the highest score is selected as the optimal grasping point P g , achieving scientific and precise grasping point selection. The scoring formula comprehensively considers friction, material elasticity, contact area, normal deviation, surface roughness, and the relationship between the grasping point position and the center of mass of the target object, and quantifies the stability of the grasping point in multiple dimensions. By selecting the point with the highest grasping stability score, the robot arm's ability to adapt to uncertain factors during the grasping process is significantly improved, the grasping accuracy, stability and success rate are enhanced, and the problem of object slipping, grasping failure or damage is effectively avoided.

[0101] S4: Generate a grasping strategy based on the optimal grasping point and high-precision digital twin model, and verify the feasibility of the grasping strategy.

[0102] S4.1: Based on the high-precision digital twin model of the target object and the optimal grasping point P g Based on the gradient optimization method, the path smoothness, grasping accuracy and obstacle avoidance ability are comprehensively considered to identify the configuration parameters of the optimal grasping path of the robot arm. The expression is:

[0103]

[0104] Among them, q * (t) is the configuration parameter of the optimal grasping path of the robot arm at time t, q 0 is the initial joint angle configuration parameter of the robot, t 0 is the starting time of path planning, t is the current time, p(t') is the spatial position of the end of the robot at time t', Indicates the rate of change of the robot arm joint configuration to the obstacle distance, represents the gradient of the path smoothing term, q(t') is the joint angle configuration parameter of the robot at time t', and t' represents the starting time t of path planning 0 At any time between t and the current time t, ρ is the weight coefficient for dynamically adjusting the attraction of the grasping point, and β is the weight coefficient for dynamically adjusting the obstacle avoidance strength;

[0105] Configuration parameters q of the optimal grasping path of the robot arm* (t), generate the optimal grasping path of the robot arm;

[0106] It should be noted that, through the gradient optimization method, according to the high-precision digital twin model of the target object and the optimal grasping point P g , comprehensively considering path smoothness, grasping accuracy and obstacle avoidance, and identifying the optimal grasping path of the robot arm. The path planning formula dynamically balances the grasping point attraction, obstacle avoidance and path smoothness, ensuring that the robot arm can efficiently and safely complete the grasping action in complex dynamic scenes. This method not only improves the accuracy and execution efficiency of path planning, reduces the energy consumption and time cost of the robot arm movement, but also significantly reduces the risk of collision with obstacles during the grasping process, ensuring the safety and success rate of grasping operations in complex scenes.

[0107] S4.2: Calculate the optimal gripping force of the robot gripper at the gripping point based on the friction coefficient, contact area, elastic modulus, and roughness uniformity of the gripping point. The expression is:

[0108]

[0109] Among them, F c is the optimal gripping force of the robot arm at the grasping point, m is the mass of the target object, g is the gravitational acceleration, σ max Indicates the maximum allowable value of roughness, k e is the influence coefficient of elastic modulus;

[0110] It should be noted that the optimal gripping force F of the robot gripper is calculated based on the friction coefficient, contact area, elastic modulus and roughness uniformity of the gripping point. c , ensuring that the gripper can provide sufficient friction to stabilize the target object during the grasping process, while avoiding damage or deformation of the target object due to excessive force. This formula takes into account the physical properties of the target object (such as friction coefficient, elastic modulus and surface roughness) and quality requirements, and dynamically adjusts the clamping force. By reasonably distributing the clamping force, the stability and safety of the grasping action are improved, the risk of grasping failure is reduced, and the integrity of the target object is effectively protected. It is particularly suitable for grasping objects of different materials and shapes.

[0111] S4.3: Based on the optimal grasping path of the robot and the optimal gripping force F of the robot at the grasping point c , generate a crawling strategy;

[0112] Furthermore, based on the optimal grasping path, the motion trajectory and posture change of the robot arm are determined, and the optimal gripping force F is combined c , adjust the mechanical output of the robot arm gripper to generate a complete grasping strategy. For example, when grasping a glass, the robot arm adjusts the mechanical output of the robot arm gripper according to q* (t) bypass the obstacle to reach the specified grasping angle and pass F c Accurately apply appropriate clamping force to ensure gripping stability and avoid squeezing and damage to the glass.

[0113] S4.4: Execute the grasping strategy in the high-precision digital twin model, simulate the grasping process, and verify the optimal grasping path and the optimal gripping force F of the robot arm at the grasping point c feasibility;

[0114] Furthermore, after the grasping strategy is generated, it is mapped to a high-precision digital twin model for virtual simulation, by simulating the movement of the robotic arm along the optimal grasping path and applying the optimal clamping force F at the grasping point. c , observe whether the target object can be grasped successfully. For example, in the simulation, verify whether the path is smooth and obstacle-avoiding, whether the gripping force is sufficient to grasp a ceramic vase with an unstable center of gravity, and analyze whether the simulation fails due to incorrect strategies.

[0115] S4.5: During the simulated grasping process, if the target object slips, deforms, or the robot arm collides with an obstacle, the current grasping strategy is considered infeasible;

[0116] Furthermore, during the simulation, if the target object slips due to insufficient clamping force, or deforms due to excessive clamping force, or the optimal grasping path fails to effectively avoid obstacles and collides with obstacles in the scene, the current grasping strategy is judged to be infeasible. For example, when the robot arm tries to grasp a rubber ball, if the clamping force is too large in the simulation and the ball is flattened, or the path planning fails to avoid obstacles and causes the robot arm to contact obstacles, the grasping strategy needs to be re-optimized.

[0117] S4.6: During the simulated grasping process, if the target object does not slip or deform, and the robot arm does not collide with obstacles, the current grasping strategy is considered feasible.

[0118] Furthermore, if the grasping process is simulated in the digital twin model, the target object can be grasped smoothly without slipping or deformation, and the robot arm can successfully avoid obstacles and complete the grasping along the optimal grasping path, then the grasping strategy is considered feasible. c If a porcelain plate is grasped and placed at the target location along the planned path and around obstacles, the strategy is considered validated for practical execution.

[0119] S5: Dynamically execute intelligent crawling strategies based on the verification results.

[0120] S5.1: When the current grabbing strategy is feasible, execute the current grabbing strategy and record the feedback data during the actual execution process;

[0121] It should be noted that recording feedback data can be used to analyze deviations during actual execution, optimize grasping paths and clamping force parameters, and continuously iterate and improve grasping strategies, thereby improving the grasping accuracy, stability, and adaptability of the robot arm in complex scenarios.

[0122] S5.2: When the current grasping strategy is not feasible, the current grasping strategy is not executed, the feedback data during the simulated grasping process is recorded, and based on the feedback data during the simulated grasping process, the current grasping strategy is optimized and re-verified until the verification result of the current grasping strategy is feasible.

[0123] Furthermore, when the current grasping strategy is not feasible, the grasping strategy can be optimized by recording the feedback data during the simulated grasping process (such as path deviation, insufficient clamping force causing slippage, excessive clamping force causing deformation, or obstacle avoidance failure causing collision, etc.), and the grasping strategy can be optimized in a targeted manner. For example, if the simulation finds that the optimal path of the robot arm fails to completely avoid obstacles, the obstacle avoidance weight coefficient β in the path planning can be adjusted; if the target object slips, the optimal clamping force F can be re-evaluated. c The weight of the friction coefficient is increased. The optimized grasping strategy is re-verified in the digital twin model until the verification result is feasible, thus ensuring the safety and success rate of the actual grasping.

[0124] This embodiment also provides a robotic arm grasping control system based on digital twins, including: a twin model construction module, a twin model correction module, a grasping point recognition module, a strategy generation module and a strategy execution module; the twin model construction module is used to collect target object data and grasping scene data, and construct a digital twin model of the target object and the grasping scene; the twin model correction module is used to monitor the dynamic changes of the target object and the grasping scene in real time, and dynamically correct the digital twin model to generate a high-precision digital twin model; the grasping point recognition module is used to extract the characteristics of the target object based on the high-precision digital twin model and identify the optimal grasping point; the strategy generation module is used to generate a grasping strategy based on the optimal grasping point and the high-precision digital twin model, and verify the feasibility of the grasping strategy; the strategy execution module is used to dynamically execute the intelligent grasping strategy according to the verification results.

[0125] This embodiment also provides a computer device, which is suitable for the case of a robotic arm grasping control method based on digital twins, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the robotic arm grasping control method based on digital twins proposed in the above embodiment.

[0126] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0127] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by the processor, the robot arm grasping control method based on digital twins as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, disk or optical disk.

[0128] In summary, the present invention improves the accuracy and stability of the robot arm's grasping by: real-time correction of the digital twin model and identification of the best grasping point. The real-time correction of the twin model dynamically adapts to scene changes and enhances flexibility and reliability; the best grasping point identification comprehensively evaluates the grasping stability and optimizes the grasping safety and success rate. Combined with the generation and verification of the grasping strategy, the whole process is optimized, solving the problems of insufficient model accuracy and low grasping reliability, and is suitable for complex dynamic scenes.

[0129] Example 2, referring to Table 1, is the second example of the present invention. In order to further verify the technical solution of the present invention, experimental simulation data of a robotic arm grasping control method based on digital twins is provided.

[0130] In order to verify the grasping performance advantage of the digital twin-based robotic arm grasping control method of the present invention over the prior art in dynamic scenes, the following experiments were designed:

[0131] The performance of the present invention is compared with that of the existing static three-dimensional point cloud-based grasping technology in dynamic grasping scenarios, and the differences between the two technologies in grasping success rate, path planning time and grasping stability are evaluated.

[0132] The existing technology uses a static 3D point cloud modeling method to obtain the 3D point cloud data of the target object through a depth camera to generate a geometric model and a rough physical property model (such as friction coefficient, elastic modulus) of the target object. The grasping strategy is based on the static point cloud to calculate the best grasping point, and the path planning uses the traditional RRT (rapid random tree search) algorithm to generate the robot arm motion path. Due to the lack of real-time dynamic monitoring and model update functions, the performance of this technology is limited when the target object or obstacle changes dynamically. The planned grasping path cannot adapt to scene changes, which easily leads to grasping failure.

[0133] First, the experiment aims to verify the performance difference between the robot arm grasping control method based on digital twins and the existing static point cloud grasping technology in dynamic grasping scenarios. A ceramic cylinder with a rough surface is selected as the experimental object, and a complex experimental scene containing dynamic moving obstacles and fixed obstacles is set up. In the experiment, the method of the present invention collects target object and grasping scene data through a depth camera, RGB camera and lidar, constructs and corrects a high-precision digital twin model in real time, and completes real-time monitoring of the dynamic changes of the target object and the distribution status of obstacles.

[0134] Secondly, through the physical properties and geometric model data of the digital twin model, the method of the present invention extracts the surface characteristics of the target object, including roughness, normal distribution, friction coefficient, etc., uses the multi-parameter scoring formula to calculate the stability score of the grasping point, and selects the grasping point with the highest score as the optimal grasping point P. g At the same time, the robot grasping path q is generated based on the gradient optimization method * (t), and calculate the optimal clamping force F based on the physical parameters of the target object c , ensuring the accuracy and stability of mechanical distribution during the grasping process.

[0135] Next, by simulating the grasping strategy in the digital twin model and verifying the smoothness of the grasping path, obstacle avoidance ability, and reliability of the clamping force, the grasping operation was performed and data such as the grasping success rate, path planning time, and grasping stability score were recorded. At the same time, in order to compare the performance of existing technologies, the static point cloud grasping technology was used to perform the same experiment. In static point cloud grasping, the target object model and grasping path are generated through static three-dimensional point clouds, but the real-time adjustment capability of the dynamic changes of the target object and obstacles is lacking, resulting in a low grasping success rate and stability.

[0136] The details are shown in Table 1 below:

[0137] Table 1 Dynamic grabbing technology performance comparison experimental data table

[0138]

[0139] Through experimental data analysis, it is found that the grasping success rate of the method of the present invention reaches 96%, which is much higher than the 68% of the prior art; the path planning time is shortened to 2.3 seconds, which is significantly more efficient than the 4.8 seconds of the prior art; the grasping stability score is significantly improved to 92.7, while the prior art is only 65.4. The experimental results show that the adaptability, grasping accuracy and efficiency of the method of the present invention in dynamic scenes are significantly better than the prior art, successfully overcoming the deficiency that the static point cloud grasping technology cannot be adjusted in real time in a dynamic environment, which fully reflects the innovation and practical value of the present invention.

[0140] The digital twin-based robotic arm grasping control method monitors the dynamic changes of the target object and the grasping scene in real time, dynamically corrects the digital twin model, and combines the multi-parameter scoring algorithm and the gradient optimization path planning method to significantly improve the grasping success rate, path planning efficiency and grasping stability in dynamic grasping scenes. Experimental results show that the adaptability and grasping accuracy of the present invention in dynamic scenes are far superior to the existing grasping method based on static point clouds, successfully overcoming the deficiency of the existing technology that cannot adapt to dynamic changes in real time.

[0141] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A robot arm grasping control method based on digital twin, characterized in that: include, Collect target object data and grasping scene data, and build digital twin models of target objects and grasping scenes; Monitor the dynamic changes of target objects and grasping scenes in real time, and dynamically modify the digital twin model to generate a high-precision digital twin model; Based on the high-precision digital twin model, the characteristics of the target object are extracted and the optimal grasping point is identified; Generate a grasping strategy based on the optimal grasping point and high-precision digital twin model, and verify the feasibility of the grasping strategy; Based on the verification results, the intelligent crawling strategy is dynamically executed.

2. The digital twin-based robotic arm grasping control method according to claim 1, characterized in that: The target object data includes location information, three-dimensional point cloud data and material characteristic data of the target object; The grabbing scene data includes the spatial distribution and dynamic changes of obstacles.

3. The digital twin-based robotic arm grasping control method according to claim 2, characterized in that: The specific steps of constructing the digital twin model of the target object and the grasping scene are as follows: Based on the position information and 3D point cloud data of the target object, a 3D geometric model of the target object is constructed through a triangular mesh reconstruction algorithm; Extracting the friction coefficient from the material property data of the target object, and matching the elastic modulus from the material database according to the material type of the target object; Integrate the friction coefficient, elastic modulus and three-dimensional geometric model of the target object to generate a complete physical property model; Based on the spatial distribution of obstacles in the grasping scene, a two-dimensional environment map is constructed using occupancy grid mapping. Combined with the dynamic change information of obstacles, the two-dimensional environment map is merged with the geometric model of the target object in a unified coordinate system to generate a complete scene layout model. The three-dimensional geometric model, physical property model and scene layout model of the target object are integrated to generate a digital twin model of the target object and the grasping scene.

4. The robot arm grasping control method based on digital twin according to claim 3, characterized in that: The real-time monitoring of the dynamic changes of the target object and the grasping scene, and the dynamic correction of the digital twin model to generate a high-precision digital twin model, the specific steps are as follows: Through the depth camera and RGB camera, the position change, posture change and material property change of the target object are monitored in real time; The real-time position and dynamic change information of obstacles in the grabbing scene are collected through LiDAR; Based on the real-time position information and posture changes of the target object, the spatial position and orientation of the 3D geometric model in the digital twin model are updated; According to the changes in the material properties of the target object, the friction coefficient and elastic modulus of the target object are updated, and the physical property model is corrected; According to the real-time position and dynamic change information of obstacles in the grasping scene, the position and rotation state of obstacles in the scene layout model are corrected in real time, and the occupation probability and distribution state of obstacles are updated in the occupation grid map; A high-precision digital twin model is generated based on the corrected three-dimensional geometric model, physical property model and scene layout model.

5. The digital twin-based robotic arm grasping control method according to claim 4, characterized in that: Based on the high-precision digital twin model, the characteristics of the target object are extracted and the optimal grasping point is identified. The specific steps are as follows: Through the three-dimensional geometric model and material property data of the target object, the surface roughness, normal distribution, friction coefficient distribution and material elastic properties of the target object are extracted using the surface property analysis algorithm; Filter the smooth surface areas according to the surface roughness and friction coefficient distribution; Filter surface uneven areas based on normal distribution; Mark the graspable area by filtering the surface area of ​​the target object; Based on the edited graspable area, combined with the surface roughness, normal distribution, friction coefficient distribution and material elastic properties of the target object, the grasping stability score of each graspable area is calculated, and the expression is: Among them, μ f is the friction coefficient distribution value of the grasping point, α is the nonlinear weight coefficient of the friction coefficient, E is the elastic modulus of the target object, A c is the area of ​​contact between the robot and the target object, A t is the total surface area of ​​the graspable area of ​​the target object, Δφ is the normal deviation angle of the grasping point, κ is the nonlinear adjustment coefficient of the normal deviation angle, d is the distance from the grasping point to the center of mass of the target object, λ is the adjustment coefficient of the distance offset, and σ s is the standard deviation of surface roughness in the contact area, γ is the attenuation coefficient of roughness uniformity, η is the nonlinear adjustment coefficient of contact area standardization, and Q is the grasping stability score; Select the grasping point with the highest grasping stability score Q in the marked graspable area as the best grasping point P g .

6. The digital twin-based robotic arm grasping control method according to claim 5, characterized in that: The method generates a grasping strategy based on the optimal grasping point and the high-precision digital twin model, and verifies the feasibility of the grasping strategy. The specific steps are as follows: Based on the high-precision digital twin model of the target object and the optimal grasping point P g Based on the gradient optimization method, the path smoothness, grasping accuracy and obstacle avoidance ability are comprehensively considered to identify the configuration parameters of the optimal grasping path of the robot arm. The expression is: Among them, q * (t) is the configuration parameter of the optimal grasping path of the robot arm at time t, q0 is the initial joint angle configuration parameter of the robot arm, t0 is the starting time of path planning, t is the current time, p(t') is the spatial position of the end of the robot arm at time t', Indicates the rate of change of the robot arm joint configuration to the obstacle distance, represents the gradient of the path smoothing term, q(t') is the joint angle configuration parameter of the robot at time t', t' represents any time between the start time t0 of path planning and the current time t, ρ is the weight coefficient for dynamically adjusting the attraction of the grasping point, and β is the weight coefficient for dynamically adjusting the obstacle avoidance strength; Configuration parameters q of the optimal grasping path of the robot arm * (t), generate the optimal grasping path of the robot arm; According to the friction coefficient, contact area, elastic modulus and roughness uniformity of the grasping point, the optimal clamping force of the robot gripper at the grasping point is calculated, and the expression is: Among them, F c is the optimal gripping force of the robot arm at the grasping point, m is the mass of the target object, g is the gravitational acceleration, σ max Indicates the maximum allowable value of roughness, k e is the influence coefficient of elastic modulus; Based on the optimal grasping path of the robot arm and the optimal gripping force F of the robot arm at the grasping point c , generate a crawling strategy; Execute the grasping strategy in the high-precision digital twin model, simulate the grasping process, and verify the optimal grasping path and the optimal gripping force F of the robot arm at the grasping point c feasibility; In the simulated grasping process, if the target object slips, deforms, or the robot arm collides with an obstacle, the current grasping strategy is considered infeasible; In the simulated grasping process, if the target object does not slip or deform, and the robot arm does not collide with obstacles, the current grasping strategy is considered feasible.

7. The robot arm grasping control method based on digital twin according to claim 6, characterized in that: According to the verification results, the intelligent crawling strategy is dynamically executed. The specific steps are as follows: When the current crawling strategy is feasible, execute the current crawling strategy and record the feedback data during the actual execution process; When the current crawling strategy is not feasible, the current crawling strategy is not executed, the feedback data during the simulated crawling process is recorded, and based on the feedback data during the simulated crawling process, the current crawling strategy is optimized and re-verified until the verification result of the current crawling strategy is feasible.

8. A robot arm grasping control system based on digital twin, based on the robot arm grasping control method based on digital twin according to any one of claims 1 to 7, characterized in that: Including twin model construction module, twin model correction module, grasping point recognition module, strategy generation module and strategy execution module; Twin model building module, used to collect target object data and grasping scene data, and build digital twin models of target objects and grasping scenes; The twin model correction module is used to monitor the dynamic changes of the target object and the grasping scene in real time, and dynamically correct the digital twin model to generate a high-precision digital twin model; The grasping point recognition module is used to extract the characteristics of the target object and identify the optimal grasping point based on the high-precision digital twin model; Strategy generation module, used to generate grasping strategies based on the optimal grasping points and high-precision digital twin models, and verify the feasibility of the grasping strategies; The strategy execution module is used to dynamically execute the intelligent crawling strategy based on the verification results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the digital twin-based robotic arm grasping control method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the digital twin-based robotic arm grasping control method according to any one of claims 1 to 7 are implemented.

Images