Workpiece three-dimensional point cloud data set generation method based on joint simulation

Abstract

The invention relates to a workpiece three-dimensional point cloud data set generation method based on joint simulation. The method comprises the steps that workpiece three-dimensional models of different types and sizes are constructed in batches; a working system is built, the working system completes simulation configuration of the mechanical arm and the depth camera in the Gazebo platform, and motion control over the mechanical arm in the Gazebo platform is achieved through a MoveIt! Configuration file; by changing simulation environment files in a working system, batch loading of three-dimensional models of workpieces of different types and sizes is achieved, and environment parameters are adjusted; the mechanical arm is controlled to move to different poses through the MoveIt! Configuration file; acquiring and storing point cloud information of the workpiece through a depth camera under different poses; and repeating the above processes to finally obtain a three-dimensional point cloud data set of the workpiece. Compared with the prior art, automatic acquisition of point clouds of workpieces of different types and sizes under different poses can be realized, and the efficiency and quality of point cloud data set generation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud processing, and in particular to a method for generating a three-dimensional point cloud dataset of workpieces based on joint simulation. Background Art

[0002] As a common form of three-dimensional data, point clouds have relatively wide applications in fields such as industrial production, autonomous driving, and robotics due to their high-precision three-dimensional space expression ability. With the development of deep learning technology, using deep neural networks to perform processing methods such as classifying, segmenting, and three-dimensional reconstructing point clouds has become a current research hotspot.

[0003] The performance of deep learning models depends to a large extent on large-scale and high-quality datasets. However, compared with two-dimensional images, the acquisition of three-dimensional point cloud data is more complex and inefficient. Existing publicly available point cloud datasets are mostly in the fields of autonomous driving and medical images, and datasets in the industrial production field are scarce, which limits the further development and application of point cloud deep learning technology in industrial production.

[0004] In this regard, the invention with the publication number CN114581609A discloses a method for generating a three-dimensional point cloud dataset based on a physical operation engine. After loading the three-dimensional models of objects and scenes, randomly initialize the object poses and initialize the PhysX physical world and perform simulations; then set up a shooting system in OpenGL, input the object poses in the PhysX physical world and the material and surface texture parameters in the three-dimensional models into OpenGL, and draw in OpenGL to generate the initial point cloud of the scene; then obtain the semantic and instance label images of the scene through color mapping, and add labels to each point of the scene point cloud through color inverse mapping, and use them together with the pose label data as a three-dimensional point cloud dataset.

[0005] This solution only combines physical simulation and simulation rendering to realize the simulation acquisition of point cloud data, but its simulation process is a static process and cannot realize the automatic acquisition of point clouds of different types and sizes of workpieces in different poses in industrial production. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a method for generating a three-dimensional point cloud dataset of workpieces based on joint simulation, realizing the automatic acquisition of point clouds of different types and sizes of workpieces in different poses in industrial production, improving the efficiency and quality of generating point cloud datasets, and providing an effective way for obtaining high-quality datasets of workpiece point clouds in industrial production.

[0007] The purpose of the present invention can be achieved through the following technical solutions: A method for generating a three-dimensional point cloud dataset of workpieces based on joint simulation, comprising the following steps: S1: Batch construct 3D models of workpieces with different types and sizes. S2: Set up a working system, which includes a control system with a robotic arm as the core and a vision system with a depth camera as the core. The working system completes the simulation configuration of the robotic arm and the depth camera in the Gazebo platform, and realizes the motion control of the robotic arm in the Gazebo platform through the MoveIt! configuration file. S3: By changing the simulation environment file in the working system, batch load 3D models of workpieces with different types and sizes, and adjust the environmental parameters. S4: Control the robotic arm to move to different poses through the MoveIt! configuration file. S5: Obtain and store the point cloud information of the workpiece through the depth camera at different poses. S6: Repeat steps S3 - S5 to perform the processes of loading 3D models of workpieces, adjusting environmental parameters, controlling the robotic arm, and obtaining and storing the point cloud information of the workpiece through the depth camera until the preset stop condition is reached, and finally obtain the 3D point cloud dataset of the workpiece.

[0008] Further, the process of setting up the working system in step S2 is specifically as follows: S201: Complete the simulation configuration of the robotic arm and the depth camera in the Gazebo platform through the Setup Assistant module. During the configuration process of the Setup Assistant module, load the robotic arm model with the depth camera configured through the xacro file, and generate a collision exemption matrix through collision detection. Create a virtual joint, associate the robotic arm model with the simulation environment file of the Gazebo platform, and realize the unification of coordinate systems. Add a motion planning group, and select a kinematic solver and a motion planning algorithm. Configure the ROS Control module, and select a controller and related types. Finally, generate a simulation environment file and related configuration files. S202: Adjust the MoveIt! configuration file to realize the motion control of the robotic arm in the Gazebo platform.

[0009] Further, the adjustment of the MoveIt! configuration file to realize the motion control of the robotic arm in the Gazebo platform is specifically as follows: Adjust the effort parameter of the joint component in the MoveIt! configuration file to ensure the effective control of each joint of the robotic arm; adjust the plugin parameter in the MoveIt! configuration file to ensure the normal use of the controller.

[0010] Further, step S1 is specifically as follows: in the Rhino platform, call the rhinoscriptsyntax library, and batch build 3D models of workpieces of different types and sizes through Python.

[0011] Further, step S3 is specifically as follows: by adjusting the model module in the simulation environment file, load 3D models of workpieces of different types and sizes; adjust the environmental parameters in the simulation environment file according to requirements, and the environmental parameters include light intensity, light angle, and gravitational acceleration.

[0012] Further, in step S4, set the required pose by assigning values to all joint components in the MoveIt! configuration file.

[0013] Further, the depth camera is connected to the end of the robotic arm, and a coordinate system transformation matrix of the depth camera relative to the world coordinate system of the working system is pre-acquired, which is used to convert the point cloud information obtained by the depth camera into the world coordinate system.

[0014] Further, in step S5, the process of obtaining and storing the point cloud information of the workpiece by the depth camera is specifically as follows: Obtain the point cloud information of the workpiece in the depth camera coordinate system through the depth camera, publish it in the form of a topic, set a subscription object to subscribe to the point cloud topic, and convert the point cloud information from the depth camera coordinate system to the world coordinate system through the coordinate system transformation matrix and store it.

[0015] Further, in step S6, write a total control program through Python to integrate steps S3 - S5, repeatedly execute the process of loading the 3D model of the workpiece, adjusting the environmental parameters, controlling the robotic arm, and obtaining and storing the point cloud information of the workpiece by the depth camera, set a specified number of loops, and realize the batch automatic acquisition of point cloud data, and finally obtain a 3D point cloud dataset of the workpiece.

[0016] Further, the processing process of the total control program includes the following steps: S61: Create combinations of all possible models, sizes, and postures, specifically including the following sub-steps: S611: Initialize the combination list; S612: For each model path in the preset model path list, execute step S613 respectively; S613: For each size within the preset size range, execute step S614 respectively; S614: For each posture in the preset posture list, execute step S615 respectively; S615: Create a parameter group including the model path, size, and posture, and add the parameter group to the combination list; S62: For each set of parameters in the combined list, step S621 is executed separately. If an exception occurs, the error information is recorded and the current parameter set is skipped, and the next loop is continued; S621: Replace the model in the simulation environment file of the Gazebo platform, and set the parameters in the combined list as the new model path; S622: Start the simulation in the Gazebo platform; S623: Control the manipulator in the Gazebo platform to move to the specified pose through the MoveIt! configuration file; S624: Obtain and store the point cloud data at each pose; S63: Shut down the simulation of the Gazebo platform and end the overall control program.

[0017] Compared with the prior art, the present invention has the following advantages: (1) Flexibility and scalability: Through the joint simulation of Gazebo and MoveIt!, the present invention configures the simulation of the manipulator and the depth camera through the Gazebo platform. By modifying the simulation environment file, batch loading of 3D models of different types and sizes of workpieces can be achieved, as well as adjustment of environmental parameters; the movement control of the manipulator in the Gazebo platform is realized through the MoveIt! configuration file, and movement to different poses can be achieved. Overall, a flexible way of processing 3D models of workpieces is provided. Users can easily perform batch modeling through the Rhino platform, and load different workpiece models and simulate different environmental conditions by modifying the simulation environment file.

[0018] (2) High efficiency: Through the automated batch loading and simulation process, the present invention significantly improves the efficiency of generating 3D point cloud datasets. Traditional methods for generating datasets often require manual data collection and processing, which is not only time-consuming but also inefficient. The present invention provides an overall control program that executes in a loop through an automated process, and can generate a large amount of high-quality point cloud data in a short time.

[0019] (3) Generation of high-quality datasets: By simulating different lighting conditions and other environmental information, and precisely controlling the manipulator to move to different poses, and cooperating with the depth camera to obtain high-precision point cloud information, the present invention can generate high-quality 3D point cloud datasets that reflect real-world conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic flowchart of a method for generating a 3D point cloud dataset of workpieces based on joint simulation provided in an embodiment of the present invention; Figure 2It is a visual process schematic diagram of a method for generating a three-dimensional point cloud dataset of workpieces based on co-simulation provided in an embodiment of the present invention; Figure 3 It is three-dimensional models of workpieces of different types and sizes and the acquired point cloud maps provided in an embodiment of the present invention. Detailed implementation manners

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0023] It should be noted that: similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0024] Embodiment 1 In view of the problems existing in the foregoing prior art, this embodiment provides a method for generating a three-dimensional point cloud dataset of workpieces based on co-simulation. Refer to Figure 1 and Figure 2 , and a specific example is used to illustrate the usage steps of the method: S1: Batch construct three-dimensional models of workpieces of different types and sizes; In this embodiment, three-dimensional models of workpieces are batch-built through the Rhino platform (3D modeling software). Specifically, the rhinoscriptsyntax library (a lightweight python script library for simplifying the programmatic control of the Rhino platform) is called, and three-dimensional models of workpieces of different types and sizes are batch-built through python and stored in the stl file format (a standardized file format specifically designed for 3D printing). Taking the generation of the three-dimensional model of the corner joint as an example, the pseudo code is as follows: "Generation of the three-dimensional model of the corner joint Input: - [length1_min, length1_max]: Value range of the length of the first joint - [width1_min, width1_max]: Range of values for the width of the first joint - [height1_min, height1_max]: Range of values for the height of the first joint - [length2_min, length2_max]: Range of values for the length of the second joint - [width2_min, width2_max]: Range of values for the width of the second joint - [height2_min, height2_max]: Range of values for the height of the second joint - [distance_var_min, distance_var_max]: Range of values for the distance between joints - [step_length1, step_width1, step_height1, step_length2, step_width2, step_height2, step_distance_var]: Parameter step size Output: - STL file - Weld seam information CSV file 1. Initialize the parameter range and step size - Define length1 from length1_min to length1_max, with step size step_length1 - Define width1 from width1_min to width1_max, with step size step_width1 - Define height1 from height1_min to height1_max, with step size step_height1 - Define length2 from length2_min to length2_max, with step size step_length2 - Define width2 from width2_min to width2_max, with step size step_width2 - Define height2 from height2_min to height2_max, with step size step_height2 - Define distance_var from distance_var_min to distance_var_max with a step of step_distance_var 2. Generate parameter combinations: - For each length1 within the length1 range: - For each width1 within the width1 range: - For each height1 within the height1 range: - For each distance_var within the distance_var range: - Calculate length2 = length1 - Calculate width2 = height1 - Calculate height2 = 0.6 * width1 - If 0.3 * width1 <= distance_var <= 0.7 * width1: - Create a parameter group {length1, width1, height1, length2, width2, height2, distance_var} - Add it to the parameter list 3. Batch generate models: - For each set of parameters in the parameter list: 3.1. Convert the unit from millimeters to meters 3.2. Create the first joint model 3.3. Create the second joint model 3.4. Merge the two joints 3.5. Calculate the weld position 3.6. Convert the unit back to millimeters 3.7. Export the STL file 3.8. Record the weld information into a CSV file 4. End batch generation 5. Return the output: - The generated STL file - The weld information CSV file See Figure 3 , the 3D models of workpieces of different types and sizes are shown in the figure. Here, V-shaped butt workpieces, I-shaped butt workpieces, lap workpieces, and fillet workpieces are specifically shown.

[0025] S2: Set up the working system, which includes a control system centered around a robotic arm and a vision system centered around a depth camera. The working system completes the simulation configuration of the robotic arm and the depth camera in the Gazebo platform (a high-fidelity physical simulation platform in the field of robot development), and realizes the motion control of the robotic arm in the Gazebo platform through the configuration file of MoveIt! (an open-source software framework for robot motion planning); The process of setting up the working system in step S2 is specifically as follows: S201: Complete the simulation configuration of the robotic arm and the depth camera in the Gazebo platform through the Setup Assistant module (the core configuration tool of the robot motion planning framework MoveIt!). During the configuration process of the Setup Assistant module, load and configure the robotic arm model with a depth camera through the xacro file (a high-level description language for defining robot models), and generate a collision exemption matrix through collision detection; in this embodiment, the robotic arm is selected as KUKA-KR16, and the depth camera is selected as RealSense D435; generate a collision exemption matrix according to 10,000 times of collision detection to reduce the processing time of motion planning; Create a virtual joint, associate the robotic arm model with the simulation environment file (world file) of the Gazebo platform to achieve the unification of coordinate systems, ensure the unification of the base coordinate system and the world coordinate system, and at the same time fix the robotic arm in the simulation environment; Add a motion planning group, select a kinematic solver and a motion planning algorithm; in this embodiment, select KDLKinematicsPlugin as the kinematic solver and select the RRT motion planning algorithm; Configure the ROS Control module and select the controller and related types; Finally, generate a simulation environment file and related configuration files to complete the simulation of the robotic arm and the depth camera in Gazebo; S202: Adjust the MoveIt! configuration file to achieve the motion control of the robotic arm in the Gazebo platform. Specifically, adjust the effort parameter of the joint component (the core component for describing the motion relationship between rigid body parts) in the MoveIt! configuration file to ensure the effective control of each joint of the robotic arm; adjust the plugin-related parameters in the MoveIt! configuration file to ensure the normal use of the controller.

[0026] The working system in the simulation world is as Figure 2 shown.

[0027] S3: Realize the batch loading of 3D models of workpieces of different types and sizes and adjust the environmental parameters by changing the simulation environment file in the working system. Specifically, by adjusting the model module (a systematic encapsulation module for the overall physical object and its attributes in robot development) in the simulation environment file (world file), the loading of 3D models of workpieces of different types and sizes is realized. The relevant pseudo-code is as follows; adjust the environmental parameters in the simulation environment file (world file) according to requirements. The environmental parameters include light intensity, light angle, gravitational acceleration, etc.

[0028] "Pseudo-code for the model adjustment algorithm in the simulation environment file Input: - models_directory: Directory path for storing model files - world_file_path: Path to the Gazebo world file (world file) - old_model_name: Model name to be replaced - new_model_path: Path to the new model file Output: - Updated Gazebo simulation environment file Define the function replace_model_in_gazebo_world(models_directory, world_file_path, old_model_name, new_model_path): 1. Load the Gazebo world file: Open the world_file_path file Read the content of the world_file into the variable world_content 2. Traverse all models in the world file: For each model in world_content: 2.1. Check if the model name matches old_model_name: If model.name is equal to old_model_name: 2.1.1. Find and update the URI path of the model to new_model_path: Update model.uri to new_model_path 2.1.2. Save changes to the world file: Write the modified world_content back to the world_file_path file 2.1.3. Output Return 2.2. If no matching model name is found: Output "Replacement failed: Model with model name old_model_name not found" Return " S4: Control the robotic arm to move to different poses through the MoveIt! configuration file; Specifically, by assigning values to all joint components in the MoveIt! configuration file, the required posture is set, and the robotic arm is controlled by MoveIt! to move to different poses.

[0029] In this embodiment, 3 postures are set for each type of workpiece. Taking the corner joint as an example, see Table 1.

[0030] Table 1 Robotic arm poses corresponding to the corner joint S5: Obtain and store the point cloud information of the workpiece through the depth camera at different poses; Specifically, the depth camera is connected to the end of the robotic arm, and the coordinate transformation matrix of the depth camera relative to the world coordinate system of the working system is pre-acquired, which is used to transform the point cloud information obtained by the depth camera into the world coordinate system.

[0031] Obtain the point cloud information of the workpiece in the depth camera coordinate system through the depth camera, publish it in the form of a topic, set a subscription object to subscribe to the point cloud topic, and transform the point cloud information from the depth camera coordinate system to the world coordinate system through the coordinate transformation matrix and store it.

[0032] In this embodiment, the depth camera is a RealSense D435, which is connected to the end of the robotic arm in the "Eye-in-Hand" manner. The transformation matrix of the depth camera relative to the base coordinate system (world coordinate system) can be easily obtained in the simulation environment.

[0033] The workpiece point cloud information obtained by the depth camera is published in the form of a topic. " / d435 / depth / color / points" is the topic for publishing the point cloud information in this example. Set the subscription object to subscribe to the point cloud topic, and convert the point cloud information from the depth camera coordinate system to the world coordinate system through the coordinate transformation matrix to obtain the real point cloud information of the workpiece in the world coordinate system. Store the obtained point cloud information in the pcd format.

[0034] S6: Repeat steps S3 - S5 to perform the processes of loading the 3D model of the workpiece, adjusting the environmental parameters, controlling the robotic arm, and obtaining and storing the point cloud information by the depth camera until the preset stop condition is reached, and finally obtain the 3D point cloud dataset of the workpiece.

[0035] Specifically, write a master control program in Python to integrate steps S3 - S5, repeat the processes of loading the 3D model of the workpiece, adjusting the environmental parameters, controlling the robotic arm, and obtaining and storing the point cloud information, set a specified number of loops to achieve batch automatic acquisition of point cloud data, and finally obtain the 3D point cloud dataset of the workpiece.

[0036] The general processing process of the master control program includes: S61: Create combinations of all possible models, sizes, and postures, which specifically include the following sub - steps: S611: Initialize the combination list; S612: For each model path in the preset model path list, execute step S613 respectively; S613: For each size within the preset size range, execute step S614 respectively; S614: For each posture within the preset posture list, execute step S615 respectively; S615: Create a parameter group including the model path, size, and posture, and add the parameter group to the combination list; S62: For each set of parameters in the combination list, execute step S621 respectively. If an exception occurs, record the error information and skip the current parameter group, and continue the next loop; S621: Replace the model in the simulation environment file of the Gazebo platform, and set the parameters in the combination list as the new model path; S622: Start the simulation in the Gazebo platform; S623: Control the robotic arm in the Gazebo platform to move to the specified posture through the MoveIt! configuration file; S624: Obtain and store the point cloud data at each posture; S63: Close the simulation of the Gazebo platform and end the master control program.

[0037] The corresponding main control program pseudocode is as follows: "Main control program algorithm pseudocode Input: - models_directory: The directory path storing model files - world_file_path: The path of the Gazebo world file (world file) - base_path: The basic path for storing point cloud files Output: - Automatically replace the model in the Gazebo simulation environment, control the robotic arm to obtain point cloud data, and save the data to the specified path 1. Initialize the required parameters: 1.1. Set the model folder path to models_directory 1.2. Set the world file path to world_file_path 1.3. Obtain the list of all model paths from models_directory 1.4. Set the basic path for storing point cloud files to base_path 1.5. Set other relevant parameters (such as robotic arm control parameters, attitude list, etc.) 2. Create combinations of all possible models, sizes, and attitudes: 2.1. Initialize the combination list as [] 2.2. For each model path in the model path list: 2.2.1. For each size in the size range: 2.2.1.1. For each attitude in the attitude list: 2.2.1.1.1. Create a parameter group {model path, size, attitude} 2.2.1.1.2. Add the parameter group to the combination list 3. In the main program: 3.1. For each set of parameters in the combination list: 3.1.1. Replace the model in the Gazebo world file: 3.1.1.1. Call the replace_model_in_gazebo_world function, passing in world_file_path, old_model_name, and the new model path 3.1.2. Try to execute the following steps: 3.1.2.1. Start the Gazebo simulation 3.1.2.2. Control the robotic arm to move to the specified pose 3.1.2.3. Obtain the point cloud data at each pose and save it to the corresponding directory under base_path 3.1.3. If an exception occurs, perform exception handling: 3.1.3.1. Record the error information into the log file 3.1.3.2. Skip the current parameter group and continue with the next loop 3.2. Close the Gazebo simulation 4. End the main program The three-dimensional point clouds of workpieces of different types and sizes are shown in Figure 3 , which specifically shows the V-shaped butt joint workpiece, I-shaped butt joint workpiece, lap joint workpiece, and corner joint workpiece

[0038] In summary, the method for generating a three-dimensional point cloud dataset of workpieces based on the combined simulation of Gazebo and MoveIt! provided by the present invention provides an innovative solution for the efficient acquisition and processing of three-dimensional point cloud data in industrial production. This method realizes the automatic and high-quality acquisition of point cloud data for various workpieces in industrial production through Rhino's efficient batch modeling, combined with robotic arm control and point cloud data acquisition of depth cameras under the combined simulation of Gazebo and MoveIt!. The present invention significantly improves the quality and efficiency of point cloud dataset generation and provides important support for the development of point cloud deep learning technology in the field of industrial production

[0039] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field of the present invention through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art shall fall within the protection scope determined by the claims

Claims

1. A method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation, characterized in that, It includes the following steps: S1: Batch build 3D models of workpieces with different types and sizes; S2: Set up a working system, which includes a control system with a robotic arm as the core and a vision system with a depth camera as the core. The working system completes the simulation configuration of the robotic arm and the depth camera in the Gazebo platform, and realizes the motion control of the robotic arm in the Gazebo platform through the MoveIt! configuration file; S3: By changing the simulation environment file in the working system, batch load 3D models of workpieces with different types and sizes, and adjust the environmental parameters; S4: Control the robotic arm to move to different poses through the MoveIt! configuration file; S5: Obtain and store the point cloud information of the workpiece through the depth camera at different poses; S6: Repeat steps S3 - S5 to perform the processes of loading 3D models of workpieces, adjusting environmental parameters, controlling the robotic arm, and obtaining and storing the point cloud information of the workpiece through the depth camera until the preset stop condition is reached, and finally obtain a 3D point cloud dataset of the workpiece.

2. The method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation according to claim 1, wherein The process of setting up the working system in step S2 is specifically as follows: S201: Complete the simulation configuration of the robotic arm and the depth camera in the Gazebo platform through the Setup Assistant module. During the configuration process of the Setup Assistant module, load the robotic arm model with the depth camera configured through the xacro file, and generate a collision exemption matrix through collision detection; Create a virtual joint, associate the robotic arm model with the simulation environment file of the Gazebo platform, and realize the unification of coordinate systems; Add a motion planning group, and select a kinematic solver and a motion planning algorithm; Configure the ROS Control module and select a controller and related types; Finally, generate a simulation environment file and related configuration files; S202: Adjust the MoveIt! configuration file to realize the motion control of the robotic arm in the Gazebo platform.

3. A method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation according to claim 2, characterized in that The adjustment of the MoveIt! configuration file to realize the motion control of the robotic arm in the Gazebo platform is specifically as follows: Adjust the effort parameter of the joint component in the MoveIt! configuration file to ensure the effective control of each joint of the robotic arm; adjust the plugin parameter in the MoveIt! configuration file to ensure the normal use of the controller.

4. A method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation according to claim 1, characterized in that, Step S1 is specifically to call the rhinoscriptsyntax library in the Rhino platform and batch build 3D models of workpieces with different types and sizes through python.

5. A method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation according to claim 1, characterized in that Step S3 is specifically to load 3D models of workpieces with different types and sizes by adjusting the model module in the simulation environment file; adjust the environmental parameters in the simulation environment file according to requirements, and the environmental parameters include light intensity, light angle, and gravitational acceleration.

6. A method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation according to claim 3, wherein, In step S4, set the required pose by assigning values to all joint components in the MoveIt! configuration file.

7. A method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation according to claim 1, characterized in that The depth camera is connected to the end of the robotic arm, and a coordinate system transformation matrix of the depth camera relative to the world coordinate system of the working system is pre-acquired, which is used to transform the point cloud information acquired by the depth camera into the world coordinate system.

8. A method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation according to claim 7, characterized in that, In step S5, the process of acquiring and storing the point cloud information of the workpiece by the depth camera is specifically as follows: Acquire the point cloud information of the workpiece in the depth camera coordinate system through the depth camera, publish it in the form of a topic, set a subscription object to subscribe to the point cloud topic, and transform the point cloud information from the depth camera coordinate system to the world coordinate system through the coordinate system transformation matrix and store it.

9. A method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation according to claim 1, wherein, In step S6, write a master control program in Python to integrate steps S3 - S5, repeatedly execute the processes of loading the 3D model of the workpiece, adjusting the environmental parameters, controlling the robotic arm, and acquiring and storing the point cloud information of the depth camera, set a specified number of loops to achieve batch automatic acquisition of point cloud data, and finally obtain a 3D point cloud dataset of the workpiece.

10. A method for generating a three-dimensional point cloud data set of a workpiece based on co-simulation according to claim 9, characterized in that The processing process of the master control program includes the following steps: S61: Create combinations of all possible models, sizes, and postures, specifically including the following sub-steps: S611: Initialize the combination list; S612: For each model path in the preset model path list, execute step S613 respectively; S613: For each size within the preset size range, execute step S614 respectively; S614: For each posture within the preset posture list, execute step S615 respectively; S615: Create a parameter group including the model path, size, and posture, and add the parameter group to the combination list; S62: For each group of parameters in the combination list, execute step S621 respectively. If an exception occurs, record the error information and skip the current parameter group, and continue the next loop; S621: Replace the model in the simulation environment file of the Gazebo platform, and set the parameters in the combination list as the new model path; S622: Start the simulation in the Gazebo platform; S623: Control the robotic arm in the Gazebo platform to move to the specified posture through the MoveIt! configuration file; S624: Acquire and store the point cloud data at each posture; S63: Close the simulation of the Gazebo platform and end the master control program.

Images