Development method of robot guiding platform software

Abstract

The invention provides a method for developing robot guide platform software, which comprises the following steps of: designing a scene simulation rendering module which is used for storing working condition information in a robot application scene and simulating the working condition information; a 3D visual processing module is designed, the module is used for storing various different 3D visual processing tool codes, and recognition of a target object is achieved in the mode that 3D visual processing tools are used in a combined mode; designing a robot motion control module which is used for storing a plurality of different motion control tool codes and performing corresponding control on the robot based on corresponding motion control tools; designing an industrial communication module, wherein the module is used for storing a plurality of different communication tool codes; designing a user interface module which is used for storing different types of control codes and providing a corresponding user interface in response to a dragging instruction of a user; the design task module is used for storing operation information of robot guiding platform software; and designing a software operation control module. For the robot guiding platform software developed according to the method, an application adopts the robot guiding platform software to develop robot vision software, and the deployment time of robot vision application can be greatly shortened.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and more specifically, relates to a method for developing robot guidance platform software. Background Art

[0002] Machine vision technology uses cameras to replace the human eye in making decisions. Robotic vision, a key branch of machine vision, uses visual technology to guide robots to perform actions such as grasping workpieces. 3D vision is a key area of ​​vision technology. Guiding robots to complete tasks based on 3D vision requires 3D-based robot vision software, which can be implemented through graphical programming or traditional programming languages. With the growing application of robots, the requirements for the development cycle and functionality of robot vision software are also increasing. Therefore, using graphical programming to customize robot vision software is a future trend. Graphical programming is implemented on the basis of robot guidance platform software. Therefore, building 3D-based robot guidance platform software that can meet the specific needs of different users through graphical programming is a key issue that needs to be addressed. Summary of the Invention

[0003] In view of this, the present invention proposes a method for developing robot guidance platform software, the development method comprising the following steps:

[0004] Step S100: Designing a scene simulation rendering module, wherein the scene simulation rendering module is used to store working condition information in the robot application scene and simulate the working condition information;

[0005] Step S200: Design a 3D vision processing module, wherein the 3D vision processing module is used to store a plurality of different 3D vision processing tool codes and realize target object recognition by combining and using the 3D vision processing tools;

[0006] Step S300: Designing a robot motion control module, wherein the robot motion control module is used to store a plurality of different motion control tool codes and to control the robot accordingly based on the corresponding motion control tools;

[0007] Step S400: Design an industrial communication module, wherein the industrial communication module is used to store codes of multiple different communication tools, each corresponding to a different communication protocol interface;

[0008] Step S500: Designing a user interface module, wherein the user interface module is used to store different types of control codes and provide a corresponding user interface in response to a user's drag command;

[0009] Step S600: Design a task module, where the task module is used to store the operation information of the robot guidance platform software;

[0010] Step S700: Design a software operation control module.

[0011] Optionally, in step S100, the algorithm interface of the simulation rendering scene module adopts a plug-in interface form, including a path planning plug-in, a collision detection plug-in, a robot kinematics forward and inverse solution plug-in, and a simulation rendering plug-in, and each plug-in is designed with the same function interface for module call and parameter transmission;

[0012] The path planning plug-in is used to plan the path points according to the robot's movement starting point and movement ending point;

[0013] The collision detection plug-in is used to detect whether the robot in the scene will collide with obstacles in the scene based on path planning;

[0014] The robot kinematics plug-in is used to convert the robot's path point pose into the robot's joint angle according to the robot's DH parameter model;

[0015] The simulation rendering plug-in is used to perform human-computer interaction and display on the scene model.

[0016] Optionally, the path planning plug-in is configured to be able to add different path planning algorithms for selection and calling;

[0017] The robot kinematics forward and inverse solution plug-in imports the robot's DH parameter model based on the geometric analysis method interface and automatically generates the robot kinematics forward and inverse solution interface;

[0018] The simulation rendering plug-in uses Coin3D as the basic display and rendering framework.

[0019] Optionally, in step S100 , the scene models in the simulation rendering scene module are divided into six categories, namely, camera model, robot model, end effector model, tooling model, workpiece model and obstacle model.

[0020] Optionally, in step S200, the 3D vision processing tools included in the 3D vision processing module are divided into three categories, namely, point cloud processing tools, 2D image deep learning combined with point cloud data processing tools, and 2D image processing combined with point cloud data processing tools;

[0021] The point cloud processing tool includes a point cloud data preprocessing tool for identifying the workpiece posture based on point cloud matching;

[0022] The 2D image deep learning combined with point cloud data processing tool is used to perform instance segmentation or object recognition in the 2D image based on deep learning, and map the segmented positions to the 3D point cloud to obtain the final pose of the workpiece;

[0023] The 2D image processing combined with the point cloud data processing tool is used to map the 2D line into a 3D line according to the 3D plane where the 2D line is located, and to generate the final pose of the workpiece according to the 3D line, the normal vector of the 3D plane, and the intersection of the 3D line and the 3D plane.

[0024] Optionally, in step S300, the motion control tools stored in the robot motion control module include a robot data conversion tool, a robot absolute movement instruction tool, and a robot relative movement instruction tool;

[0025] The robot data conversion tool is used to call the interface provided by the simulation rendering plug-in to convert the object posture recognized by the 3D vision processing module into the robot posture and generate the optimal path point for obstacle avoidance;

[0026] The robot absolute movement instruction tool is used to send a series of absolute path postures to the robot so that the robot moves according to the specified postures;

[0027] The robot relative movement instruction tool is used to move a specified distance in a predetermined direction based on the current position to obtain a new absolute posture, and send the new absolute posture to the robot so that the robot moves according to the specified posture.

[0028] Optionally, in step S200, each 3D vision processing tool of the 3D vision processing module is bound to a unique identification ID, and the same 3D vision processing tool is bound to different IDs when it is assigned multiple times, and each 3D vision processing tool includes input and output parameter information and tool name information;

[0029] In step S300, each motion control tool of the robot motion control module is bound to a unique identification ID. The same motion control tool is bound to different IDs when allocated multiple times. Each motion control tool contains input and output parameter information and tool name information.

[0030] Optionally, in step S500, the user interface module includes code encapsulation of different types of controls, and the codes of all controls have the same basic function interface;

[0031] The user interface module is used to implement related operations on controls by calling basic function interfaces, and the related operations include adding and deleting controls, setting control properties, and refreshing data.

[0032] Optionally, in step S600 , the task is divided into a main task and a logic task;

[0033] The main task is a task used for running, and includes information about all tools that need to be run; the logic task is included in the main task; and the logic task is bound to the corresponding logic tool.

[0034] Optionally, the software operation control module provides an interface for the final real-time operation of the robot guidance platform software, including cyclic operation, single operation and stop operation.

[0035] The robot guidance platform software development method of the present invention designs the robot guidance platform software into a scene simulation rendering module, a 3D vision processing module, a robot motion control module, an industrial communication module, a user interface module, a task module, and a software operation control module. The robot guidance platform software obtained using the development method of the present invention has the following beneficial effects:

[0036] The simulation scene settings are flexible and can be built according to various robot application scenarios;

[0037] The tool is highly scalable. Universal interfaces are designed for 3D vision processing tools, motion control tools, and communication tools, which can be added as needed without affecting other software functions.

[0038] Functional tasks support complex logical operations and can easily meet different application requirements;

[0039] User customization of control display ensures the flexibility of human-machine interface display.

[0040] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention may be better understood by referring to the following description taken in conjunction with the accompanying drawings, in which the same or similar reference numerals are used throughout the drawings to designate the same or similar parts.

[0042] Figure 1 A schematic flow chart of a method for developing robot guidance platform software according to an embodiment of the present invention is shown;

[0043] Figure 2 A flow chart of a method for developing robot guidance platform software for implementing disordered grasping by a KUKA robot according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to more fully understand the technical solution of the present invention, the exemplary embodiments of the present invention will be described in more comprehensive and detailed in conjunction with the accompanying drawings below. Obviously, the one or more embodiments of the present invention described below are only one or more of the specific ways of implementing the technical solution of the present invention, and are not exhaustive. It should be understood that other ways belonging to a general inventive concept can be used to implement the technical solution of the present invention, and should not be limited by the exemplary embodiments described. Based on one or more embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.

[0045] Example: Figure 1 FIG2 shows a flow chart of a method for developing software for a robot guidance platform according to an embodiment of the present invention. Figure 1 The robot guidance platform software development method according to an embodiment of the present invention includes the following steps:

[0046] Step S100: Design a scene simulation rendering module, which is used to store working condition information in the robot application scene and simulate the working condition information;

[0047] Step S200: Design a 3D vision processing module, which is used to store a variety of different 3D vision processing tool codes and realize target object recognition by combining and using the 3D vision processing tools;

[0048] Step S300: Design a robot motion control module, which is used to store a variety of motion control tool codes and control the robot accordingly based on the corresponding motion control tools;

[0049] Step S400: Design an industrial communication module, which is used to store codes of multiple different communication tools, each corresponding to a different communication protocol interface;

[0050] Step S500: Designing a user interface module, where the user interface module is used to store different types of control codes and provide a corresponding user interface in response to a user's drag command;

[0051] Step S600: Design a task module, where the task module is used to store the operation information of the robot guidance platform software;

[0052] Step S700: Design a software operation control module.

[0053] Furthermore, in step S100 of the embodiment of the present invention, the algorithm interface of the simulation rendering scene module adopts the form of a plug-in interface, including a path planning plug-in, a collision detection plug-in, a robot kinematics forward and inverse solution plug-in, and a simulation rendering plug-in. Each plug-in is designed with the same function interface for module call and parameter transmission;

[0054] The path planning plug-in is used to plan the path points according to the robot's movement starting point and movement ending point;

[0055] The collision detection plug-in is used to detect whether the robot in the scene will collide with obstacles in the scene based on path planning;

[0056] The robot kinematics plug-in is used to convert the robot's path point pose into the robot's joint angles based on the robot's DH parameter model;

[0057] The simulation rendering plug-in is used to perform human-computer interaction and display of scene models.

[0058] Furthermore, in an embodiment of the present invention, the path planning plug-in is configured to be able to add different path planning algorithms for selection and calling, including the Astar algorithm and the BiRRT algorithm;

[0059] The robot kinematics forward and inverse solution plug-in imports the robot's DH parameter model based on the geometric analysis method interface and automatically generates the robot kinematics forward and inverse solution interface;

[0060] The simulation rendering plug-in uses Coin3D as the basic display and rendering framework, which can achieve rapid function development to display the model in the scene and set the parameters of the model.

[0061] Furthermore, in step S100 of the embodiment of the present invention, the scene models in the simulation rendering scene module are divided into six categories, and the interface for adding and setting is implemented in the simulation rendering plug-in. The six categories of scene models are camera model, robot model, end effector model, tooling model, workpiece model and obstacle model.

[0062] Specifically, in an embodiment of the present invention, the simulation rendering plug-in provides a setting interface for adding and removing camera models. The camera model needs to import a hand-eye calibration parameter file. The hand-eye calibration file is generated by another hand-eye calibration tool. This file contains the conversion matrix for converting camera data from the camera coordinate system to the robot coordinate system. The simulation rendering plug-in provides parameter interfaces such as the addition and removal settings of the robot model, the tool system settings, and the joint angle limit settings. The simulation rendering plug-in provides settings for adding and removing the end effector model and the gripping point settings of the end effector. The simulation rendering plug-in provides settings for adding, removing, and size of tooling models. The simulation rendering plug-in provides settings for adding and removing workpiece models and the gripping point settings of workpieces. The simulation rendering plug-in provides settings for adding and removing obstacle models and the size of obstacles. The simulation rendering plug-in provides an interactive interface for moving the model in the scene with the mouse. The simulation rendering plug-in provides an API interface, through which other modules can modify the position of the tooling model and send robot-guided task instructions.

[0063] Furthermore, in step S200 of the embodiment of the present invention, the 3D vision processing tools included in the 3D vision processing module are divided into three categories, namely, point cloud processing tools, 2D image deep learning combined with point cloud data processing tools, and 2D image processing combined with point cloud data processing tools;

[0064] Point cloud processing tools include point cloud data preprocessing tools, which are used to identify workpiece poses based on point cloud matching; point cloud data preprocessing tools are used for preprocessing operations such as point cloud sampling and point cloud segmentation;

[0065] 2D image deep learning combined with point cloud data processing tools are used to perform instance segmentation or object recognition in 2D images based on deep learning, and map the segmented locations to 3D point clouds to obtain the final pose of the workpiece;

[0066] The processing tools for 2D image processing combined with point cloud data are used to map 2D lines into 3D lines based on the 3D plane where the 2D lines are located, and generate the final pose of the workpiece based on the normal vectors of the 3D lines and 3D planes and the intersection of the 3D lines and 3D planes; the processing tools for 2D image processing combined with point cloud data mainly include tools such as line and circle finding in 2D.

[0067] Specifically, in an embodiment of the present invention, the 3D visual processing module includes a series of visual processing tools, each of which has different processing functions, and the visual processing tools can be parameter-transferred from top to bottom and used in combination.

[0068] Furthermore, in step S300 of the embodiment of the present invention, the motion control tools stored in the robot motion control module include a robot data conversion tool, a robot absolute movement instruction tool, and a robot relative movement instruction tool;

[0069] The robot data conversion tool is used to call the interface provided by the simulation rendering plug-in to convert the object posture recognized by the 3D vision processing module into the robot posture and generate the optimal path point for obstacle avoidance;

[0070] The robot absolute movement command tool is used to send a series of absolute path poses to the robot so that the robot moves according to the specified poses;

[0071] The robot relative movement instruction tool is used to move a specified distance in a predetermined direction based on the current position to obtain a new absolute posture, and send the new absolute posture to the robot so that the robot moves according to the specified posture.

[0072] Furthermore, in step S200 of this embodiment of the present invention, each 3D vision processing tool in the 3D vision processing module is associated with a unique identifier ID. The same 3D vision processing tool may be associated with different IDs when assigned multiple times. Each 3D vision processing tool includes input and output parameter information and a tool name. This configuration allows for combining multiple 3D vision processing tools and transferring parameters, thereby making the recognition function of the 3D vision processing module more flexible and powerful.

[0073] Furthermore, in step S300 of this embodiment of the present invention, each motion control tool in the robot motion control module is associated with a unique identifier ID. Multiple assignments of the same motion control tool may be associated with different IDs. Each motion control tool includes input and output parameter information and a tool name. This configuration allows for combining multiple motion control tools and transferring parameters, enabling complex path motion control and more flexible robot control.

[0074] Specifically, in step S400 of the embodiment of the present invention, each communication tool in the industrial communication module implements each commonly used communication protocol for communicating with a robot or PLC device.

[0075] Furthermore, in step S500 of the embodiment of the present invention, the user interface module includes code encapsulation of different types of controls, and the codes of all controls have the same basic function interface;

[0076] The user interface module is used to implement related operations on controls by calling basic function interfaces. Related operations include adding and deleting controls, setting control properties, and refreshing data.

[0077] Specifically, in step S500 of an embodiment of the present invention, the user interface module implements mouse event manipulation of controls, enabling the alignment of different controls. In addition to implementing conventional text display controls, button input controls, numeric input controls, and image and point cloud display controls, the user interface module also encapsulates the model rendering display function in step S100 as a separate control. The model rendering display control requires an input interface to receive information from the 3D vision processing module and the robot motion control module to update the position of the model in the scene in real time. The tool that receives information from the 3D vision processing module and the robot motion control module assigns two variables to the industrial communication module, representing the name of the display control and whether it is to be displayed. With this configuration, when implementing the 3D vision processing module and the robot motion control module, the refresh function to the control is implemented based on the two set variables. To achieve greater flexibility in the user interface module's display functionality, the user interface module specifically encapsulates the display tool. The tool's function can dynamically add multiple display information. The input of each display information is associated with information from the 3D vision processing module and the robot motion control module. The output of the display information is associated with the control name.

[0078] Furthermore, in step S600 of the embodiment of the present invention, the tasks are divided into main tasks and logic tasks;

[0079] The main task is the task used for running, which contains all the tool information that needs to be run; the logical task is included in the main task; the logical task is bound to the corresponding logical tool.

[0080] Specifically, in step S600 of this embodiment of the present invention, the task module must contain information about all tools required to run in the previous steps, including 3D vision processing tools, robotic motion control tools, industrial communication tools, and specialized display tools. The task module utilizes a nested design concept, meaning that multiple logical tasks can be nested within task information, and tasks can be nested within logical tasks. Tool information can be added to each layer of tasks.

[0081] Furthermore, in an embodiment of the present invention, the software operation control module provides an interface for the robot guidance platform software to finally run in real time, including cyclic operation, single operation and stop operation.

[0082] Specifically, in an embodiment of the present invention, the software operation control module designs the interface for the final real-time operation of the robot guidance platform software, including loop operation, single operation, and stop operation. Loop operation controls the task to run continuously multiple times, single operation controls the task to run only once, and the stop operation function can jump out of the operation during the task execution. When running, each main task is assigned a thread, and each thread runs a main task. All tools and logic tasks within each main task run in their respective threads. When stopping, each main task jumps out of its own thread at the location of the currently running tool and closes the thread.

[0083] Figure 2 The flowchart of the method for developing robot guidance platform software for realizing disordered grasping of a KUKA robot according to an embodiment of the present invention is shown. Figure 2 Assuming that the robot guidance platform software does not have the simulation function for the KUKA robot KR10_R1420 model and does not have the 3D point cloud matching function, this example illustrates how to add a code interface to the robot guidance platform software. Specifically, the following steps are included:

[0084] Step S1: Expand the simulation rendering scene module to adapt to the KR10_R1420 robot.

[0085] First download the urdf description file and stl model file of KR10_R1420.

[0086] In order to make this robot model compatible with the robot kinematics plug-in in the extended simulation rendering scene, the ikfast robot inverse solver is used to generate C++ kinematics parsing code, and the code is generated into a dll file with the same name as the robot model. In this way, the robot kinematics plug-in can directly call the corresponding dll code interface according to the model name to be added.

[0087] Similarly, in order to make the simulation of this robot model compatible with the simulation rendering plug-in in the simulation rendering scene, the robot urdf file is converted into a Collada file using the urdf converter. The simulation rendering plug-in uses the Coin3D open source library to parse the Collada file for display and simulation. Therefore, if a new robot model is added, no additional code is required, only the corresponding Collada file of the robot needs to be generated.

[0088] In robot-generated tooling scenarios, pallets and material frames are often used. Both types are initialized and described using a Collada file format. Basic information includes length, width, height, and position. The material frame also includes thickness information. After placing the Collada file in the corresponding location, code is added to the simulation rendering scene to identify the Collada file at the specified location and then display it in the tooling scene. Furthermore, different parameter dialog boxes are designed to set the tooling parameters based on the tooling type. The material frame parameters have an additional thickness parameter compared to the pallet parameters.

[0089] In addition, if the point cloud matching method is used for 3D workpiece identification, then the object model, namely the workpiece model, needs to be set. In the workpiece model section of the simulation rendering plug-in, an interface is added to import the STL file of the model. When the workpiece model is parsed and displayed in the simulation rendering plug-in, two small 3D coordinate systems are used to represent the position of the workpiece. One is the center position coordinate system displayed at the center position of the workpiece model according to the direction of the model's bounding box, and the other coordinate system is displayed at the point where the workpiece is actually to be grasped by the robot gripper. At the same time, adding the workpiece parameter setting code can set the grasping position of the workpiece. After adding the workpiece model module to the simulation rendering plug-in, if you add different 3D workpieces later, you do not need to add additional code. That is, you can import the stl file for parameter setting.

[0090] Step S2: Add a point cloud matching algorithm tool to the 3D vision processing module.

[0091] The algorithm tools in the 3D vision processing module all share the same functional interface, divided into the tool add interface, tool setup interface, tool run interface, tool load interface, tool delete interface, and tool display interface. The point cloud matching algorithm is encapsulated as a plug-in according to these six interfaces. The point cloud matching tool implements the point cloud matching tool add interface, which creates a new point cloud matching algorithm instance and binds it to a unique ID. The point cloud matching tool implements the point cloud matching tool setup interface, which creates a dialog box for setting the algorithm parameters. The run interface implements the point cloud matching algorithm using the configured parameters. The point cloud matching algorithm uses the PPF algorithm. The point cloud matching tool load interface uploads previously saved point cloud matching parameter information to the algorithm instance with the corresponding ID. This parameter information is saved by consolidating all tool information through a task. The point cloud matching tool delete interface removes and releases the algorithm instance created with the corresponding ID and releases it from runtime memory. The point cloud matching algorithm tool display interface uses the window number passed in to refresh the point cloud matching results to the corresponding window.

[0092] After implementing the above interface, the code of the point cloud matching algorithm can be stored in the machine, and then the tool can be called in the task module.

[0093] Step S3: Correspondingly, both the robot motion control tool and the industrial communication tool utilize the same interface implementation. The corresponding industrial communication tool is encapsulated based on the communication protocol and format used by the newly added KR10_R1420 robot. For example, if this robot's communication protocol uses TCP / IP, a universal encapsulation of the communication format allows for dynamic configuration of prefixes, suffixes, delimiters, and data. If a newly added robot uses a similar communication protocol, no new code is required.

[0094] Step S4: Create a task module. Calls to all tool interface functions (add, set, run, load, delete, and display) must be placed within the task module. When calling the task module, the corresponding ID is passed in, and the corresponding tool will operate on the algorithm instance with the corresponding ID. When the task module runs in a loop, it will sequentially call the execution interface and corresponding display interface of all tools within the task.

[0095] Step S5, add the required control code to the human-computer interaction interface. The controls involved in the display of point cloud matching information include the point cloud display control. The point cloud display control needs to support the display of point clouds and models as well as the display of coordinate systems after matching. The interface implementation of this control adopts the common control interface of the human-computer interaction interface. It includes the control property setting interface and the display data interface. The implementation of the point cloud window control property setting interface includes attribute parameters such as window title, color, and size. The implementation of the display data interface includes the addition and display of multiple point clouds and models, as well as the addition and display of coordinate systems. It is more convenient to implement this control using VTK. When the display interface of the point cloud matching tool is called in the task, the display interface inside the point cloud matching tool will call the unified display data interface of the control to refresh the point cloud and other data.

[0096] Although one or more embodiments of the present invention have been described above, it should be understood by those skilled in the art that the present invention can be implemented in any other form without departing from its spirit and scope. Therefore, the embodiments described above are illustrative and not restrictive, and many modifications and substitutions will be apparent to those skilled in the art without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for developing robot guidance platform software, characterized in that: include: Step S100: Designing a scene simulation rendering module, wherein the scene simulation rendering module is used to store working condition information in the robot application scene and simulate the working condition information; Step S200: Design a 3D vision processing module, wherein the 3D vision processing module is used to store a plurality of different 3D vision processing tool codes and realize target object recognition by combining and using the 3D vision processing tools; Step S300: Designing a robot motion control module, wherein the robot motion control module is used to store a plurality of different motion control tool codes and to control the robot accordingly based on the corresponding motion control tools; Step S400: Design an industrial communication module, wherein the industrial communication module is used to store codes of multiple different communication tools, each corresponding to a different communication protocol interface; Step S500: Designing a user interface module, wherein the user interface module is used to store different types of control codes and provide a corresponding user interface in response to a user's drag command; Step S600: Design a task module, where the task module is used to store the operation information of the robot guidance platform software; Step S700: Design a software operation control module.

2. The method for developing robot guidance platform software according to claim 1, characterized in that: In step S100, the algorithm interface of the simulation rendering scene module adopts the form of a plug-in interface, including a path planning plug-in, a collision detection plug-in, a robot kinematics forward and inverse solution plug-in and a simulation rendering plug-in. Each plug-in is designed with the same function interface for module call and parameter transmission; The path planning plug-in is used to plan the path points according to the robot's movement starting point and movement ending point; The collision detection plug-in is used to detect whether the robot in the scene will collide with obstacles in the scene based on path planning; The robot kinematics plug-in is used to convert the robot's path point pose into the robot's joint angle according to the robot's DH parameter model; The simulation rendering plug-in is used to perform human-computer interaction and display on the scene model.

3. The method for developing robot guidance platform software according to claim 2, characterized in that: The path planning plug-in is configured to be able to add different path planning algorithms for selection and calling; The robot kinematics forward and inverse solution plug-in imports the robot's DH parameter model based on the geometric analysis method interface and automatically generates the robot kinematics forward and inverse solution interface; The simulation rendering plug-in uses Coin3D as the basic display and rendering framework.

4. The method for developing robot guidance platform software according to claim 3, characterized in that: In step S100, the scene models in the simulation rendering scene module are divided into six categories, namely camera model, robot model, end effector model, tooling model, workpiece model and obstacle model.

5. The method for developing robot guidance platform software according to claim 4, characterized in that: In step S200, the 3D vision processing tools included in the 3D vision processing module are divided into three categories: point cloud processing tools, 2D image deep learning combined with point cloud data processing tools, and 2D image processing combined with point cloud data processing tools; The point cloud processing tool includes a point cloud data preprocessing tool for identifying the workpiece posture based on point cloud matching; The 2D image deep learning combined with point cloud data processing tool is used to perform instance segmentation or object recognition in the 2D image based on deep learning, and map the segmented positions to the 3D point cloud to obtain the final pose of the workpiece; The 2D image processing combined with the point cloud data processing tool is used to map the 2D line into a 3D line according to the 3D plane where the 2D line is located, and to generate the final pose of the workpiece according to the 3D line, the normal vector of the 3D plane, and the intersection of the 3D line and the 3D plane.

6. The method for developing robot guidance platform software according to claim 5, characterized in that: In the step S300, the motion control tools stored in the robot motion control module include a robot data conversion tool, a robot absolute movement instruction tool, and a robot relative movement instruction tool; The robot data conversion tool is used to call the interface provided by the simulation rendering plug-in to convert the object posture recognized by the 3D vision processing module into the robot posture and generate the optimal path point for obstacle avoidance; The robot absolute movement instruction tool is used to send a series of absolute path postures to the robot so that the robot moves according to the specified postures; The robot relative movement instruction tool is used to move a specified distance in a predetermined direction based on the current position to obtain a new absolute posture, and send the new absolute posture to the robot so that the robot moves according to the specified posture.

7. The method for developing robot guidance platform software according to claim 6, characterized in that: In step S200, each 3D vision processing tool of the 3D vision processing module is bound to a unique identification ID. The same 3D vision processing tool is bound to different IDs when it is assigned multiple times. Each 3D vision processing tool includes input and output parameter information and tool name information. In step S300, each motion control tool of the robot motion control module is bound to a unique identification ID. The same motion control tool is bound to different IDs when allocated multiple times. Each motion control tool contains input and output parameter information and tool name information.

8. The method for developing robot guidance platform software according to claim 7, characterized in that: In the step S500, the user interface module includes code encapsulation of different types of controls, and the codes of all controls have the same basic function interface; The user interface module is used to implement related operations on controls by calling basic function interfaces, and the related operations include adding and deleting controls, setting control properties, and refreshing data.

9. The method for developing robot guidance platform software according to claim 8, characterized in that: In step S600, tasks are divided into main tasks and logic tasks; The main task is a task used for running, and includes information about all tools that need to be run; the logic task is included in the main task; and the logic task is bound to the corresponding logic tool.

10. The method for developing robot guidance platform software according to claim 9, characterized in that: The software operation control module provides an interface for the final real-time operation of the robot guidance platform software, including cyclic operation, single operation and stop operation.

Images