Robot operation process development method and device

Abstract

The invention belongs to the technical field of nuclear power, and particularly relates to a robot operation process development method and device. According to the method, the process point location is formed according to the CAD design file of the robot and the application scene of the robot, the consistency of the robot operation process flow and original design parameters of the robot, the operation object and the operation environment is improved, rework debugging caused by inconsistency is effectively reduced, and development requirements are better met; the generation of the technological process and the selection of the technological point location are both generated by selecting dragging through a graphical interface, and the robot executable code is automatically formed according to the finally generated technological process, so that the editing process is visual and efficient, the complexity of technological programming and scene space selection is remarkably reduced, the development efficiency is improved, and the development cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of nuclear power technology, and in particular relates to a robot operation process development method and device. Background Art

[0002] Robotic process development refers to the development and optimization of various process operations within the robotic manufacturing process, performed according to specific steps and specifications. Traditional robotic process development relies on robot scripts or secondary development using the robotic SDK. This approach played an important role in the early stages of robotics development, but with the continuous advancement of technology and the increasing complexity of application scenarios, its drawbacks have gradually become apparent.

[0003] Under the traditional development model, developers need to develop each robot project from scratch, which leads to a large amount of repetitive work, increased labor costs and technical difficulty. Furthermore, compared with other fields, the nuclear industry has higher requirements for the operating accuracy and safety of nuclear-related robots. The operations of nuclear-related robots differ significantly from those of conventional robots. This requires a large amount of customized development, making it difficult to quickly adjust and optimize robot functions. This also prolongs the development cycle, increases code complexity and the difficulty of subsequent maintenance and upgrades. In view of these factors, there is an urgent need to explore new development models and technical means to improve development efficiency, reduce development costs, and better adapt to market demand. Summary of the Invention

[0004] In order to overcome the problems existing in the related art, a method and device for developing a robot operation process are provided.

[0005] According to one aspect of an embodiment of the present disclosure, a method for developing a robotic operation process is provided, the method comprising:

[0006] Step 1: Display the process point picking interface according to the instruction of picking process points. The process point picking interface is used to parse and generate a simulation scene according to the CAD design file of the imported process scene, and determine the process point corresponding to the interactive operation based on the selection of any area in the displayed simulation scene. The process point includes the graphic elements in the process point and their position and posture information;

[0007] Step 2: Display the process flow editing interface according to the process flow editing instructions. The process flow editing interface is used to form a process flow by dragging and combining process components in the interface and editing parameters. Process components are used to control the robot to achieve preset actions. The process flow consists of multiple process components in a logical order.

[0008] Step 3: The process flow editing interface is also used to synchronously display the process points generated by the process point picking interface, and bind the process nodes in the process flow with the corresponding process points according to the binding operation to generate the final process flow;

[0009] Step 4: Generate a robot executable script based on the generated process flow time sequence list and script configuration parameters.

[0010] In one possible implementation, the process point picking interface includes a three-dimensional visualization interface, a simulation scene information interface, a point capture interface, and a process pre-processing interface;

[0011] The simulation scenario information interface is used to display multiple alternative simulation scenarios;

[0012] The three-dimensional visualization interface is used to display the selected simulation scene in a three-dimensional visualization form;

[0013] The point capture interface is used to capture the position and posture information of graphic elements in any area of ​​the simulation scene in an interactive manner. The types of graphic elements include points, lines, and surfaces. The point capture interface captures the position and posture of points in the simulation scene; the position, posture, and direction of lines; and the position and posture of surfaces.

[0014] The pre-processing interface is used to modify and adjust the position and posture information of the captured process points; the pre-processing content includes any one or more of the following:

[0015] Adjust the position and posture of the modification point; adjust the position and posture of the line, adjust the line direction, discretize the line, and adjust the line normal; adjust the position and posture of the surface, discretize the surface, and generate the tangent and normal based on the surface.

[0016] In one possible implementation, the expected features in the simulation environment can be captured as process points in the three-dimensional visualization interface according to selection. The expected features are the geometric features of the graphic elements in the simulation environment, including endpoints, center points, circle centers, edges, tangents, normals, sections, straight lines, curves, planes, and surfaces.

[0017] In one possible implementation, the process flow editing interface includes a process editor interface, a process component library interface, and a component property interface;

[0018] The process component library interface is used to display multiple preset process components. Different process components have different parameters and different operation methods. Each process component can be dragged to the process editor interface, and the parameters of the process component can be filled and modified through the component property interface;

[0019] The process editor interface is used for real-time visual display. The process components dragged into the process editor interface form the preliminary framework of the process flow according to the logical order designed by the user.

[0020] The component property interface is used to edit the properties of the process component selected in the process editor interface according to the editing instructions.

[0021] In a possible implementation, the process component type is: end motion; the process component parameters are: target position, target speed; the parameter types are: point position, floating point type;

[0022] The process component type is: joint motion; the process component parameters are: joint ID, target position, target speed; the parameter types are: integer, point, floating point;

[0023] The process component type is: IO tool control; the process component parameter is: IO point; the parameter type is: IO address;

[0024] The process component type is: delay; the process component parameter is: delay time; the parameter type is: floating point;

[0025] The process component type is: conditional judgment; the process component parameter is: judgment condition; the parameter type is: Boolean;

[0026] The process component type is: for loop; the process component parameter is: number of loops; the parameter type is: integer;

[0027] The process component type is: While loop; the process component parameter is: loop condition; the parameter type is: Boolean.

[0028] In one possible implementation, in the process editor interface, the process flow is represented in the form of a tree, and each process component is a node of the tree, and the node has a unique ID; the process components form a continuous process flow through the parent-child node relationship; each process component includes a foreground component graphic control, background component operating parameters and component operating logic.

[0029] In one possible implementation, the process flow editing interface also includes a process point interface, which synchronizes and displays the process points generated by the process point picking interface. Each attribute in the component attribute interface can provide a drop-down box to display the process points that are compatible with the function of the process component.

[0030] In a possible implementation, step 4 further includes: generating a script executable by the robot controller according to the time sequence list of the process flow, and configuring the generated script, where different types of robots correspond to different types of robot script formats.

[0031] In a possible implementation, step 4 further includes:

[0032] Step 41, traverse each node in the process flow and determine the control type of each node;

[0033] Step 42: determine the control type of each node according to the control target of the node, read the parameters corresponding to the control type of the node, and generate a script according to the parsing rules corresponding to the control type.

[0034] In one possible implementation, step 42 includes:

[0035] Step 421: If the control type of the node is process control, the process control type of the node is parsed, the process control parameters of the node are read, and the script template corresponding to the process control type is called to fill in the read parameters to generate a process control script;

[0036] Step 422: If the control type of the node is motion control, the motion control type of the node is parsed, the motion control parameters of the node are read, and the script template corresponding to the motion control type is called to fill in the read parameters to generate a motion control script.

[0037] Step 423: If the control type of the node is IO control, the IO control parameters of the node are read, and a script corresponding to the IO control type is called to fill in the read parameters to generate an IO control script.

[0038] In one possible implementation, the method further includes: step 5, verifying the generated process flow, simulating and verifying the points passed in the process flow, outputting a verification report based on the verification results, and displaying deviation information when deviations occur in the verification to assist in adjusting the original process flow. The process editor interface has a simulation control button that can perform process simulation flow control, including start, pause, stop and other control actions.

[0039] In one possible implementation, step 5 also includes step 51, process simulation, which simulates and runs the final generated process flow, breaks the process flow into time series at fixed time intervals, determines the position and output of the robot at each moment in the time series, and updates the position of the robot in the three-dimensional scene to generate a process animation.

[0040] In one possible implementation, step 5 also includes step 52, controller-in-the-loop simulation, connecting the robot controller to the simulation verification loop of the running development platform. According to the determined process flow, the robot controller analyzes the process flow and issues execution instructions, and the simulation scene of the three-dimensional visualization interface accepts the controller instructions and performs synchronous movement.

[0041] In one possible implementation, the development platform configures a communication interface with the controller and establishes a socket-based Ethernet data channel based on the API provided by the controller. After connecting to the actual robot controller, the development platform uses this data channel to issue the process flow and receive data returned by the controller. The robot controller returns position instructions for each axis and IO output control instructions, which can be synchronized in the development platform's 3D visual simulation interface.

[0042] In a possible implementation, step 5 further includes: discretizing the process flow at fixed time intervals, and performing verification on each discretized process point by sequentially performing the following steps:

[0043] Step 53: Call the robot inverse solution algorithm to determine whether there is an inverse solution for the process flow;

[0044] Step 54: if it is determined that the process flow does not have an inverse solution, outputting a verification result indicating that the process flow verification has passed;

[0045] Step 55: If it is determined that the process flow has an inverse solution, a singularity determination algorithm is called to calculate the Jacobian matrix to check whether the process flow has a singularity;

[0046] Step 56: If it is determined that the process flow has an inverse solution, the forward solution algorithm is called according to the joint angles in the process flow to calculate the position of each connecting rod, and the collective collision detection algorithm is called according to the position of each connecting rod to verify whether the process flow generates a collision;

[0047] Step 57: Output the verification results of step 55 and step 56. The verification results include the accessibility, singularity, and collision interference of the specified points under the configuration.

[0048] In a possible implementation, the process flow editing interface further includes a process template interface, which displays multiple alternative process templates. Step 2 further includes:

[0049] Step 21: The process flow editing interface forms a process flow according to the process template selected in the process template interface and the related parameter editing operations;

[0050] Step 22: Create a new process template based on the newly generated process flow in the process flow editing interface and add it to the process module interface.

[0051] In one possible implementation, both the process template and the process flow are XML files. The process flow editing interface sets a process flow XML parser and generator, so that the process editor has the function of converting XML and process flow tree into each other; when the process editor imports and opens templates and process files, the XML file parser is used to parse the XML file into a process flow tree data structure; when the process editor is saved as a file or template, the XML file generator is used to generate the process flow tree as an XML file for saving.

[0052] According to another aspect of an embodiment of the present disclosure, a robot operation process development device is provided, the device comprising:

[0053] The process point picking module is used to display the process point picking interface according to the instruction of picking the process point. The process point picking interface is used to parse and generate the simulation scene according to the CAD design file of the imported process scene, and determine the process point corresponding to the interactive operation based on the selection of any area in the displayed simulation scene. The process point includes the graphic elements and their position and posture information;

[0054] The process flow editing module is used to display the process flow editing interface according to the instructions for editing the process flow. The process flow editing interface is used to form a process flow by dragging and combining process components in the interface and editing parameters. The process components are used to control the robot to achieve preset actions. The process flow is composed of multiple process components in a logical order;

[0055] The binding module is used to control the process flow editing interface to synchronously display the process points generated by the process point picking interface, and bind the process nodes in the process flow with the corresponding process points according to the binding operation to generate the final process flow;

[0056] The script generation module is used to generate a robot executable script based on the generated process flow time sequence list and script configuration parameters.

[0057] According to another aspect of an embodiment of the present disclosure, a robot operation process development device is provided, the device comprising:

[0058] processor;

[0059] a memory for storing processor-executable instructions;

[0060] The processor is configured to execute the above method.

[0061] According to another aspect of an embodiment of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above method is implemented.

[0062] The beneficial effects of the present disclosure are as follows: the robot operation process development method provided by the present disclosure adopts a process point picking interface to import a CAD design file of a parsed process scene to generate a simulation scene, and determines the process points of the simulation scene according to the user's selection in the simulation scene, adopts a process flow editing interface to form a process flow according to the user's drag combination and parameter editing operations on the process components in the interface, and synchronizes and displays the process points generated by the process point picking interface, binds the process nodes and corresponding process points according to the user's needs, and forms a final process flow. The present disclosure forms process points based on the CAD design files of the robot and its application scene, which is conducive to improving the consistency of the robot operation process flow with the original design parameters of the robot, the operation object and the operation environment, effectively reducing the rework and debugging caused by inconsistencies, and better adapting to development needs; the generation of the process flow and the selection of the process points are both generated by dragging and dropping through the graphical interface, and the robot executable code is automatically generated according to the final generated process flow. The editing process is intuitive and efficient, significantly reducing the complexity of process programming and scene space selection, improving development efficiency and reducing development costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flowchart of a method for developing a robot operation process shown in an embodiment of the present disclosure.

[0064] Figure 2 It is a schematic diagram of the process point picking interface shown in an embodiment of the present disclosure.

[0065] Figure 3 It is a schematic diagram of the process flow editing interface shown in an embodiment of the present disclosure.

[0066] Figure 4 It is a flowchart of another robot operation process development method shown in an embodiment of the present disclosure.

[0067] Figure 5 It is a flowchart of the robot script generation shown in an embodiment of the present disclosure.

[0068] Figure 6 Schematic diagram of the controller-in-the-loop simulation functional architecture shown in an embodiment of the present disclosure.

[0069] Figure 7 It is a flow chart of process flow verification shown in an embodiment of the present disclosure.

[0070] Figure 8 It is a block diagram of a robot operation process development device shown in an embodiment of the present disclosure.

[0071] Figure 9 It is a block diagram of another robot operation process development device shown in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0072] The present disclosure will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] Unless otherwise defined, the technical and scientific terms used in this disclosure have the same meanings as those generally understood by those skilled in the art to which this disclosure belongs; the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit this disclosure; the term "including" and any variations thereof in this disclosure are intended to cover non-exclusive inclusions. Obviously, the embodiments described in this disclosure are only some of the embodiments of this disclosure, not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those of ordinary skill in the art without making any creative work are within the scope of protection of this disclosure.

[0074] References to "embodiments" in this disclosure mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the disclosure. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0075] Figure 1 This is a flow chart of a method and apparatus for developing a robot operation process shown in an embodiment of the present disclosure. The method can be executed by a terminal device, wherein the terminal device can be a server, a desktop computer, a laptop computer, a tablet computer, etc. The terminal device can also be, for example, a wearable device based on AR or VR. The embodiment of the present disclosure does not limit the type of terminal device. Figure 1 As shown, the method includes:

[0076] Step 1: The terminal device displays a process point picking interface based on the user's instruction to pick a process point to the development platform. The process point picking interface is used to parse and generate a simulation scene based on the imported CAD design file of the process scene, and determine the process point corresponding to the interactive operation based on the selection of any area in the displayed simulation scene. The process point includes the graphic elements in the process point and their position and posture information;

[0077] As an example of this embodiment, Figure 2 As shown, the process point picking interface includes a 3D visualization interface, a simulation scene information interface, a point capture interface, and a process preprocessing interface;

[0078] The simulation scene information interface is used to display multiple alternative simulation scenes (the simulation scene may include components of the robot itself, robot operation objects, or scenes where the robot performs tasks, etc. This disclosure does not limit the specific form of the simulation scene, as long as the simulation scene is related to the process flow).

[0079] The 3D visualization interface is used to display the selected simulation scene in a 3D visualization format. After the user selects the simulation scene to be displayed on the simulation scene information interface, the design platform can use the CAD engine to import and parse the corresponding CAD design file of the scene and display the corresponding simulation scene on the simulation scene information interface.

[0080] The point capture interface is used to capture the position and posture information of graphic elements in any area of ​​the simulation scene in an interactive manner. For example, the interactive operation can be clicking, circling, and other operations on the simulation scene interface, and according to the position information corresponding to the interactive operation, the process point corresponding to the position information and its position and posture information are obtained; the types of graphic elements include points, lines, and surfaces, which can capture the position and posture of points in the simulation scene; the position, posture, and direction of lines; and the position and posture of surfaces.

[0081] In a possible implementation, after selecting a graphic element category in the point capture interface, a graphic element of the corresponding category can be selected in the three-dimensional visualization interface.

[0082] In one possible implementation, see Figure 3 and Figure 4 The pre-processing interface is used to modify and adjust the position and posture information of the captured process points. The pre-processing content includes any one or more of the following:

[0083] Adjust the position and posture of the modification point; adjust the position and posture of the line, adjust the line direction, discretize the line, and adjust the line normal; adjust the position and posture of the surface, discretize the surface, and generate the tangent and normal based on the surface.

[0084] In one possible implementation, the expected features in the simulation environment can be captured as process points according to selection in the three-dimensional visualization interface. The expected features are the geometric features of the graphic elements in the simulation environment, including endpoints, center points, center points of circles, edges, tangents, normals, sections, straight lines, curves, planes, curved surfaces, etc. In this way, the positions of the operating objects and motion environments associated with the robot process flow can be located more accurately as needed, which significantly improves the accuracy of the robot's execution of process actions (such as alignment, grasping, etc.) and can effectively reduce additional positioning auxiliary operations in the process flow.

[0085] Step 2: The terminal device displays the process flow editing interface according to the user's instruction to edit the process flow to the development platform. The process flow editing interface is used to form a process flow by dragging and combining process components in the interface and editing parameters. The process components are used to control the robot to perform preset actions (such as Figure 2 As shown, the process components can be joint motion, end motion, IO tool control, delay, condition judgment, loop, etc. The present disclosure does not limit the specific form of the process components). The process flow is composed of multiple process components in a logical order, which can be expressed as the sequence and process logic of the robot operation process steps.

[0086] As an example of this embodiment, Figure 3 As shown, the process flow editing interface includes the process editor interface, the process component library interface, and the component property interface; the process component library interface is used to display multiple preset process components. The process component library is the basis of the process design function and stores process components in the form of a library. Different process components have different parameters and different operation methods. Each process component can be dragged into the process editor interface. The parameters of the process component can be filled and modified through the component property interface. The operation method of each process component corresponds to the component type, thereby avoiding incompatibility between the process component operation method and the component type. The process editor interface is used for real-time visual display. The process components dragged into the process editor interface form the preliminary framework of the process flow according to the logical order designed by the user. The user can drag and adjust the position of each process component in the process editor interface. The component property interface is used to edit the properties of the selected process component in the process editor interface according to the editing instructions. For example, after the user clicks on the process component in the process editor interface, the property value of the process component will be displayed in real time in the component property bar. The user can edit the relevant parameters and properties of the process component (for example, the starting point, end point, movement mode, movement speed, etc. of the process component).

[0087] For example, the parameters of each process component are listed in Table 1.

[0088] Table 1

[0089] Serial number Process component type Process component parameters Parameter Type 1. Terminal motion Target position, target speed Point, floating point 2. Joint movement Joint ID, target position, target velocity Integer, point, floating point 3. IO tool control IO points IO address 4. Delay Delay time floating-point type 5. Conditional judgment Judgment conditions Boolean 6. for loop Number of cycles Integer 7. While Loop Loop Condition Boolean

[0090] In the process editor interface, the user-designed process flow is represented in the form of a tree. Each process component is a node of the tree. The node has a unique ID, which is used to determine the unique ID when arranging the process. The process components form a continuous process flow through the parent-child node relationship. In each process flow, starting from the root node of the tree and traversing downwards, the entire process flow can be completed. Each process component includes a foreground component graphic control, a background component operating parameter, and a component operating logic method.

[0091] It should be noted that step 1 and step 2 may be performed simultaneously or in sequence, for example, step 1 may be performed before step 2, or step 1 may be performed after step 2.

[0092] Step 3: The process flow editing interface is also used to synchronously display the process points generated by the process point picking interface, and according to the binding operation, bind the process nodes in the process flow with the corresponding process points, fill the processed process points into the time series list of the process flow, and generate the final process flow.

[0093] Step 4: Generate a robot executable script based on the generated process flow time sequence list and script configuration parameters.

[0094] The present invention forms process points based on the CAD design files of the robot and its application scenarios, which is conducive to improving the consistency of the robot's operation process flow with the original design parameters of the robot, the operation object and the working environment, effectively reducing rework and debugging caused by inconsistencies, and better adapting to development needs; the generation of the process flow and the selection of process points are both generated by dragging and dropping in the graphical interface, and the robot executable code is automatically generated according to the final generated process flow. The editing process is intuitive and efficient, which significantly reduces the complexity of process programming and scene space selection, improves development efficiency and reduces development costs.

[0095] As an example of this embodiment, see Figure 3 The process flow editing interface also includes a process point interface. The process point interface can synchronize and display the process points generated by the process point picking interface. Each attribute in the component attribute interface can provide a drop-down box to display the process points that are compatible with the function of the process component for users to choose from. For example, each prevents the occurrence of incompatibility between process nodes and process points.

[0096] In a possible implementation, step 4 also includes: generating a script executable by the robot controller according to the time sequence list of the process flow designed by the user, and configuring the generated script, with different types of robots corresponding to different types of robot script formats.

[0097] Figure 5This is a schematic diagram of the process of generating a robot script according to an embodiment of the present disclosure. Figure 5 As shown, step 4 also includes: step 41 and step 42.

[0098] Step 41: traverse each node in the process flow and determine the control type of each node.

[0099] Step 42: determine the control type of each node according to the control target of the node, read the parameters corresponding to the control type of the node, and generate a script according to the parsing rules corresponding to the control type.

[0100] For example, step 42 includes: step 421 to step 423.

[0101] Step 421, if the control type of the node is process control, then parse the process control type of the node (for example, delay, judgment, loop), read the process control parameters of the node, and call the script template corresponding to the process control type to fill in the read parameters to generate a process control script.

[0102] Step 422: If the control type of the node is motion control, the motion control type of the node (e.g., MoveL, MoveC, MoveJ) is parsed, the motion control parameters of the node are read, and the script template corresponding to the motion control type is called to fill in the read parameters to generate a motion control script.

[0103] Step 423: If the control type of the node is IO control, the IO control parameters of the node are read, and a script corresponding to the IO control type is called to fill in the read parameters to generate an IO control script.

[0104] In this way, the present disclosure can translate the process flow described in the process flow tree into the robot's control instructions in a targeted manner according to the robot's control objectives (IO control, motion control) and motion types (end straight line, end arc, joint motion), and output it as a robot script, thereby improving conversion efficiency and reducing error rate.

[0105] In one possible implementation, the method further includes: Step 5, verifying the generated process flow, performing simulation and verification on the points passed through in the process flow, outputting a verification report based on the verification results, and displaying deviation information when deviations occur during the verification to assist in adjusting the original process flow. The process editor interface has simulation control buttons that can control the process simulation flow, including control actions such as start, pause, and stop.

[0106] In one possible implementation, step 5 also includes step 51, process simulation, which simulates the process flow finally generated, breaks the process flow into time series at fixed time intervals, determines the position and output of the robot at each moment in the time series, and updates the position of the robot in the three-dimensional scene to generate a continuous and smooth process animation.

[0107] In one possible implementation, step 5 also includes step 52, controller-in-the-loop simulation, which connects the robot controller to the simulation verification loop of the development platform running on the terminal device. Based on the determined process flow, the robot controller analyzes the process flow and issues execution instructions. The simulation scene in the 3D visualization interface receives the controller instructions and performs synchronous motion. The controller-in-the-loop simulation establishes communication between the robot controller and the development platform. This allows for real-world verification of the rationality and effectiveness of the process flow through real-world feedback from the robot controller.

[0108] Figure 6 This is a schematic diagram of the controller-in-the-loop simulation functional architecture shown in an embodiment of the present disclosure. The development platform is configured with a communication interface with the controller, and a socket-based Ethernet data channel can be established according to the API provided by the controller. After connecting the actual robot controller, the development platform can complete the issuance of the process flow (robot script or process instruction) and the reception of the controller return data based on this data channel. The robot controller returns the position instructions of each axis and the output control instructions of the IO. The position instructions and the output control instructions of the IO can be synchronized in the three-dimensional visual simulation interface of the development platform.

[0109] After the robot controller is connected to the development platform, it should still function normally and output control commands even when the robot is not connected. The development platform can control the running process of the in-the-loop simulation through simulation control buttons, including starting, pausing, and stopping.

[0110] Figure 7 This is a flow chart of process flow verification shown in an embodiment of the present disclosure. Step 5 also includes: discretizing the process flow at fixed time intervals, and verifying each discrete process point using the following steps in sequence:

[0111] Step 53: Call the robot inverse solution algorithm to determine whether there is an inverse solution for the process flow.

[0112] Step 54 : If it is determined that the process flow does not have an inverse solution, a verification result indicating that the process flow has passed verification is output.

[0113] Step 55: When it is determined that the process flow has an inverse solution, a singularity determination algorithm is called to calculate the Jacobian matrix to check whether the process flow has a singularity.

[0114] Step 56: If it is determined that the process flow has an inverse solution, the forward solution algorithm is called according to the joint angles in the process flow to calculate the position of each link, and the collective collision detection algorithm is called according to the position of each link to verify whether the process flow generates a collision.

[0115] Step 57: Output the verification results of step 55 and step 56. The verification results include the accessibility, singularity, and collision interference of the specified points under the configuration.

[0116] In a possible implementation, the process flow editing interface further includes a process template interface, which displays a plurality of alternative process templates, and step 2 further includes steps 21 and 22.

[0117] Step 21: The process flow editing interface generates a process flow based on the selected process template and related parameter editing operations in the process template interface. The present disclosure can further integrate functions of mature processes through pre-set process templates, thereby further improving process development efficiency.

[0118] Step 22, based on the newly generated process flow in the process flow editing interface, a new process template is formed and added to the process module interface, thereby iteratively updating the process template according to each process flow development to better adapt to the needs of technology updates.

[0119] As an example of this embodiment, the process template and the process flow are both XML files, and the process flow editing interface sets a process flow XML parser and generator, so that the process editor has the function of converting between XML and process flow tree.

[0120] When importing and opening templates and process files in the process editor, the XML file parser is used to parse the XML file into a process flow tree data structure.

[0121] When the process editor is saved as a file or template, the process flow tree can be generated as an XML file and saved using the XML file generator.

[0122] According to another aspect of an embodiment of the present disclosure, a robot operation process development device is provided, the device comprising:

[0123] The process point picking module is used to display the process point picking interface according to the instruction of picking the process point. The process point picking interface is used to parse and generate the simulation scene according to the CAD design file of the imported process scene, and determine the process point corresponding to the interactive operation based on the selection of any area in the displayed simulation scene. The process point includes the graphic elements and their position and posture information;

[0124] The process flow editing module is used to display the process flow editing interface according to the instructions for editing the process flow. The process flow editing interface is used to form a process flow by dragging and combining process components in the interface and editing parameters. The process components are used to control the robot to achieve preset actions. The process flow is composed of multiple process components in a logical order;

[0125] The binding module is used to control the process flow editing interface to synchronously display the process points generated by the process point picking interface, and bind the process nodes in the process flow with the corresponding process points according to the binding operation to generate the final process flow;

[0126] The script generation module is used to generate a robot executable script based on the generated process flow time sequence list and script configuration parameters.

[0127] The description of the above-mentioned device has been explained in detail in the description of the above-mentioned method and will not be repeated here.

[0128] Figure 8 It is a block diagram of a robot operation process development device shown in an embodiment of the present disclosure. For example, the device 800 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0129] Reference Figure 8 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0130] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the device described above. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0131] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or device operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0132] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 800.

[0133] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0134] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0135] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0136] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800. The sensor assembly 814 can also detect changes in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and temperature changes of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0137] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0138] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0139] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions that can be executed by the processor 820 of the apparatus 800 to perform the above method.

[0140] Figure 9 1 is a block diagram of another robot operation process development device according to an embodiment of the present disclosure. For example, the device 1900 can be provided as a server. Figure 9The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0141] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output (I / O) interface 1958. The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.

[0142] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the apparatus 1900 to perform the above-described method.

[0143] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0144] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0145] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0146] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0147] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0148] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0149] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0150] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0151] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for developing a robot operation process, characterized in that: The method comprises: Step 1: Display the process point picking interface according to the instruction of picking process points. The process point picking interface is used to parse and generate a simulation scene according to the CAD design file of the imported process scene, and determine the process point corresponding to the interactive operation based on the selection of any area in the displayed simulation scene. The process point includes the graphic elements in the process point and their position and posture information; Step 2: Display the process flow editing interface according to the process flow editing instructions. The process flow editing interface is used to form a process flow by dragging and combining process components in the interface and editing parameters. Process components are used to control the robot to achieve preset actions. The process flow consists of multiple process components in a logical order. Step 3: The process flow editing interface is also used to synchronously display the process points generated by the process point picking interface, and bind the process nodes in the process flow with the corresponding process points according to the binding operation to generate the final process flow; Step 4: Generate a robot executable script based on the generated process flow time sequence list and script configuration parameters.

2. The method according to claim 1, characterized in that The process point picking interface includes a 3D visualization interface, a simulation scene information interface, a point capture interface, and a process pre-processing interface; The simulation scenario information interface is used to display multiple alternative simulation scenarios; The three-dimensional visualization interface is used to display the selected simulation scene in a three-dimensional visualization form; The point capture interface is used to capture the position and posture information of graphic elements in any area of ​​the simulation scene in an interactive manner. The types of graphic elements include points, lines, and surfaces. The point capture interface captures the position and posture of points in the simulation scene; the position, posture, and direction of lines; and the position and posture of surfaces. The pre-processing interface is used to modify and adjust the position and posture information of the captured process points; the pre-processing content includes any one or more of the following: Adjust the position and posture of the modification point; adjust the position and posture of the line, adjust the line direction, discretize the line, and adjust the line normal; adjust the position and posture of the surface, discretize the surface, and generate the tangent and normal based on the surface.

3. The method according to claim 1, characterized in that In the 3D visualization interface, you can capture the expected features in the simulation environment as process points according to your choice. The expected features are the geometric features of the graphic elements in the simulation environment, including endpoints, center points, circle centers, edges, tangents, normals, sections, straight lines, curves, planes, and surfaces.

4. The method according to claim 1, wherein The process flow editing interface includes the process editor interface, process component library interface, and component property interface; The process component library interface is used to display multiple preset process components. Different process components have different parameters and different operation methods. Each process component can be dragged to the process editor interface, and the parameters of the process component can be filled and modified through the component property interface; The process editor interface is used for real-time visual display. The process components dragged into the process editor interface form the preliminary framework of the process flow according to the logical order designed by the user. The component property interface is used to edit the properties of the process component selected in the process editor interface according to the editing instructions.

5. The method according to claim 1, wherein The process component type is: end motion; the process component parameters are: target position, target speed; the parameter types are: point position, floating point; The process component type is: joint motion; the process component parameters are: joint ID, target position, target speed; the parameter types are: integer, point, floating point; The process component type is: IO tool control; the process component parameter is: IO point; the parameter type is: IO address; The process component type is: delay; the process component parameter is: delay time; Parameter type: floating point; The process component type is: conditional judgment; the process component parameter is: judgment condition; the parameter type is: Boolean; The process component type is: for loop; the process component parameter is: number of loops; the parameter type is: integer; The process component type is: While loop; the process component parameter is: loop condition; the parameter type is: Boolean.

6. The method according to claim 1, wherein In the process editor interface, the process flow is represented in the form of a tree. Each process component is a node in the tree, and each node has a unique ID. The process components form a continuous process flow through the parent-child node relationship; each process component includes a foreground component graphic control, background component operation parameters and component operation logic.

7. The method according to claim 1, characterized in that The process flow editing interface also includes a process point interface, which synchronizes and displays the process points generated by the process point picking interface. Each attribute in the component attribute interface can provide a drop-down box to display the process points that are compatible with the function of the process component.

8. The method according to claim 1, characterized in that Step 4 also includes: generating a script executable by the robot controller according to the time sequence list of the process flow, and configuring the generated script. Different types of robots correspond to different types of robot script formats.

9. The method according to claim 1, characterized in that Step 4 also includes: Step 41, traverse each node in the process flow and determine the control type of each node; Step 42: determine the control type of each node according to the control target of the node, read the parameters corresponding to the control type of the node, and generate a script according to the parsing rules corresponding to the control type.

10. The method according to claim 9, characterized in that Step 42 includes: Step 421: If the control type of the node is process control, the process control type of the node is parsed, the process control parameters of the node are read, and the script template corresponding to the process control type is called to fill in the read parameters to generate a process control script; Step 422: If the control type of the node is motion control, the motion control type of the node is parsed, the motion control parameters of the node are read, and the script template corresponding to the motion control type is called to fill in the read parameters to generate a motion control script. Step 423: If the control type of the node is IO control, the IO control parameters of the node are read, and a script corresponding to the IO control type is called to fill in the read parameters to generate an IO control script.

11. The method according to claim 1, wherein The method also includes: step 5, verifying the generated process flow, simulating and verifying the points passed in the process flow, outputting a verification report based on the verification results, and displaying deviation information when deviations occur in the verification to assist in adjusting the original process flow. The process editor interface has a simulation control button that can control the process simulation flow, including control actions such as start, pause, and stop.

12. The method according to claim 11, characterized in that Step 5 also includes step 51, process simulation, which simulates the process flow finally generated, breaks the process flow into time series at fixed time intervals, determines the position and output of the robot at each moment in the time series, and updates the position of the robot in the three-dimensional scene to generate a process animation.

13. The method according to claim 1, wherein Step 5 also includes step 52, controller-in-the-loop simulation, which connects the robot controller to the simulation verification loop of the running development platform. According to the determined process flow, the robot controller analyzes the process flow and issues execution instructions. The simulation scene of the three-dimensional visualization interface accepts the controller instructions and performs synchronous movement.

14. The method according to claim 13, characterized in that The development platform configures the communication interface with the controller and can establish a socket-based Ethernet data channel according to the API provided by the controller; After connecting to the actual robot controller, the development platform completes the issuance of process flow and the reception of controller return data based on this data channel; the robot controller returns the position instructions of each axis and the output control instructions of the IO. The position instructions and the output control instructions of the IO can be synchronized in the 3D visual simulation interface of the development platform.

15. The method according to claim 13, characterized in that Step 5 also includes: discretizing the process flow at fixed time intervals, and verifying each discretized process point by sequentially performing the following steps: Step 53: Call the robot inverse solution algorithm to determine whether there is an inverse solution for the process flow; Step 54: if it is determined that the process flow does not have an inverse solution, outputting a verification result indicating that the process flow verification has passed; Step 55: If it is determined that the process flow has an inverse solution, a singularity determination algorithm is called to calculate the Jacobian matrix to check whether the process flow has a singularity; Step 56: If it is determined that the process flow has an inverse solution, the forward solution algorithm is called according to the joint angles in the process flow to calculate the position of each connecting rod, and the collective collision detection algorithm is called according to the position of each connecting rod to verify whether the process flow generates a collision; Step 57: Output the verification results of step 55 and step 56. The verification results include the accessibility, singularity, and collision interference of the specified points under the configuration.

16. The method according to claim 1, wherein The process flow editing interface also includes a process template interface, which displays multiple alternative process templates. Step 2 also includes: Step 21: The process flow editing interface forms a process flow according to the process template selected in the process template interface and the related parameter editing operations; Step 22: Create a new process template based on the newly generated process flow in the process flow editing interface and add it to the process module interface.

17. The method according to claim 16, characterized in that Both process templates and process flows are XML files. The process flow editing interface sets the process flow XML parser and generator, so that the process editor has the function of converting between XML and process flow tree. When importing or opening templates and process files in the process editor, the XML file parser is used to parse the XML file into a process flow tree data structure; when saving the process editor as a file or template, the XML file generator is used to generate the process flow tree as an XML file for saving.

18. A robot operation process development device, characterized in that: The device comprises: The process point picking module is used to display the process point picking interface according to the instruction of picking the process point. The process point picking interface is used to parse and generate the simulation scene according to the CAD design file of the imported process scene, and determine the process point corresponding to the interactive operation based on the selection of any area in the displayed simulation scene. The process point includes the graphic elements and their position and posture information; The process flow editing module is used to display the process flow editing interface according to the instructions for editing the process flow. The process flow editing interface is used to form a process flow by dragging and combining process components in the interface and editing parameters. The process components are used to control the robot to achieve preset actions. The process flow is composed of multiple process components in a logical order; The binding module is used to control the process flow editing interface to synchronously display the process points generated by the process point picking interface, and bind the process nodes in the process flow with the corresponding process points according to the binding operation to generate the final process flow; The script generation module is used to generate a robot executable script based on the generated process flow time sequence list and script configuration parameters.

19. A robot operation process development device, characterized in that: The device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 17.

20. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 17 is implemented.

Images