Multi-robot construction man-machine interaction method based on mixed reality technology

Abstract

The invention provides a multi-robot construction man-machine interaction method based on a mixed reality technology. The method comprises the following steps: 1, interactive design: aligning mixed reality and a construction environment, adjusting component design, and registering a designed result; a second step of robot task allocation: marking a plurality of robots and corresponding components with different colors respectively, and establishing a most suitable corresponding relationship for the robots and the components; and 3, robot path planning: observing the whole motion process of the robot through mixed reality, and adjusting the positions of the robot base and the tail end to avoid collision. Compared with an existing method, the method has the advantages that the learning cost is greatly reduced, and the design and construction integration task oriented to multi-robot man-machine cooperation is completed more visually and quickly.

Description

Technical Field

[0001] The present invention relates to the application of mixed reality, particularly to the application of mixed reality in the field of robot construction, and more particularly to a multi-robot construction human-computer interaction method based on mixed reality technology. Background Art

[0002] In recent years, the rapid development of robot technology has made it possible for robots to be applied in the construction field, especially in multi-robot collaboration. Through collaborative operations, multi-robot systems can significantly improve the efficiency and accuracy of construction operations, reduce labor costs, and optimize resource allocation. However, there are still some challenges in the application of existing technologies in multi-robot construction, especially in the interaction between robots and the construction environment.

[0003] Currently, most construction tasks of multi-robot collaboration rely on traditional programming methods for path planning and task allocation. This method not only requires a high technical threshold but also lacks an intuitive operation interface, making it necessary for operators to accumulate a large amount of time and experience in task planning, execution, and adjustment. In addition, there are also major technical problems in the docking between robots and the environment, task allocation, and real-time feedback during the collaboration process. Especially in the case of a complex construction environment with numerous components, how to effectively communicate with the robot system through an interaction interface has become the key to improving the efficiency and accuracy of robot construction.

[0004] The introduction of augmented reality (AR) and mixed reality (MR) technologies provides new solutions to these problems. Through AR / MR technologies, real-time interaction with the real construction environment can be achieved in a virtual space, enabling more intuitive and convenient design and adjustment. In summary, based on the existing technologies, the present invention further explores the human-computer interaction method in multi-robot construction assisted by mixed reality. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems in the existing technologies and propose a multi-robot construction human-computer interaction method based on mixed reality technology.

[0006] The present invention is realized through the following technical solutions. The present invention proposes a multi-robot construction human-computer interaction method based on mixed reality technology, and the method includes the following steps:

[0007] Interactive Design:

[0008] (1) Align mixed reality with the construction environment by scanning a marker code or setting a spatial anchor point;

[0009] (2) Adjust the component design by adjusting the interactive elements or the cursor through gestures or a controller;

[0010] (3) Judge the design results, and register the selected design results through the interaction behavior of the mixed reality button;

[0011] Robot task allocation:

[0012] (4) Mark multiple robots and their selection cursors with different colors respectively to facilitate subsequent task allocation;

[0013] (5) Use the cursor to select the corresponding components for each robot;

[0014] (6) Observe the potential collision limits and establish the corresponding relationship between the robots and the components;

[0015] Robot path planning:

[0016] (7) Observe the postures of the corresponding robots at each key position during the process in combination with the construction environment through mixed reality;

[0017] (8) Avoid collisions by changing the postures of the robots by adjusting the cursors that control the positions of the bases and the ends of the corresponding robots;

[0018] (9) Start the robot after previewing and confirming the entire movement process of the corresponding robot through mixed reality.

[0019] Furthermore, in the method, a human-computer interaction system based on mixed reality is established to complete the multi-robot collaborative construction task. The working steps of the human-computer interaction system are as follows:

[0020] 1) Use the kangaroo for structural performance simulation and the houdini plug-in for form generation on the grasshopper platform to create a program according to user requirements, and set the form generation logic and adjustment and change logic of the components to be constructed;

[0021] 2) Integrate the visual and interactive mixed reality program created by the software platform unity toolkit openXR and the grasshopper plug-in fologram with the program in step 1), so that the form adjustment and change are driven by the user's interaction behavior;

[0022] 3) Connect the path points involved in the component construction process in step 1) to the kukaprc or robots robot path simulation program, and integrate the obtained robot motion postures with the mixed reality program in step 2), so that the virtual robots are visualized in the construction environment, and the motion states, postures, bases and end coordinates of the robots can be changed by the user's interaction behavior;

[0023] 4) Transmit the robot motion posture parameters to be executed obtained in step 3) to the robot PLC via the Profinet or Modbus protocol, and connect the control of the transmission behavior to the mixed reality program in step 2) so that the transmission instruction is driven by the user's interaction behavior;

[0024] 5) Program the corresponding robot by writing script code to call the instruction passed into the robot PLC in step 4), so that the robot executes the action obtained after the user interaction behavior in all the above steps.

[0025] Further, the marker code or anchor point in step (1) needs to be preset at a specific position in the construction environment, and the relative positions between the marker codes or anchor points should be ensured to be accurate;

[0026] The interactive element in step (2) should be the component itself or a newly created control point.

[0027] Further, the mixed reality button in step (3) should generate a change in the Boolean value when affected by the interaction behavior, and is used to control component design, robot path simulation and operation.

[0028] Further, the specific marking with different colors in step (4) specifically refers to: different robots are displayed in completely different colors in the mixed reality, and the interactive elements and the assigned components related to each robot have the same color as the corresponding robot.

[0029] Further, the selection method in step (5) is to mark the component closest to the cursor or the component that intersects with the cursor.

[0030] Further, the key positions in step (7) refer to the initial point, the final point, the pause points during the process and the points where the state of the end effector changes of the robot.

[0031] Further, the posture control cursor in step (8) has an invariant relative position relationship with the robot base or the end coordinates, and the latter changes in the same way as the former changes.

[0032] The present invention also proposes an electronic device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the multi-robot construction human-computer interaction method based on mixed reality technology.

[0033] The present invention also proposes a computer-readable storage medium for storing computer instructions, and when the computer instructions are executed by a processor, it implements the steps of the multi-robot construction human-computer interaction method based on mixed reality technology.

[0034] The beneficial effects of the present invention:

[0035] In the interactive design stage, users can directly change the design of components through their physical interaction behaviors, making the design process more intuitive. At the same time, the design is directly carried out at the corresponding position in the construction environment, providing users with the most realistic pre-perception of the construction results of the design; in the robot task allocation stage, users can intuitively distinguish the final pose simulations of different robots, select the cursor and the allocated components through different colors. They can also observe the final poses of multiple robots simultaneously and consider potential collision risks during task allocation; in the robot path planning stage, users can observe the motion poses of robots during the entire construction process in advance through mixed reality and intuitively understand the collision situations between robots and the construction environment. At the same time, they can directly adjust the poses of robots at important process points through interactive control points and see the adjustment results in real time, which is particularly effective for construction tasks that require robots to move a large range during the construction process.

[0036] The present invention can utilize the reliability of robots and the wisdom of humans simultaneously, promoting the replacement of a large number of manual labor by robots while retaining the positive role of manual labor in the automation process. The present invention reduces uncertain work such as complex computer path planning and collision detection, and at the same time provides a more intuitive and user-friendly human-machine interaction method, which greatly reduces the learning cost of robot construction and makes the deployment of multi-robot construction simpler and more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0038] Figure 1 It is a flowchart of a human-machine interaction method for multi-robot construction based on mixed reality technology;

[0039] Figure 2 It is a technical implementation path diagram of a human-machine interaction system established by a human-machine interaction method for multi-robot construction based on mixed reality technology. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0041] Combined withFigure 1 - Figure 2 , the present invention proposes a multi-robot construction human-computer interaction method based on mixed reality technology, and the method includes the following steps:

[0042] Interactive design:

[0043] (1) Align the mixed reality with the construction environment by scanning the marker code or setting the spatial anchor point;

[0044] (2) Adjust the interactive elements or the cursor to adjust the component design through gestures or the controller;

[0045] (3) Judge the design result, and register the selected design result through the interactive behavior of the mixed reality button;

[0046] Robot task allocation:

[0047] (4) Mark multiple robots and their selection cursors with different colors respectively for subsequent task allocation;

[0048] (5) Use the cursor to select the corresponding component for each robot;

[0049] (6) Observe the potential collision limits and establish the corresponding relationship between the robots and the components;

[0050] Robot path planning:

[0051] (7) Observe the postures of the corresponding robots at each key position during the process in combination with the construction environment through the mixed reality;

[0052] (8) Change the robot posture to avoid collision by adjusting the cursor that controls the base and end positions of the corresponding robot;

[0053] (9) Start the robot after previewing and confirming the entire movement process of the corresponding robot through the mixed reality.

[0054] In the method, a human-computer interaction system based on mixed reality is established to complete the multi-robot collaborative construction task, and the working steps of the human-computer interaction system are as follows:

[0055] 1) Use the structure performance simulation kangaroo and the form generation houdini plug-ins of the grasshopper platform to create a program according to the user's needs, and set the form generation logic and adjustment change logic of the components to be built;

[0056] 2) Integrate the visual interactive mixed reality (augmented reality) program created by the software platform unity toolkit openXR and the grasshopper plug-in fologram with the program in step 1), so that the form adjustment and change are driven by the user's interactive behavior;

[0057] 3) Connect the waypoints involved in the component construction process in step 1) to the KUKA PRC or robots robot path simulation program, and integrate the obtained robot motion postures with the mixed reality program in step 2), so that the virtual robot can be visualized in the construction environment, and the robot motion state, posture, base and end coordinates can be changed by the user's interaction behavior;

[0058] 4) Transmit the robot motion postures to be executed obtained in step 3), such as robot joint rotation angles, speeds and other parameters, to the robot PLC through the Profinet or Modbus protocol (through communication methods such as UDP or TCP), and connect the control of the transmission behavior to the mixed reality program in step 2), so that the transmission instructions are driven by the user's interaction behavior;

[0059] 5) Program the corresponding robot by writing script code to call the instructions passed into the robot PLC in step 4), so that the robot executes the actions obtained after the user's interaction behavior in all the above steps.

[0060] The marker codes or anchor points in step (1) need to be preset at specific positions in the construction environment, and the relative positions between the marker codes or anchor points should be ensured to be accurate;

[0061] The interactive elements in step (2) should be the component itself or the control points created additionally.

[0062] The mixed reality button in step (3) should generate a change in the Boolean value when affected by the interaction behavior, and is used to control the component design, robot path simulation and operation.

[0063] The different colors marked in step (4) specifically refer to: different robots display completely different colors in the mixed reality, and the interactive elements and the assigned components related to each robot have the same color as the corresponding robot.

[0064] The selection method in step (5) is to mark the component closest to the cursor or the component that intersects with the cursor.

[0065] The key positions in step (7) refer to the initial point, the final point, the pause points during the process and the points where the state of the end effector changes of the robot.

[0066] The posture control cursor in step (8) has an invariant relative position relationship with the robot base or the end coordinates, and the latter changes in the same way as the former changes.

[0067] Embodiment

[0068] See Figure 1 , the multi-robot construction human-computer interaction method based on the mixed reality technology of the present invention includes the following steps:

[0069] (1) Interactive design: Align the mixed reality with the built environment by scanning the marker code or setting the spatial anchor, and then use gestures or controllers to adjust the scannable and interactive elements or the cursor, so as to modify the component design. Register the selected design result through the interaction behavior with the mixed reality button to complete the feedback and confirmation of the design process.

[0070] The scanned marker code is generally fixed in a planar form on the ground, wall or other planes in the built environment that do not have a relative positional relationship with the environment, and this position needs to be continuously recognized by the camera of the mixed reality device.

[0071] There can be many gestures used, depending on the gesture recognition code used. Generally, common gestures include two-finger pinch, click, grab, zoom in, fist clench, etc.

[0072] The mixed reality button is a virtual interactive geometric body shown in the form of a button, with various shapes, but generally it needs to be attached to a plane in a certain environment and cannot move or follow an object and maintain an unchanged relative position.

[0073] (2) Robot task assignment: To facilitate subsequent task assignment, first mark multiple robots and their selection cursors with different colors, and then select the corresponding components for each robot through the cursor. Observe the potential collision limits during this process to ensure the most suitable correspondence between the robots and the components, thereby optimizing task execution.

[0074] The multiple robots assigned tasks generally refer to the case where the number of robots is greater than or equal to 2.

[0075] The colors used for marking can be freely defined by the user according to the built environment. The selection cursor can be any geometric shape that can be understood as the function of the cursor, and the cursor can be freely moved in space.

[0076] Selecting components needs to be done by moving the cursor. The components to be built need to be divided in advance according to the determined construction logic into parts that can be completed by a single run of a robot. Complete the correspondence of the relationship by setting the spatial relationship between the cursor and each part, such as the closest distance or the intersection of aggregates.

[0077] The observation of potential collision limits is achieved by the user wearing the mixed reality device and observing the influence of virtual robots and real objects in the built environment at the same time.

[0078] (3) Robot path planning: Through mixed reality, observe the combination of the poses of the corresponding robots at key positions during the process and the construction environment. On this basis, change the robot poses to avoid collisions by adjusting and controlling the cursors at the base and end positions of the corresponding robots, and preview the entire movement process of the corresponding robots through mixed reality. After confirmation, start the robots to execute tasks, so as to achieve safe and efficient control of the robot movement.

[0079] The key positions refer to the positions where collisions are likely to occur during the movement of the robots, such as when the speed or movement type changes, or when there are actions of the end effector, etc.

[0080] The cursors for controlling the base and end positions of the robot are similar to the cursors in step (2), and can be any geometric shape that can be understood as the function of the cursor and can be freely moved in space.

[0081] The cursor and the base or end maintain a fixed relative position, and when the cursor moves, the base and end move the same distance in the same direction.

[0082] The observation of potential collision limitations is also achieved by the user wearing a mixed reality device to simultaneously observe the influence of the virtual robot and real objects in the construction environment, including the built parts of the components.

[0083] See Figure 2 , for the multi-robot construction human-computer interaction method based on mixed reality technology of the present invention, a human-computer interaction system needs to be established, and the related technologies include the following parts:

[0084] 1) Establish a visual and interactive mixed reality program: The program needs to create an interactive mixed reality geometric body in the mixed reality device according to the given geometric body.

[0085] The program needs to implement two-way data transmission between the mixed reality device and the computer, generally using communication methods such as tcp or udp to transmit digital information such as the coordinates and dimensions of the mixed reality geometric body and information such as boolean values representing interactive behaviors to each other.

[0086] The mixed reality device generally includes head-mounted devices such as HoloLens, Meta Quest Pro, Magic Leap, or handheld mobile devices such as mobile phones and tablets.

[0087] The program is generally implemented through webxr exporter, ar foundation, openxr toolkit under the unity platform, or fologram, twinbuild, mindesk plugins under the grasshopper platform.

[0088] 2) Set the logic for the generation and adjustment of the shape of the component to be built: The program needs to determine the geometric generation method of the component, and within a certain range, the output result is continuously adjusted as the input variables change.

[0089] The program is generally completed on a computer and implemented through plugins such as kangaroo, houdini, lunchbox, and weaverbird under the grasshopper platform.

[0090] 3) Establish a robot path simulation program to obtain the robot's motion posture: The program calculates the position and posture of each process of every m joints and the base of the robot according to a series of positions and postures that the robot's end needs to reach.

[0091] The program is generally completed on a computer and implemented through plugins such as furobot, kukaprc, robots, and robot components under the grasshopper platform.

[0092] 4) Establish a program to transfer the robot parameters to the plc: List the angles or position data of each joint and the base of each position that the robot needs to reach, and the program sends the data to the plc in sequence to make the robot move.

[0093] The robot motion parameters transmitted are generally speed, motion type, Cartesian coordinates of the end target point, or the rotation angles of each axis of the robot, the position of the robot base, the state of the end effector, etc.

[0094] In this case, the plc device uses a Siemens S7 control cabinet, the robot uses Kuka's KR20 and KR70 robots, and in addition, a tracked mobile platform and a truss are used to move the robot base.

[0095] The method of realizing the transfer of robot parameters to the plc is generally achieved through communication methods such as profinet, ethernet / ip, and modbus.

[0096] The combined action of technologies 1) and 2) realizes that the adjustment and change of the shape of the built component are driven by the user's interaction behavior.

[0097] The combined action of technologies 1) and 3) realizes that the motion state, posture, base, and end coordinates of the robot can be changed by the user's interaction behavior.

[0098] The combined action of technologies 1) and 4) realizes that the transfer instruction is driven by the user's interaction behavior.

[0099] The combined action of technologies 3) and 4) realizes that the robot executes the set actions after all user interaction behaviors.

[0100] The present invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the multi-robot construction human-computer interaction method based on mixed reality technology are implemented.

[0101] The present invention also provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps of the multi-robot construction human-computer interaction method based on mixed reality technology are implemented.

[0102] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include but not limited to these and any other suitable types of memory.

[0103] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a high-density digital video disc (DVD)), or a semiconductor medium (such as a solid state disc (SSD)), etc.

[0104] In the implementation process, the steps of the above method can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by the hardware processor or executed by the combination of the hardware and software modules in the processor. The software module may be located in a mature storage medium in the art such as a random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0105] It should be noted that the processor in the embodiments of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method embodiments can be completed by the integrated logic circuit in the hardware of the processor or instructions in the form of software. The above-mentioned processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0106] The above has introduced in detail a multi-robot construction human-computer interaction method based on mixed reality technology proposed by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A multi-robot construction human-machine interaction method based on mixed reality technology, characterized in that: The method comprises the following steps: Interactive Design: (1) Aligning mixed reality with the built environment by scanning a marker code or setting a spatial anchor point; (2) Adjusting interactive elements or cursor adjustment components through gestures or controllers; (3) Judging the design results, registering the selected design results through the interactive behavior of the mixed reality button; Robot task allocation: (4) Use different colors to mark multiple robots and their selection cursors to facilitate subsequent task allocation; (5) Use the cursor to select the corresponding component for each robot; (6) Observe potential collision constraints and establish corresponding relationships between robots and components; Robot path planning: (7) Observe the posture of the corresponding robot at each key position in the process in combination with the construction environment through mixed reality; (8) Change the robot posture to avoid collision by adjusting the cursors that control the corresponding robot base and end positions; (9) Preview and confirm the entire movement process of the corresponding robot through mixed reality and then start the robot.

2. The method according to claim 1, characterized in that In the method, a human-machine interaction system based on mixed reality is established to complete the multi-robot collaborative construction task. The working steps of the human-machine interaction system are as follows: 1) Use the structural performance simulation kangaroo and form generation houdini plug-ins of the grasshopper platform to create programs according to user needs, set the shape generation logic and adjustment change logic of the components to be built; 2) Integrate the interactive mixed reality program created by the Unity toolkit OpenXR and the Grasshopper plug-in Fologram with the program in step 1) so that the shape adjustment and change are driven by the user's interactive behavior; 3) Connect the path points involved in the component construction process of step 1) to the kukaprc or robots robot path simulation program, integrate the obtained robot motion posture with the mixed reality program of step 2), so that the virtual robot can be visualized in the construction environment, and the robot motion state, posture, base and end coordinates can be changed by the user's interactive behavior; 4) The robot motion posture parameters to be executed obtained in step 3) are transmitted to the robot PLC via the Profinet or Modbus protocol, and the control of the transmission behavior is connected to the mixed reality program in step 2) so that the transmission instruction is driven by the user's interactive behavior; 5) Program the corresponding robot by writing script code to call the instructions of step 4) passed into the robot PLC, so that the robot executes the actions obtained after the user interaction behaviors in all the above steps.

3. The method according to claim 1, characterized in that The marking codes or anchor points in step (1) need to be pre-set at specific locations in the construction environment, and the relative positions between the marking codes or anchor points must be accurate; The interactive element in step (2) should be the widget itself or a control point created separately.

4. The method according to claim 1, characterized in that The mixed reality button in step (3) should generate a Boolean value change when affected by interactive behavior, which is used to control component design, robot path simulation and operation.

5. The method according to claim 1, characterized in that Marking with different colors in step (4) specifically means that different robots are displayed in completely different colors in the mixed reality, and the interactive elements and assigned components related to each robot have the same color as the corresponding robot.

6. The method according to claim 1, characterized in that The selection method in step (5) is to mark the component closest to the cursor or the component that intersects with the cursor.

7. The method according to claim 1, characterized in that The key positions in step (7) refer to the robot's initial point, final point, pause point during the process, and point where the end effector state changes.

8. The method according to claim 1, characterized in that The posture control cursor in step (8) has an unchanged relative position relationship with the robot base or end coordinates, and the latter changes in the same way as the former changes.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images