Collaborative robot interaction control method and apparatus, storage medium, and electronic device

Abstract

The application relates to a collaborative robot interaction control method and device, a storage medium and electronic equipment. The method comprises the following steps: displaying a virtual robot corresponding to a physical robot in a three-dimensional scene; in response to a user triggering operation on a target joint of the virtual robot, displaying a normal line corresponding to the target joint, and displaying a draggable anchor point on the normal line; in response to a user dragging operation on the anchor point, determining motion control data of the target joint according to a dragging direction and a displacement of the dragging operation, controlling the physical robot to move based on the motion control data, and updating a pose of the virtual robot according to a real-time motion state of the physical robot. The application can intuitively display a robot motion displacement and an instantaneous speed in a graphical manner.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of industrial automation, and in particular to a collaborative robot interaction control method and device, a storage medium and an electronic device. BACKGROUND

[0002] The existing collaborative robot interaction control display method is to display the text of offset value and instantaneous speed in the interface, and control the related joint movement of the robot through several virtual buttons. This interaction method is not intuitive enough, which increases the understanding cost and time cost of the user. SUMMARY

[0003] The present application provides a collaborative robot interaction control method and device, a storage medium and an electronic device, which can intuitively display the robot movement offset and instantaneous speed in a graphical manner.

[0004] According to a first aspect of the present application, a collaborative robot interaction control method is provided, comprising:

[0005] displaying a virtual robot corresponding to a physical robot arm in a three-dimensional scene;

[0006] in response to a user triggering operation on a target joint of the virtual robot, displaying a normal line corresponding to the target joint, and displaying a draggable anchor point on the normal line;

[0007] in response to a user dragging operation on the anchor point, determining movement control data of the target joint according to the dragging direction and displacement of the dragging operation, controlling the movement of the physical robot arm based on the movement control data, and updating the pose of the virtual robot according to the real-time movement state of the physical robot arm.

[0008] Optionally, in response to the user triggering operation on the target joint of the virtual robot, the normal line corresponding to the target joint is displayed, and the draggable anchor point is displayed on the normal line, comprising:

[0009] in response to a user triggering operation on a target joint of the virtual robot, highlighting the rotation axis of the target joint;

[0010] drawing an auxiliary disc according to the rotation axis of the target joint and the normal line of the auxiliary disc, and drawing the draggable anchor point in the normal line;

[0011] displaying the normal line and the anchor point on the normal line.

[0012] Optionally, the movement control data of the target joint is determined according to the dragging direction and the displacement of the dragging operation of the user on the anchor point, the physical robot arm is controlled to move based on the movement control data, and the pose of the virtual robot arm is updated according to the real-time movement state of the physical robot arm, including:

[0013] The dragging direction and the displacement of the dragging operation of the user on the anchor point are obtained in response to the dragging operation of the user on the anchor point.

[0014] The rotation direction is determined according to the dragging direction, the rotation angle and the angular velocity are determined according to the displacement, and the rotation direction, the rotation angle and the angular velocity are taken as the movement control data of the target joint.

[0015] The movement control data is sent to the controller of the physical robot arm, so that the controller controls the physical robot arm to move based on the movement control data.

[0016] The movement state data fed back in real time during the movement of the physical robot arm is received, and the pose of the virtual robot arm is adjusted in real time according to the movement state data.

[0017] Optionally, the normal line has a positive direction and a negative direction, the dragging direction includes dragging towards the positive direction and dragging towards the negative direction, and the displacement is the distance between the starting position and the ending position of the dragging operation.

[0018] Optionally, the rotation direction is determined according to the mapping relationship between the preset dragging direction and the rotation direction, and the dragging direction is converted into the rotation direction.

[0019] The rotation angle and the angular velocity are determined according to the length unit of the displacement, the length unit of the displacement is converted into the velocity unit to obtain the angular velocity, and the rotation angle is calculated according to the preset angle conversion formula, the initial rotation angle and the displacement.

[0020] Optionally, the movement state data fed back in real time during the movement of the physical robot arm is received, and the pose of the virtual robot arm is adjusted in real time according to the movement state data, including:

[0021] The anchor point is drawn from the initial position of the anchor point,

[0022] The movement state data fed back in real time during the movement of the physical robot arm is received, and the movement state data includes the real-time rotation angle.

[0023] According to the rotation direction and the real-time rotation angle, the virtual mechanical arm is rendered, and the real-time rotation angle is converted into a real-time displacement amount, and the slave anchor point is controlled to move along the normal line to the anchor point to a terminal position of the anchor point according to the real-time displacement amount, until the slave anchor point moves to the terminal position, and the rendering of the virtual mechanical arm is stopped.

[0024] Optionally, the method further comprises:

[0025] The auxiliary disc is displayed while the normal line and the anchor point on the normal line are displayed.

[0026] In response to a drag operation of the anchor point by the user, a sector for indicating the rotation angle is drawn and displayed on the auxiliary disc, and the sector is changed in arc according to a real-time motion state of the physical mechanical arm, and the change in the sector arc is synchronized with the adjustment of the virtual mechanical arm pose.

[0027] The second aspect of the present application provides a collaborative robot interaction control device, comprising:

[0028] A display unit is configured to display a virtual mechanical arm corresponding to a physical mechanical arm in a three-dimensional scene.

[0029] A first response unit is configured to display a normal line corresponding to a target joint on the virtual mechanical arm in response to a trigger operation of the target joint by the user, and display a draggable anchor point on the normal line.

[0030] A second response unit is configured to determine motion control data of the target joint according to a drag direction and a displacement amount of a drag operation of the anchor point by the user in response to the drag operation of the anchor point, control the physical mechanical arm to move based on the motion control data, and update a pose of the virtual mechanical arm according to a real-time motion state of the physical mechanical arm.

[0031] The third aspect of the present application provides a computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the steps of the method of the first aspect.

[0032] The fourth aspect of the present application provides an electronic device, comprising:

[0033] A memory storing a computer program;

[0034] A processor configured to execute the computer program in the memory to implement the steps of the method of the first aspect.

[0035] The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

[0036] By presenting a virtual robot arm corresponding to the physical robot arm in a three-dimensional scene, after a user triggers a target joint, a normal line of the target joint and a draggable anchor point are displayed, so that the user can implement a control operation on the physical robot arm by dragging the anchor point, and a controlled result of the physical robot arm is fed back in real time through a virtual robot arm pose adjustment during user control. The scheme combines visual controls with a 3D scene to intuitively display a robot arm control process and a control result, so that a user can more intuitively obtain an offset of a corresponding joint of the robot and a relative offset of this control, facilitating the user to more efficiently understand a current behavior, improve a fault tolerance rate, improve interaction efficiency, and optimize a user experience.

[0037] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings incorporated in and forming a part of the specification illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application, but are not intended to limit the application.

[0039] Figure 1 is an application environment of a collaborative robot interaction control method according to an exemplary embodiment;

[0040] Figure 2 is a flowchart of a collaborative robot interaction control method according to an exemplary embodiment;

[0041] Figure 3 is a flowchart of a terminal responding to a user's triggering operation on a target joint of a virtual robot arm according to an exemplary embodiment;

[0042] Figure 4 is a flowchart of a terminal responding to a user's dragging operation on an anchor point according to an exemplary embodiment;

[0043] Figure 5 is a schematic diagram of a terminal display interface according to an exemplary embodiment;

[0044] Figure 6 is a schematic diagram of an auxiliary disc and a normal line according to an exemplary embodiment.

[0045] Figure 7 is a block diagram of a collaborative robot interaction control device according to an exemplary embodiment;

[0046] Figure 8 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0047] In order for the ordinary person skilled in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings.

[0048] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] Figure 1 An application environment of a collaborative robot interaction control method is shown, including a collaborative robot 101, a control system 103 and a terminal 105. The collaborative robot 101 includes at least one mechanical arm, which can be a multi-joint mechanical arm and / or a six-axis mechanical arm. Each mechanical arm includes a mechanical arm body, a motor and a driver, and the mechanical arm body is provided with a plurality of sensors at the end thereof for acquiring external force information, joint angle, speed information and the like. The control system 103 includes a force tracking controller, an impedance controller, a motion controller and other controllers, and the control system 103 can be integrated with the collaborative robot 101 or can be separately arranged. The terminal 105 can display a virtual mechanical arm and its controls and the like, and can receive input operations of a user, and the user can interact with the terminal 105 through a touch control mode or a mouse and keyboard input mode. The terminal 105 responds to the user's operation on the virtual mechanical arm and its controls, determines the mechanical arm motion control data, and then sends the motion control data to the control system 103, and the control system 103 controls the mechanical arm action of the collaborative robot 101 according to the motion control data, and feeds back the motion result to the terminal 105 in real time. In this process, the terminal 105 is specifically configured to: display a virtual mechanical arm corresponding to a physical mechanical arm in a three-dimensional scene; in response to a user's triggering operation on a target joint of the virtual mechanical arm, display a normal line corresponding to the target joint, and display a draggable anchor point on the normal line; in response to a user's dragging operation on the anchor point, determine motion control data of the target joint according to a dragging direction and a displacement amount of the dragging operation, control the physical mechanical arm to move based on the motion control data, and update a pose of the virtual mechanical arm according to a real-time motion state of the physical mechanical arm.

[0050] The present application provides a collaborative robot interaction control method, which can be executed by the terminal. Please refer to Figure 2 , the collaborative robot interaction control method comprises the steps of:

[0051] S201, display a virtual robot arm corresponding to a real robot arm in a three-dimensional scene.

[0052] S203, in response to a user triggering operation on a target joint of the virtual robot arm, display a normal line corresponding to the target joint, and display anchor points that can be dragged on the normal line.

[0053] S205, in response to a user dragging operation on the anchor points, determine motion control data of the target joint according to a dragging direction and a displacement amount of the dragging operation, control motion of the real robot arm based on the motion control data, and update a pose of the virtual robot arm according to a real-time motion state of the real robot arm.

[0054] In one possible implementation, step S203 can include steps S301-S305.

[0055] S301, in response to a user triggering operation on a target joint of the virtual robot arm, highlight a rotation axis of the target joint.

[0056] The triggering operation can be a clicking operation, and the highlighting manner can include highlighting, darkening, and / or bolding. For example, a user clicks any joint of the virtual robot arm, and the clicked joint is the target joint. Then, the target joint is highlighted, and the rotation axis of the target joint is bolded.

[0057] S303, draw an auxiliary disc according to the rotation axis of the target joint, draw a normal line of the auxiliary disc, and draw the anchor points that can be dragged in the normal line.

[0058] The auxiliary disc is drawn with any point on the rotation axis of the target joint as a center, the normal line is drawn in the center of the auxiliary disc, the normal line has a positive direction and a negative direction, and the anchor points that can be dragged are drawn in the normal line. The anchor points are controls, which provide an input interface for the user. The user drags the anchor points by using a mouse or a touch control manner. The anchor points always move on the normal line, and the motion relationship of the anchor points is mutually exclusive. That is, any anchor point approaching another anchor point triggers all the anchor points to perform a position rotation.

[0059] The normal line corresponds to the auxiliary disc, and the anchor points on the normal line can be mapped to the auxiliary disc. Specifically, the number of anchor points on the normal line can be 2, 4, 8, or 16, and the embodiment does not limit the number of anchor points. The auxiliary disc is distributed with virtual anchor points consistent with the number of anchor points on the normal line, and the virtual anchor points can be evenly distributed on the auxiliary disc. For example, the auxiliary disc can be evenly cut according to the number of virtual anchor points, and each cutting point is provided with a virtual anchor point. The cutting rule is linear, y=x, and x is evenly distributed from -π to +π. The normal line can also be cut, and each cutting point is provided with an anchor point, but the anchor points on the normal line are not evenly distributed. The cutting rule of the normal line is y=tan(x), and x is evenly distributed from -π to +π, where y is the number of anchor points and x is the number of virtual anchor points. In this way, the anchor points on the normal line form a corresponding relationship with the virtual anchor points on the auxiliary disc. When the anchor points on the normal line are dragged, the displacement of the anchor points can be reflected in the rotation of the corresponding virtual anchor points on the auxiliary disc, so that the displacement can be converted into a rotation angle.

[0060] As shown in FIG. 8, Figure 6 As shown in FIG. 8, four anchor points are provided on the normal line. The first anchor point A0 is located at the position where the normal line and the center of the auxiliary disc coincide. The second anchor point A1 is located at a position offset by one unit from A0 in the positive direction of the normal line. The third anchor point A2 is located at a position offset by one unit from A0 in the negative direction of the normal line. The fourth anchor point A3 is located at a positive or negative infinity position of the normal line. Since any anchor point on the normal line is exclusive with its adjacent anchor point, when any one anchor point is dragged in the positive or negative direction of the normal line, the positions of A1, A0, A2, and A3 will be rotated once. Assuming that A0 is dragged and moves closer to A1, A0 will move to the original position of A1, A2 will move to the original position of A0, A3 will move to the original position of A2, and A1 will move to the original position of A3.

[0061] S305, display the normal line and the anchor points on the normal line.

[0062] The auxiliary disc and the virtual anchor points drawn by the terminal are used to determine the motion control data, and these contents can not be displayed when presented to the user. Only the normal line and the anchor points on the normal line related to subsequent user operations are displayed.

[0063] In one possible implementation, step S205 can include steps S401-S407.

[0064] S401, in response to the user's dragging operation on the anchor point, obtaining the dragging direction and displacement of the dragging operation.

[0065] The normal has a positive direction and a negative direction, the dragging direction includes dragging towards the positive direction and dragging towards the negative direction; the displacement is the distance between the start position and the end position of the dragging operation. Assuming that the initial position coordinates of the dragged anchor point are (x0, y0, z0) and the coordinates of the end position are (x1, y1, z1), the displacement of the anchor point is

[0066] The determining the rotation direction according to the dragging direction includes: converting the dragging direction into a rotation direction according to a preset mapping relationship between the dragging direction and the rotation direction. For example, the corresponding rotation direction when dragging towards the positive direction can be the clockwise direction, and the corresponding rotation direction when dragging towards the negative direction can be the counterclockwise direction.

[0067] S403, determining a rotation direction according to the dragging direction, determining a rotation angle and an angular velocity according to the displacement, and taking the rotation direction, the rotation angle and the angular velocity as the motion control data of the target joint.

[0068] The determining the rotation angle and the angular velocity according to the displacement includes: converting the length unit of the displacement into the velocity unit to obtain the angular velocity. It is worth noting that during the dragging of the anchor point, it is difficult for the user to achieve uniform speed through manual operation, so the angular velocity is not likely to be a constant value, but changes with the speed of the user dragging the anchor point.

[0069] The rotation angle is calculated according to a preset angle conversion formula, an initial rotation angle and the displacement. Assuming that the initial rotation of the target joint is θ (radian) and the target rotation angle of the target joint is φ, there is φ = atan(tanθ + L), and φ is the required rotation angle of the target joint.

[0070] S405, sending the motion control data to a controller of the physical robot arm, so that the controller controls the physical robot arm to move based on the motion control data.

[0071] If the angular velocity in the motion control data is v, the rotation direction is clockwise, and the rotation angle is φ, then controlling the physical robot arm to move according to the motion control data includes: controlling the physical robot arm to rotate at an angle of φ in the clockwise direction at an angular velocity of v.

[0072] S407, receiving motion state data fed back in real time during the movement of the physical robot arm, and adjusting the pose of the virtual robot arm in real time according to the motion state data.

[0073] The step S407 specifically comprises: drawing a slave anchor point at the initial position of the anchor point; receiving motion state data fed back in real time during the motion of the physical robot arm, the motion state data comprising a real-time rotation angle; rendering the virtual robot arm according to the rotation direction and the real-time rotation angle, converting the real-time rotation angle into a real-time displacement amount, and moving the slave anchor point along the normal line to the anchor point to a final position of the anchor point according to the real-time displacement amount until the slave anchor point moves to the final position, and stopping rendering the virtual robot arm. Thus, the anchor point can feed back the input state of the user, and the slave anchor point can feed back the actual state of the robot arm.

[0074] In a possible implementation, the above-mentioned collaborative robot interaction control method further comprises: displaying the auxiliary disc while displaying the normal line and the anchor points on the normal line; in response to a drag operation of the user on the anchor points, drawing and displaying a sector on the auxiliary disc for indicating the rotation angle, and changing the sector radian according to the real-time motion state of the physical robot arm, the sector radian change being synchronous with the adjustment of the virtual robot arm pose.

[0075] Similarly to the angular velocity, the sector changes with the change of the speed at which the user drags the anchor points. In order to display more intuitively, the color of one side of the sector can express the angular velocity of the joint using the red-green-blue color spectrum. The faster the rotation, the greater the angular velocity, and the redder the color.

[0076] In actual implementation, the terminal can provide the user with an option of whether to perform sector display, so that the user can select and display as needed.

[0077] Figure 5 An example display screen of the terminal is shown in FIG. 4. Figure 5 The normal line and the auxiliary disc of the target joint of the virtual robot arm are highlighted, there are four anchor points a, b, c and d on the normal line, the anchor point d is invisible in the view at infinity, and the right-hand rule is adopted, that is, the direction indicated by the thumb when the hand holds the normal line is the positive direction. The user drags the anchor point a to the anchor point b, the virtual robot arm adjusts the position in real time, and the auxiliary disc simultaneously represents the rotation angle of the physical robot arm in real time in the form of a sector.

[0078] When the user operates, in order to maintain the motion of the physical robot arm, the user needs to keep the finger pressing the screen (or the mouse drag is not released), the terminal periodically sends an instruction to the controller, the controller discards the previous instruction after receiving the instruction, and controls the robot arm to execute the current instruction. When the user's finger is separated from the screen (or the mouse is released), the terminal sends a stop command to the controller, the operation is terminated immediately, and the physical robot arm and the virtual robot arm stop moving and adjusting the pose.

[0079] The present disclosure presents a virtual robot arm corresponding to a physical robot arm in a three-dimensional scene, displays a normal line of a target joint and a draggable anchor point after a user triggers the target joint, so that the user implements a control operation on the physical robot arm by dragging the anchor point, and feeds back a controlled result of the physical robot arm in real time through a virtual robot arm pose adjustment during user control. The present scheme combines visual control with a 3D scene, intuitively displays a robot control process and a control result, and enables a user to more intuitively obtain an offset of a corresponding joint of the robot and a relative offset of this control, so as to facilitate the user to more efficiently understand a current behavior, improve a fault tolerance, improve interaction efficiency, and optimize a user experience.

[0080] Figure 7 is a block diagram of a collaborative robot interaction control device according to an exemplary embodiment.

[0081] Please refer to Figure 7 The robot motion control device comprises:

[0082] The display unit 710 is configured to display a virtual robot arm corresponding to a physical robot arm in a three-dimensional scene.

[0083] The first response unit 720 is configured to, in response to a triggering operation of a target joint on the virtual robot arm by a user, display a normal line corresponding to the target joint, and display a draggable anchor point on the normal line.

[0084] The second response unit 730 is configured to, in response to a dragging operation of the anchor point by the user, determine motion control data of the target joint according to a dragging direction and a displacement of the dragging operation, control motion of the physical robot arm based on the motion control data, and update a pose of the virtual robot arm according to a real-time motion state of the physical robot arm.

[0085] In a feasible implementation, the first response unit 720 is further configured to, in response to the triggering operation of the target joint on the virtual robot arm by the user, highlight a rotation axis of the target joint, draw an auxiliary disc according to the rotation axis of the target joint and a normal line of the auxiliary disc, and draw the draggable anchor point in the normal line, and display the normal line and the anchor point on the normal line.

[0086] In a feasible implementation, the second response unit 730 is further configured to: in response to the user's drag operation on the anchor point, acquire a drag direction and a displacement of the drag operation; determine a rotation direction according to the drag direction, and determine a rotation angle and an angular velocity according to the displacement; take the rotation direction, the rotation angle and the angular velocity as the motion control data of the target joint; send the motion control data to a controller of the physical robot arm, so that the controller controls the physical robot arm to move based on the motion control data; and receive motion state data fed back in real time during the movement of the physical robot arm, and adjust the pose of the virtual robot arm in real time according to the motion state data.

[0087] As to the apparatus in the above-mentioned embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0088] In the example embodiments, the present application further provides a computer readable storage medium storing a computer program, for example, a memory storing a computer program, which can be executed by a processor to complete the collaborative robot interaction control method. Optionally, the storage medium can be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0089] In the example embodiments, the present application further provides an electronic device including a memory and a processor, the memory storing a computer program; and the processor is configured to execute the computer program in the memory to implement the steps of the above-mentioned collaborative robot interaction control method.

[0090] Please refer to Figure 8 In some embodiments, the electronic device 800 can include a processor 810, a memory 820, an input / output component 830, and a communication port 840. The processor (e.g., CPU) 810 can execute program instructions in the form of one or more processors. The memory 820 includes different forms of program memory and data memory, such as a hard disk, a read-only memory (ROM), a random access memory (RAM), etc., for storing various data files processed and / or transmitted by the computer. The input / output component 830 can be used to support input / output between the processing device and other components. The communication port 840 can be connected with a network for data communication. The example processing device can include program instructions stored in a read-only memory (ROM), a random access memory (RAM), and / or other types of non-transitory storage media, which are executed by the processor 810. The method and / or process of the embodiments of the present specification can be implemented in the form of program instructions.

[0091] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0092] It is to be understood that the application is not limited to the precise construction hereinafter described and as shown in the attached drawings, and that various changes in form and detail can be made therein without departing from the scope thereof. The scope of the application should only be limited by the appended claims.

Claims

1. A method of collaborative robot interaction control, the method comprising: The method comprises: displaying a virtual robot arm corresponding to a physical robot arm in a three-dimensional scene; in response to a user triggering a target joint on the virtual robot arm, displaying a normal line corresponding to the target joint, and displaying a draggable anchor point on the normal line; in response to the user dragging the anchor point, determining motion control data of the target joint according to a dragging direction and a displacement amount of the dragging operation, controlling the physical robot arm to move based on the motion control data, and updating a pose of the virtual robot arm according to a real-time motion state of the physical robot arm; the response to the user triggering the target joint on the virtual robot arm, displaying the normal line corresponding to the target joint, and displaying the draggable anchor point on the normal line comprises: in response to the user triggering the target joint on the virtual robot arm, highlighting a rotation axis of the target joint; drawing an auxiliary disc according to the rotation axis of the target joint and a normal line of the auxiliary disc, and drawing the draggable anchor point in the normal line; displaying the normal line and the anchor point on the normal line; the response to the user dragging the anchor point, determining the motion control data of the target joint according to a dragging direction and a displacement amount of the dragging operation, controlling the physical robot arm to move based on the motion control data, and updating the pose of the virtual robot arm according to the real-time motion state of the physical robot arm comprises: in response to the user dragging the anchor point, obtaining the dragging direction and the displacement amount of the dragging operation; determining a rotation direction according to the dragging direction, determining a rotation angle and an angular velocity according to the displacement amount, and taking the rotation direction, the rotation angle, and the angular velocity as the motion control data of the target joint; sending the motion control data to a controller of the physical robot arm, so that the controller controls the physical robot arm to move based on the motion control data; receiving motion state data fed back in real time during the movement of the physical robot arm, and adjusting the pose of the virtual robot arm in real time according to the motion state data; the receiving of the motion state data fed back in real time during the movement of the physical robot arm, and the adjusting of the pose of the virtual robot arm in real time according to the motion state data comprises: drawing a from-anchor point at an initial position of the anchor point; receiving motion state data fed back in real time during the movement of the physical robot arm, the motion state data comprising a real-time rotation angle; according to the rotation direction and the real-time rotation angle, rendering the virtual robot arm, converting the real-time rotation angle into a real-time displacement amount, and controlling the from-anchor point to move along the normal line from the initial position of the anchor point to a terminal position of the anchor point according to the real-time displacement amount, until the from-anchor point moves to the terminal position of the anchor point, and stopping rendering the virtual robot arm; the method further comprises: displaying the auxiliary disc while displaying the normal line and the anchor point on the normal line. In response to the user's dragging operation on the anchor point, a sector for indicating the rotation angle is drawn and displayed on the auxiliary disc, and the sector's arc is changed according to the real-time motion state of the physical robot arm, the change of the sector's arc being synchronized with the adjustment of the virtual robot arm pose.

2. The method of claim 1, wherein, The normal has a positive direction and a negative direction, the dragging direction includes dragging toward the positive direction and dragging toward the negative direction, and the displacement is a distance between a start position and an end position of the dragging operation.

3. The method of claim 2, wherein, The determining of the rotation direction according to the dragging direction includes: converting the dragging direction into a rotation direction according to a preset mapping relationship between the dragging direction and the rotation direction. The determining of the rotation angle and the angular velocity according to the displacement includes: converting a length unit of the displacement into a velocity unit to obtain the angular velocity, and calculating the rotation angle according to a preset angle conversion formula, an initial rotation angle and the displacement.

4. A collaborative robot interaction control device, characterized by, A control method for implementing any one of claims 1-3, comprising: a display unit configured to display a virtual robot arm corresponding to a physical robot arm in a three-dimensional scene; a first response unit configured to, in response to a user's triggering operation on a target joint of the virtual robot arm, display a normal corresponding to the target joint, and display an anchor point that is draggable on the normal; a second response unit configured to, in response to a user's dragging operation on the anchor point, determine motion control data of the target joint according to a dragging direction and a displacement of the dragging operation, control the physical robot arm to move based on the motion control data, and update a pose of the virtual robot arm according to a real-time motion state of the physical robot arm.

5. A computer readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the method of any one of claims 1-3.

6. An electronic device, comprising: comprising: a memory storing a computer program; a processor configured to execute the computer program in the memory to implement the steps of the method of any one of claims 1-3.

Images