Multi-person Cooperative Interaction Method and System for Mobile Robots Based on Magnetic Force Feedback

Abstract

The present invention relates to the technical field of robot collaborative interaction, and discloses a method and system for multi-person collaborative interaction of a mobile robot based on magnetic force feedback. The method includes: setting operation parameters and selecting an operation mode; receiving the hand movement information of an operator through an interaction device as the operation information of the robot; the interaction device can follow the movement of the operator's hand; changing the feedback force transmitted by the interaction device to the operator's hand according to a force compensation algorithm; generating a motion instruction based on the operation information of a single operator or generating a motion instruction based on the operation information of multiple operators, performing predictive simulation on the motion instruction, and sending the motion instruction verified through predictive simulation to the mobile robot for execution. The present invention uses a mobile robot as the carrier of a magnetic field device to perform real-time tracking of the position of the operator's hand, and combines a mixed reality device to unify the user operation space and the feedback space, solving the defect of the separation of the operation space and the feedback space.

Description

Technical Field

[0001] The present invention relates to the technical field of robot collaborative interaction, and particularly relates to a multi-person collaborative interaction method and system for a mobile robot based on magnetic force feedback. Background Art

[0002] According to whether contact is generated during interaction, human-robot interaction methods can generally be divided into contact methods and non-contact methods. The interaction device provides an interaction channel between the user and the robot, which usually includes control buttons, joysticks, touchscreens, keyboards, etc. The contact interaction method is widely used due to its advantages such as simple structure, easy control, and low latency. However, due to the limitation of the movement range of the device, the direct contact operation method has problems such as limited operation range, high learning cost for operators, and unnatural interaction methods.

[0003] Compared with the contact interaction method, the non-contact interaction method minimizes the restrictions on the operator and increases the naturalness and experience of the interaction. Non-contact interaction methods based on gestures, voices, etc. can maximize the naturalness of the operation and the experience of the operator. However, most of these methods focus more on the control of the robot by the operator and ignore the feedback generated by the robot during the interaction process, ultimately resulting in the separation of operation and feedback in the human-robot interaction process and affecting the operation accuracy. Especially for the operation of mobile robots,

[0004] At present, most of the feedback methods for human-robot interaction rely heavily on visual feedback and lack feedback methods for other channels. And single visual feedback is likely to cause a high cognitive load on the operator. Long-term operation is likely to cause the operator's attention to be distracted, resulting in inaccurate operation. As a form of tactile feedback, force feedback plays a very important role in human-robot interaction, which can enrich the feedback information and interaction channels of the robot and improve the operation efficiency and operation accuracy. During the contact interaction process, feedback force is mainly generated through joysticks and wearable devices, but the friction, mechanical hysteresis, and other non-linear noise interferences existing in its mechanical structure greatly affect the presence of the interaction and reduce the interaction efficiency to a certain extent. Therefore, non-contact force feedback has become a popular choice for teleoperation force feedback. Common non-contact force feedback devices use air pressure and air jet methods to generate non-contact impedance force, thereby realizing non-contact force tactile perception. At the same time, among various non-contact force feedback devices, magnetic force has become the main research direction of non-contact force feedback interaction devices due to its advantages of strong stability, smooth and stable force field, and the ability to generate strong acting force within a short distance. However, the force field range provided by a single magnetic force device is limited and cannot meet the operation requirements of large-range interaction of mobile robots. Therefore, how to increase the interaction range of the magnetic force device has become a popular research issue for magnetic force-based force feedback devices.

[0005] During the human-robot interaction process, to improve the operation efficiency, the operation process may face the situation where multiple people operate the same robot simultaneously. Regarding the problems of interaction right allocation and multi-person information fusion during the multi-person collaborative operation process, it is necessary to perform fusion calculation on the multi-person operation information and map it to the operation behavior of the same robot on the basis of ensuring data security and operation transparency. The problems of mutual exclusion of interaction data and interaction right allocation during the multi-person collaborative interaction process are the main difficulties in the multi-person information fusion operation.

[0006] The existing invention patent with the publication number CN117301059A, "A Mobile Robot Teleoperation System, Method and Storage Medium", discloses a mobile robot teleoperation system, method and storage medium in the technical field of robot teleoperation, aiming to improve the control efficiency of the mobile robot and the user experience. The system includes: a mobile host, a somatosensory controller and a mobile robot that are signal-connected to the mobile host; the somatosensory controller is used to obtain gesture data within its own recognition range and transmit it to the mobile host; the mobile host is used to: determine the movement instruction corresponding to the received gesture data according to the received gesture data and the pre-determined mapping relationship between the gesture data and the movement mode of the mobile robot, and transmit it to the mobile robot; the mobile robot is configured to move according to the received movement instruction.

[0007] The existing invention patent with the publication number CN117707171A, "A Mobile Robot Control Method Based on Mixed Reality", relates to the technical field of human-computer interaction. Specifically, it is a mobile robot control method based on mixed reality. The mobile robot is provided with a path interaction system, a radar-vision integrated ranging unit, a conversion control unit and a movement control unit. The path interaction system is used to generate path information and is connected to the movement control unit to control the movement behavior of the mobile robot. The movement of the mobile robot is restricted. The radar-vision integrated ranging unit is connected to the movement control unit through the conversion control unit to control the movement of the mobile robot. The radar-vision integrated ranging unit is used to obtain the distance between the mobile robot and the obstacle. When the mobile robot is blocked during the control with the preset path data, the conversion control unit can be used to make the mobile robot switch to the movement control based on the distance data between the mobile robot and the obstacle.

[0008] The existing invention patent "A Robot Teleoperation Method Combining Virtual-Reality Fusion with Force Feedback Sensing" with the publication number CN117325147A discloses a robot teleoperation method combining virtual-reality fusion with force feedback sensing. The robot teleoperation method specifically includes the following operation steps: S1. One industrial control computer is equipped at each end. The computers communicate with each other through the TCP / IP network protocol and use the WiFi network to achieve communication. The master station operation terminal computer is connected to the force feedback device Touch and the head-mounted display, integrating a teleoperation control interface, and can send task instructions to the server through button control; the teleoperation method combining mixed reality with force feedback can improve the operation efficiency on the basis of ensuring the safety of robot operations and is regarded as the future development direction of teleoperation robots. This invention proposes an innovation in terms of the technical route. Based on the master-slave arm operation system and combined with mixed reality technology, a teleoperation robot system implementation plan based on mixed reality + force feedback is proposed.

[0009] In summary, aiming at the characteristics of large operation range, high operation accuracy requirements, and stable force feedback during the multi-person collaborative interaction process of mobile robots, in order to ensure system transparency and operation safety, the present invention combines mixed reality technology and force feedback technology to construct a set of multi-person collaborative interaction method and system for mobile robots based on magnetic force feedback. Summary of the Invention

[0010] To solve the above technical problems, the present invention provides a multi-person collaborative interaction method and system for mobile robots based on magnetic force feedback. In order to expand the force feedback space and generate multi-dimensional feedback forces, the present invention proposes to use a mobile robot as the carrier of the magnetic field device to perform real-time tracking of the operator's hand position, and combine a mixed reality device to unify the user operation space and the feedback space, so as to solve the defect of the separation of the operation space and the feedback space existing in the traditional interaction method.

[0011] To solve the above technical problems, the present invention adopts the following technical solutions:

[0012] A multi-person collaborative interaction method for mobile robots based on magnetic force feedback includes the following steps:

[0013] Set operation parameters and select an operation mode; the operation parameters include the operation weight of the operator; the operation modes include: single-person operating a single robot, information fusion operation of multi-person collaboration, and multi-person cooperating to operate multiple robots;

[0014] Receive the hand movement information of the operator through the interaction device as the operation information of the robot; the interaction device can move following the movement of the operator's hand; according to the force compensation algorithm, change the feedback force transmitted by the interaction device to the operator's hand;

[0015] A motion instruction is generated according to the operation information of a single operator or a motion instruction is generated according to the operation information of multiple operators, a prediction simulation is performed on the motion instruction, and the motion instruction verified by the prediction simulation is sent to the mobile robot for execution.

[0016] Furthermore, when the operation mode selects the multi-person collaborative information fusion operation, the prediction simulation of the motion command specifically includes:

[0017] Obtain information on the nature of the task, the task objectives, and the execution process, collect information on the task area environment, and establish a model related to the work environment;

[0018] Using the Hololens 2 client, a mixed reality environment is created based on the mission nature information, mission target information, execution process information, and mission area environment information;

[0019] Build a computer server and Hololens 2 client, establish communication between multiple clients, and share the mixed reality operating environment;

[0020] The motion instructions generated according to the operation information and operation weights of multiple operators are mapped to the virtual robot in the prediction simulation software; if the virtual robot collides with the surrounding environment, the motion instructions are corrected to obtain the final motion instructions.

[0021] Furthermore, the generating of motion instructions according to the operation information of multiple operators specifically includes:

[0022] S21. Setting an operation weight for each operator;

[0023] S22, processing the acquired operation data of each operator to eliminate unreasonable operation data;

[0024] S23. Perform weighted calculation on the operation data of each operator according to the operation weight of the operator to obtain a motion instruction.

[0025] Furthermore, the interaction device can follow the movement of the operator's hand, specifically including:

[0026] S41, receiving hand movement information;

[0027] S42, allocating the movement distances of the trolley and the robotic arm by setting the allocation ratio and the movement limit range;

[0028] S43, transmitting the allocated data to the robot arm of the interactive device, performing inverse solution of the robot arm, and sending the motion data of the car to the industrial computer of the car;

[0029] S44. Control the movement of the trolley through the trolley movement chassis control algorithm, and at the same time control the movement of the robotic arm, and based on the hand movement information, make the interaction device move following the operator when the operator moves.

[0030] Further, changing the feedback force transmitted by the interaction device to the operator's hand according to the force compensation algorithm specifically includes: The force compensation algorithm compensates the feedback force by perceiving the magnitude of the magnetic force and performing position offset on the basis of the inverse kinematics of the robotic arm.

[0031] A multi-person collaborative interaction system for a mobile robot based on magnetic force feedback, comprising:

[0032] Parameter setting module: used to set operation parameters and select operation modes; the operation parameters include the operation weights of the operators; the operation modes include: single operator operating a single robot, information fusion operation of multiple people collaborating, and multiple people cooperating to operate multiple robots;

[0033] Interaction module: Receive the hand movement information of the operator through the interaction device as the operation information of the robot; the interaction device can move following the operator's hand; change the feedback force transmitted by the interaction device to the operator's hand according to the force compensation algorithm;

[0034] Control module: Generate a motion instruction according to the operation information of a single operator or generate a motion instruction according to the operation information of multiple operators, perform predictive simulation on the motion instruction, and send the motion instruction verified by the predictive simulation to the mobile robot for execution.

[0035] Compared with the prior art, the beneficial technical effects of the present invention are:

[0036] 1. The present invention designs an interaction control method for a mobile robot, enabling the operator to effectively control and cooperate in operating the mobile robot. In the face of complex operations and unexpected situations, the teleoperation system can ensure that the robot accurately executes the operator's instructions and provides real-time feedback so that the operator can understand the state and environment of the robot. The process of integrating multiple people's operations can, to a certain extent, avoid the misoperations of individual operators and ensure the safety of the operating system.

[0037] 2. In the human-computer interaction of the teleoperation system of the present invention, the mixed reality (MR) technology is used. The mixed reality (MR) technology is a new type of interaction method platform that perceives human natural behaviors through multiple modalities, obtains multiple interaction channels such as human speech, gestures, and eye movements, and enables them to be selectively parallel and cooperative to achieve multi-channel, high-bandwidth, efficient, and location-independent human-computer interaction.

[0038] 3. The present invention designs a dedicated teleoperation interaction device and control method for a mobile robot for multi-person collaborative operation, and performs complex teleoperation tasks by perceiving the interaction mode of human natural behavior through the interaction device. The present invention designs a combination of gesture and voice interaction to control the end of the robotic arm of the mobile robot, which can achieve a more natural and user-friendly interaction mode.

[0039] 4. In view of the situation that a single-person teleoperation robot may fail to respond to emergencies and misoperations, the present invention designs an interaction system for a multi-person collaborative mobile robot to complete complex tasks. The present invention creates a virtual and real integrated environment to provide a richer collaborative experience, enables real-time information sharing, and can observe the simulation state and actual state of the mobile robot, environmental information, and real-time information on task progress in a virtual reality environment. The present invention enables multiple operators to jointly operate the same mobile robot, collaborate to execute tasks, and avoid collisions and conflicts. And based on the 5G communication protocol, multiple people can conduct real-time voice and video communication in the MR environment, discuss task progress, and adjust strategies.

[0040] 5. Based on the MR platform, the present invention designs a combination of gesture and voice interaction to control the end of the robotic arm of the explosive disposal robot, which can achieve a more natural and user-friendly interaction mode. It designs an interaction device and system for a multi-person collaborative mobile robot based on magnetic force feedback, which is mainly divided into three modes: single-person single-machine, information fusion operation of multi-person collaboration, and multi-person cooperation in operating multiple robots. It provides a large-range and contactless teleoperation interaction mode through the mobile robot multi-person collaborative interaction method and system based on magnetic force feedback.

[0041] 6. The present invention solves the technical problems existing in the prior art, such as overly redundant interaction equipment, complex operation process leading to dependence on manual operation experience, and poor interaction control effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the multi-person collaborative interaction method for a mobile robot based on magnetic force feedback in an embodiment of the present invention;

[0043] Figure 2 It is a schematic diagram of the process of controlling a mobile robot under a multi-person collaborative framework in an embodiment of the present invention;

[0044] Figure 3 It is a schematic diagram of data flow interaction processing in a multi-person collaborative mode in an embodiment of the present invention;

[0045] Figure 4 It is a schematic diagram of the magnetic force-based interaction device in an embodiment of the present invention;

[0046] Figure 5Schematic diagram of data stream interaction processing for multi-person collaborative information fusion operation in an embodiment of the present invention;

[0047] Figure 6 Schematic diagram of the process of the robotic arm collaboration algorithm adopted in an embodiment of the present invention;

[0048] Figure 7 Schematic diagram of data interaction processing of an interactive system for a multi-person collaborative mobile robot based on magnetic force feedback in an embodiment of the present invention;

[0049] Figure 8 Schematic diagram of the structure of an interactive device in an embodiment of the present invention.

[0050] In the figure: 1. Robotic arm control device; 11. Robotic arm controller; 12. Robotic arm; 13. Magnetic induction device; 2. Cart; 21. Cart body; 22. Cart positioning device; 23. Cart wheel hub; 3. Base. Specific embodiments

[0051] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0052] As Figure 1 shown, a multi-person collaborative interaction method for a mobile robot based on magnetic force feedback of the present invention includes:

[0053] S1. Use a preset software operation terminal device to run the preset software to obtain single-person operation information;

[0054] S2. Set operation parameters, set parameters such as the operation weight and command issuance frequency of the operator, select the number of mobile robots for collaborative control, and analyze the information fusion operation mode of multi-person collaboration and the mode of multi-person cooperation in operating multiple robots;

[0055] S3. Obtain the operation data of the operator, perform predictive simulation on the collaborative operation result, ensure the safety of the robot operation, avoid collisions, and issue the motion command after predictive simulation;

[0056] S4. The interactive device receives hand movement information, moves following the operator, and at the same time, based on the force compensation algorithm on the basis of receiving the information, controls the offset of the magnetic force feedback device, and changes the magnitude of the feedback force by changing the relative distance between the magnets;

[0057] S5. The mobile robot receives the motion command and executes relevant actions.

[0058] The present invention designs a magnetic force feedback interaction control method for multi-person cooperation of mobile robots, enabling multiple operators to effectively control the same mobile robot. When encountering complex operations and unexpected situations during task execution, the multi-person cooperative teleoperation system can comprehensively execute the instructions issued by the operators, ensure the safety of the robot's execution actions, and provide real-time visual and force feedback so that the operators can understand the state of the robot and the environment.

[0059] As Figure 3 shown, in the human-computer interaction of the teleoperation system of the present invention, mixed reality (MR) is a new type of interaction method platform that recognizes human natural behaviors through multi-modal perception technology, obtains multiple interaction channels such as voice, gesture, and gaze, and enables them to selectively achieve multi-channel, high-bandwidth, efficient, and location-independent human-computer interaction behaviors through parallelism and cooperation.

[0060] As Figure 2 shown, in a more specific technical solution, step S2 specifically includes:

[0061] S21. Obtain task nature information, task objective information, and execution process information, collect task area environment information, and establish a model related to the working environment;

[0062] S22. Use the Hololens 2 development tool to create a mixed reality environment according to the task nature information, task objective information, execution process information, and task area environment information;

[0063] S23. Set up a computer server side and a Hololens 2 client side, establish communication among multiple clients, and share the mixed reality operation environment;

[0064] S24. Send the mobile robot guidance commands operated by the Hololens 2 client to the prediction simulation software. Among them, the types of mobile robot guidance commands mainly include: gesture commands and voice commands:

[0065] S25. At the prediction simulation end, receive the motion commands of multiple people and comprehensively calculate the corresponding motion results according to the set motion weights of each operator;

[0066] S26. After obtaining the calculation results of multi-person cooperative operation, map the calculation results to the virtual robotic arm in the prediction simulation software as the final control result to control the robotic arm to perform motion simulation;

[0067] S27. Use the prediction simulation results to correct the motion of the robotic arm in HoloLens2 and send the correction results to HoloLens2 to correct the motion of the robotic arm in HoloLens2 so that it is the same as the motion of the robotic arm in the prediction simulation software.

[0068] In a more specific technical solution, in step S3, in the information fusion operation mode of multi-person collaboration, the obtaining of the operation data of the operator specifically includes:

[0069] S31. Obtain the operation information of multi-person collaboration, and set an operation weight for each operator;

[0070] S32. On the operation terminal, obtain the operation data of the operator;

[0071] S33. Perform data processing according to the operation weight and operation data, eliminate unreasonable operation data, and perform weighted calculation on the screened data;

[0072] S34. Dynamically update the operation data of the operator and the weighted operation data.

[0073] For the interactive processing of the information fusion operation data flow of multi-person collaboration, see Figure 5 .

[0074] The present invention designs a dedicated teleoperation interaction device and control method for a mobile robot, and executes complex tasks and provides force feedback through the interaction device to sense the natural behavior interaction method of humans. The present invention designs a method for controlling the end of the robotic arm of a mobile robot for mobile operation, and can realize a teleoperation interaction process with a larger interaction range and a more natural interaction method.

[0075] The interaction device includes a holographic glasses module, a virtual robot module, a real robot, and a feedback unit module. Among them, the holographic glasses module is mainly used for positioning the position of the operator's hand, identifying the operator's gestures, and controlling the interaction of the virtual scene and the rendering of related scenes; the virtual robot module is mainly used for calculating the kinematics of the robot in the virtual scene and controlling the position and posture of the virtual robot; the real robot control module realizes force-position hybrid control and the control of the robotic arm and force sensors; the feedback unit is used for calculating and solving parameters related to magnetic force.

[0076] As Figure 8 shown, the interaction device in this embodiment includes: a robotic arm control device 1, a trolley 2, and a base 3. The robotic arm control device 1 includes: a robotic arm controller 11, a robotic arm 12, and a magnetic induction device 13; the trolley 2 includes: a trolley body 21, a trolley positioning device 22, and trolley wheels 23.

[0077] As Figure 4 shown, in a more specific technical solution, in step S4, the motion control process of the interaction device includes:

[0078] S41. Receive the comprehensive hand motion information, where the hand motion information is derived from the processed weighted data;

[0079] S42. Allocate the movement distances of the trolley and the robotic arm by setting the allocation ratio and the movement limit range.

[0080] S43. Use the vehicle-arm cooperation algorithm for operation position following, and use the trolley movement chassis control algorithm for trolley movement control.

[0081] S44. Obtain the end pose information, and the robotic arm performs inverse kinematics.

[0082] S45. The robotic arm performs force compensation offset.

[0083] In a more specific technical solution, in step S43, the steps of using the trolley movement chassis control algorithm for trolley movement control further include: receiving the current position and the target position, planning the route through the position, identifying obstacles and automatically avoiding obstacles.

[0084] For the specific process of the vehicle-arm cooperation algorithm, see Figure 6 .

[0085] In a more specific technical solution, in step S44, the force compensation algorithm of the robotic arm is to perform position offset by sensing the magnetic force magnitude on the basis of the inverse kinematics of the robotic arm so as to compensate the feedback force: the steps of using the robotic arm force compensation algorithm to control the magnetic force feedback in the interaction process further include: anti-collision force compensation offset, force sense feedback offset.

[0086] Based on the MR platform, the present invention designs a method combining gesture and voice interaction to control the end of the robotic arm of a mobile robot, which can achieve a more natural and user-friendly interaction method. An interaction system for multi-person collaborative mobile robots is designed, which is mainly divided into three modes: single-person operation of a single robot, information fusion operation of multi-person collaboration, and multi-person cooperation in operating multiple robots. Through multiple interaction modes, it is possible to cope with the complex situations of bomb disposal tasks and dangerous environment rescue.

[0087] The system of the present invention corresponds to the method, and the technical solutions applicable to the method are equally applicable to the system. For the data interaction of the multi-person collaborative interaction system of the mobile robot based on magnetic force feedback in the present invention, see Figure 7 .

[0088] The magnetic force component in the present invention can be static magnetic force or electromagnetic force; the trolley can be a differential trolley or other devices with a movable chassis.

[0089] In summary, the present invention provides a multi-person collaborative interaction method and system for a mobile robot based on magnetic force feedback. The method includes: turning on and running the pre-set software using an MR device and a computer; selecting a mode among the modes of single-person operating a single robot, multi-person collaborative information fusion operation, and multi-person cooperating to operate multiple robots; generating a motion instruction to control the interactive device to perform hand tracking motion and the motion of the mobile robot; performing predictive simulation calculation on the motion instruction of the mobile robot in the motion simulation module; finally sending the verified control instruction to control the mobile robot; turning off the device and ending the task. The present invention solves the problems of large operation range and low operation transparency in the remote operation process of the mobile robot, invents a large-range interactive force feedback device based on magnetic force feedback, combines the MR technology to perform overall perception and information reproduction on the remote operation process, and at the same time interacts with the slave robot more naturally through gesture and voice interaction.

[0090] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention, and any reference signs in the claims should not be regarded as limiting the claims involved.

[0091] In addition, it should be understood that although this specification is described according to the embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for multi-person collaborative interaction of a mobile robot based on magnetic force feedback, characterized in that, It includes the following steps: Set operation parameters and select an operation mode; the operation parameters include the operation weights of the operators; The operation modes include: single operator operating a single robot, information fusion operation with multiple operators collaborating, and multiple operators cooperating to operate multiple robots; Receive the hand movement information of the operator through the interaction device as the operation information of the robot; the interaction device can move following the movement of the operator's hand; according to the force compensation algorithm, change the feedback force transmitted by the interaction device to the operator's hand, specifically including: the force compensation algorithm compensates the feedback force by perceiving the magnitude of the magnetic force and performing position offset based on the inverse kinematics of the robotic arm; Generate a motion instruction based on the operation information of a single operator or generate a motion instruction based on the operation information of multiple operators, perform predictive simulation on the motion instruction, and send the motion instruction verified through predictive simulation to the mobile robot for execution; When the operation mode selects the information fusion operation with multiple operators collaborating, the predictive simulation of the motion instruction specifically includes: Obtain task nature information, task target information, and execution process information, collect task area environment information, and establish a working environment-related model; Use the Hololens 2 client to create a mixed reality environment based on the task nature information, task target information, execution process information, and task area environment information; Set up the computer server side and the Hololens 2 client, establish multiple client communications, and share the mixed reality operation environment; Map the motion instruction generated based on the operation information and operation weights of multiple operators to the virtual robot in the predictive simulation software; if the virtual robot collides with the surrounding environment, correct the motion instruction to obtain the final motion instruction.

2. The multi-person collaborative interaction method of a mobile robot based on magnetic force feedback according to claim 1, characterized in that, The generation of the motion instruction based on the operation information of multiple operators specifically includes: S21. Set operation weights for each operator; S22. Process the operation data of each operator obtained, and eliminate unreasonable operation data; S23. Perform weighted calculation on the operation data of each operator according to the operation weight of the operator to obtain the motion instruction.

3. A multi-person collaborative interaction method for a mobile robot based on magnetic force feedback according to claim 1, characterized in that, The interaction device can move following the movement of the operator's hand, specifically including: S41. Receive hand movement information; S42. Allocate the movement distances of the trolley and the robotic arm by setting the allocation ratio and the movement limit range; S43. Transmit the allocated data to the robotic arm of the interaction device, perform the inverse kinematics of the robotic arm, and at the same time send the movement data of the trolley to the trolley industrial control computer; S44. Control the movement of the trolley through the trolley mobile chassis control algorithm, and at the same time control the movement of the robotic arm, and based on the hand movement information, make the interaction device move following the operator when the operator moves.

4. A multi-person collaborative interaction system for a mobile robot based on magnetic force feedback, characterized in that, It includes: Parameter setting module: used to set operation parameters and select an operation mode; The operation parameters include the operation weights of the operators; The operation modes include: single operator operating a single robot, information fusion operation with multiple operators collaborating, and multiple operators cooperating to operate multiple robots; Interaction module: Receives the hand movement information of the operator through the interaction device as the operation information of the robot; the interaction device can move following the operator's hand; according to the force compensation algorithm, changes the feedback force transmitted by the interaction device to the operator's hand, specifically including: the force compensation algorithm compensates the feedback force by perceiving the magnitude of the magnetic force and performing position offset based on the inverse kinematics of the robotic arm; Control module: Generates a motion instruction according to the operation information of a single operator or generates a motion instruction according to the operation information of multiple operators, performs predictive simulation on the motion instruction, and sends the motion instruction verified through predictive simulation to the mobile robot for execution; When the operation mode selects the information fusion operation of multi-person collaboration, the predictive simulation of the motion instruction specifically includes: Obtain the task nature information, task objective information and execution process information, collect the task area environment information and establish a model related to the working environment; Use the Hololens 2 client to create a mixed reality environment according to the task nature information, task objective information, execution process information and task area environment information; Build a computer server side and a Hololens 2 client, establish multiple client communications, and share the mixed reality operation environment; Map the motion instruction generated according to the operation information and operation weights of multiple operators to the virtual robot in the predictive simulation software; if the virtual robot collides with the surrounding environment, correct the motion instruction to obtain the final motion instruction.

Images