Multi-degree-of-freedom mechanical arm interaction device and control method thereof

Abstract

The invention provides a multi-degree-of-freedom mechanical arm interaction device and a control method thereof, and the device comprises a collection module which is used for collecting behavior data of an experiment object; the mechanical arm mechanism comprises a mechanical arm and an end effector; the end effector is detachably mounted at the tail end of the mechanical arm, and is used for an experimental subject to execute different operation tasks; the mechanical arm comprises at least six degrees of freedom and is used for moving in a three-dimensional space; the control module is electrically connected with the mechanical arm, and the control module generates a motion instruction through at least one of the behavior data and the pose feedback data of the mechanical arm and controls the mechanical arm to drive the end effector to move according to the motion instruction so as to drive the experiment object to execute a target operation task; the target operation tasks belong to different operation tasks. According to the device, the movement flexibility of the mechanical arm in the three-dimensional space and the grabbing capacity of the system for different tasks are improved, and then the interaction performance of the mechanical interaction system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic arm motion control, and particularly to a multi-degree-of-freedom robotic arm interaction device and a control method thereof. Background Art

[0002] In the cognitive and neuroscience research of non-human primates (such as monkeys), flexible and precise experimental equipment is crucial for studying the animal's motion control, learning ability, and decision-making mechanism.

[0003] In the related art, the current mechanical interaction systems for primate experiments mainly focus on joystick tasks, two-dimensional screen interactions, and robotic arm systems with fixed trajectories. Since the mechanical interaction systems are difficult to simulate natural behaviors, the interaction motion patterns of the experimental subjects are restricted, and it is difficult to achieve true autonomous control and natural three-dimensional operations. Moreover, traditional mechanical interaction systems usually use fixed grasping devices or tools to train the learning ability of experimental subjects, and it is difficult to adapt to different types of task requirements. Summary of the Invention

[0004] The present invention provides a multi-degree-of-freedom robotic arm interaction device and a control method thereof to solve the defects in the prior art that the mechanical interaction system is difficult to simulate natural behaviors, resulting in restricted interaction motion patterns of experimental subjects, and the prior art uses fixed grasping devices or tools to train the learning ability of experimental subjects, making it difficult to adapt to different types of task requirements. The device of the present invention improves the flexibility of the robotic arm moving in three-dimensional space and the grasping ability of the system for different tasks, thereby improving the interaction performance of the mechanical interaction system.

[0005] The present invention provides a multi-degree-of-freedom robotic arm interaction device, including: An acquisition module for acquiring the behavior data of the experimental subject; A robotic arm mechanism including a robotic arm and an end effector; The end effector is detachably mounted at the end of the robotic arm, and the end effector is used for the experimental subject to perform different operation tasks; The robotic arm includes at least 6 degrees of freedom, and the robotic arm is used to move in three-dimensional space; A control module is electrically connected to the robotic arm. The control module generates a motion instruction based on at least one of the behavior data and the pose feedback data of the robotic arm, and controls the robotic arm to drive the end effector to move according to the motion instruction, so as to drive the experimental subject to perform a target operation task; the target operation task belongs to the different operation tasks.

[0006] According to a multi-degree-of-freedom robotic arm interaction device provided by the present invention, the device further includes: A master control computer, communicatively connected to the controller, for sending a first time synchronization instruction to the controller; The master control computer is further configured to send a first instruction to the robotic arm through the controller, and the first instruction is used to edit different operation tasks; An acquisition card, with the master control computer and the end effector electrically connected to the acquisition card respectively, and the acquisition card is used to implement data interaction and time synchronization between different modules.

[0007] According to a multi-degree-of-freedom robotic arm interaction device provided by the present invention, the device further includes: A binocular eye movement tracking module, communicatively connected to the master control computer, for collecting eye movement data of an experimental subject; An electroencephalogram signal acquisition module, communicatively connected to the master control computer, for collecting electroencephalogram signals of the experimental subject; The master control computer is configured to send a second time synchronization instruction to the controller and a third time synchronization instruction to the electroencephalogram signal acquisition module; The master control computer is further configured to generate a second instruction according to at least one of the eye movement data and the electroencephalogram signals, so as to control the controller to control the pose state of the robotic arm according to the second instruction.

[0008] According to a multi-degree-of-freedom robotic arm interaction device provided by the present invention, the device further includes: A scenario indicator light group, deployed within the visual field of the experimental subject and electrically connected to the acquisition card; the scenario indicator light group includes a plurality of scenario indicator lights, and different scenario indicator lights correspond to different operation tasks; The master control computer is configured to send a fourth time synchronization instruction to the scenario indicator light group through the acquisition card; The master control computer is further configured to send a third instruction to the scenario indicator light group through the acquisition card to control the plurality of scenario indicator lights to change their working states.

[0009] According to a multi-degree-of-freedom robotic arm interaction device provided by the present invention, the device further includes: A reward module, electrically connected to the master control computer, for providing water source or food for the experimental subject; The master control computer is configured to send a fifth time synchronization instruction to the reward module; The master control computer is further configured to send a fourth instruction to the reward module to control the working state of the reward module.

[0010] A multi-degree-of-freedom robotic arm interaction device provided according to the present invention is characterized in that the end effector includes: An optoelectronic sensor is installed at the end of the end effector. The optoelectronic sensor is used to detect contact information between the experimental object and the end effector, and the contact information includes at least one of contact area, contact time, and pressure information.

[0011] The present invention also provides a control method for a multi-degree-of-freedom robotic arm interaction device, including: Collecting behavioral data of the experimental object based on the collection module; Generating a motion instruction based on the control module through at least one of the behavioral data and the pose feedback data of the robotic arm, and controlling the robotic arm to drive the end effector to move according to the motion instruction to drive the experimental object to perform a target operation task; Wherein, the end effector is detachably installed at the end of the robotic arm, and the end effector is used for the experimental object to perform different operation tasks; the robotic arm includes at least 6 degrees of freedom, and the robotic arm is used to move in three-dimensional space; the target operation task belongs to the different operation tasks. The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the control method of any one of the above-mentioned multi-degree-of-freedom robotic arm interaction devices.

[0012] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the control method of any one of the above-mentioned multi-degree-of-freedom robotic arm interaction devices.

[0013] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the control method of any one of the above-mentioned multi-degree-of-freedom robotic arm interaction devices.

[0014] The multi-degree-of-freedom robotic arm interaction device and its control method provided by the present invention use the collection module to collect the behavioral data of the experimental object, set the robotic arm to have at least 6 degrees of freedom, set the end effector to be detachably installed at the end of the robotic arm for the experimental object to perform different operation tasks, set the control module to generate a motion instruction through at least one of the behavioral data and the pose feedback data of the robotic arm, and control the robotic arm to drive the end effector to move according to the motion instruction to drive the experimental object to perform the target operation task, improving the flexibility of the robotic arm moving in three-dimensional space and the grasping ability of the system for different tasks, thereby improving the interaction performance of the mechanical interaction system. Description of the Drawings

[0015] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0016] Figure 1 It is one of the structural schematic diagrams of the multi-degree-of-freedom robotic arm interaction device provided by the present invention.

[0017] Figure 2 It is another structural schematic diagram of the multi-degree-of-freedom robotic arm interaction device provided by the present invention.

[0018] Figure 3 It is yet another structural schematic diagram of the multi-degree-of-freedom robotic arm interaction device provided by the present invention.

[0019] Figure 4 It is the flow schematic diagram of the control method of the multi-degree-of-freedom robotic arm interaction device provided by the present invention.

[0020] Figure 5 It is the structural schematic diagram of the electronic device provided by the present invention.

[0021] Reference numerals: 100: Acquisition module; 200: Robotic arm mechanism; 210: Robotic arm 220: End effector; 221: Photoelectric sensor; 300: Control module 400: Master control computer; 500: Acquisition card; 600: Binocular eye movement tracking module 700: EEG signal acquisition module; 800: Scenario indicator light group; 900: Reward module Detailed implementation manners

[0022] To make the objectives, technical solutions and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0023] The following will describe Figures 1-4 the multi-degree-of-freedom robotic arm interaction device and its control method of the present invention.

[0024] Figure 1 It is one of the structural schematic diagrams of the multi-degree-of-freedom robotic arm interaction device provided by the present invention, as Figure 1As shown in the figure, the multi-degree-of-freedom robotic arm interaction device includes: a collection module 100, a robotic arm mechanism 200, a robotic arm 210, an end effector 220, and a control module 300.

[0025] The collection module 100 is used to collect the behavioral data of the experimental subject.

[0026] In this embodiment, the collection module 100 can be an image or video acquisition device. For example, the collection module 100 is a camera or an image recognition device with a camera function.

[0027] In this embodiment, the behavioral data includes the posture and movement information of the experimental subject.

[0028] Taking the experimental subject as a monkey as an example, the upper limb movement and interaction behavior of the monkey are recorded by the collection module 100, and the depth information is obtained through the collected behavioral data to real-time identify the position of the monkey's hand and arm in the three-dimensional space, which is convenient for supporting subsequent data analysis.

[0029] The robotic arm mechanism 200 includes a robotic arm 210 and an end effector 220; the end effector 220 is detachably installed at the end of the robotic arm 210, and the end effector 220 is used for the experimental subject to perform different operation tasks; the robotic arm 210 includes at least 6 degrees of freedom, and the robotic arm 210 is used to move in the three-dimensional space.

[0030] In this embodiment, the robotic arm 210 includes a multi-joint link structure. For example, the robotic arm 210 includes structures such as a large arm, a small arm, a wrist, and a base, supporting at least 6 degrees of freedom, and realizing precise positioning and flexible movement in the three-dimensional space through the coordinated movement of the joints.

[0031] In this embodiment, the end effector 220 is replaceable to adapt to different task requirements; for example, the end effector 220 can adopt a standardized adaptation interface (mechanical buckle or electromagnetic lock), support quick installation and disassembly, and be compatible with different experimental task requirements (such as grasping, lifting, touch feedback).

[0032] In this embodiment, the end effector 220 can also be installed at the end of the robotic arm 210 by screws.

[0033] In this embodiment, the end effector 220 can also integrate different functional modules. For example, the end effector 220 integrates a tactile sensor or a flexible clamping structure, which is used to real-time monitor the contact force of the monkey, avoid damage caused by excessive force, and feedback to the control system through a data interface.

[0034] In this embodiment, the end effector 220 can be configured with customized actuators (such as a sphere grasping module, a lever trigger device) to adapt to different training scenarios (such as grasping a target object, pressing a switch).

[0035] The control module 300 is electrically connected to the robotic arm 210. The control module 300 generates a motion instruction based on at least one of the behavior data and the pose feedback data of the robotic arm 210, and controls the robotic arm 210 to drive the end effector 220 to move according to the motion instruction, so as to drive the experimental subject to perform the target operation task; the target operation task belongs to different operation tasks.

[0036] In this embodiment, the control module 300 can be connected to the robotic arm 210 through an electrical signal interface (such as UART, Bluetooth), receive task parameters from the upper computer (such as target pose, operation type), and generate a motion instruction based on the behavior data (the motion trajectory of the experimental subject) or the real-time pose feedback of the robotic arm 210 (joint angle, end coordinate).

[0037] In this embodiment, the control module 300 can receive the real-time pose feedback recorded by the robotic arm 210 through a network cable and send a motion instruction to the robotic arm 210.

[0038] Specifically, when it is detected that the monkey touches the end effector 220 to trigger an action, the control module 300 dynamically adjusts the traction force according to the behavior data of the monkey and the pose feedback of the robotic arm 210 to strengthen the motion training; in this embodiment, a six-degree-of-freedom robotic arm 210 is used with a replaceable physical operation module, which supports the monkey to perform actions such as grasping, pushing and pulling in three-dimensional space, and highly conforms to the natural motion behavior pattern of primates.

[0039] The multi-degree-of-freedom robotic arm interaction device provided by the embodiment of the present invention uses a collection module to collect the behavior data of the experimental subject, sets the robotic arm to have at least 6 degrees of freedom, and sets the end effector to be detachably installed at the end of the robotic arm for the experimental subject to perform different operation tasks. The control module generates a motion instruction based on at least one of the behavior data and the pose feedback data of the robotic arm, and controls the robotic arm to drive the end effector to move according to the motion instruction, so as to drive the experimental subject to perform the target operation task, improving the flexibility of the robotic arm moving in three-dimensional space and the grasping ability of the system for different tasks, and further improving the interaction performance of the mechanical interaction system.

[0040] In some embodiments, the multi-degree-of-freedom robotic arm 210 interaction device further includes: a master control computer 400 and a capture card 500.

[0041] The master control computer 400 is communicatively connected to the controller, and the master control computer 400 is used to send a first time synchronization instruction to the controller.

[0042] In this embodiment, the master control computer 400 can be connected to the controller through the UART / USB or CAN bus interface, and generate and send the first time synchronization instruction by using the NTP protocol or the custom timestamp protocol to ensure the alignment of the clock references of the two.

[0043] The master control computer 400 is also used to send the first instruction to the robotic arm 210 through the controller, and the first instruction is used to edit different operation tasks.

[0044] In this embodiment, the master control computer 400 can customize different experimental task paradigms, real-time control, and feedback calculation to ensure the collaborative work of each module.

[0045] In this embodiment, the first instruction includes the task type, path planning parameters, and the configuration of the end effector 220 (such as the opening and closing force of the gripper), and the controller can dynamically adjust the joint angles and end postures of the robotic arm 210 according to the first instruction.

[0046] The master control computer 400 and the end effector 220 are respectively electrically connected to the acquisition card 500, and the acquisition card 500 is used to realize data interaction and time synchronization between different modules.

[0047] In this embodiment, the acquisition card 500 is the data center of the multi-degree-of-freedom robotic arm 210 interaction device, and the acquisition card 500 can be responsible for collecting and synchronizing the transmission data between multiple modules through the IO port.

[0048] The multi-degree-of-freedom robotic arm interaction device provided by the embodiment of the present invention realizes time synchronization by setting up with the controller. The master control computer also sends the first instruction to the robotic arm through the controller to edit different operation tasks, supports the experimental object to perform more complex training tasks, and sets up the acquisition card to realize data interaction and time synchronization between different modules, improving the interaction efficiency of the multi-degree-of-freedom robotic arm interaction device.

[0049] In some embodiments, the multi-degree-of-freedom robotic arm 210 interaction device further includes: a binocular eye movement tracking module 600 and an electroencephalogram signal acquisition module 700100.

[0050] The binocular eye movement tracking module 600 is communicatively connected to the master control computer 400, and the binocular eye movement tracking module 600 collects the eye movement data of the experimental object.

[0051] In this embodiment, the binocular eye movement tracking module 600 includes two monocular eye trackers, which are used to monitor the eye movement information of the monkey in real time and provide the fixation point position of the monkey during the movement.

[0052] The electroencephalogram (EEG) signal acquisition module 700100 is communicatively connected to the master control computer 400, and the EEG signal acquisition module 700100 is used to acquire the EEG signals of the experimental subject.

[0053] In this embodiment, the EEG signal acquisition module 700100 includes two EEG acquisition devices to achieve multi-brain region and 128-channel signal acquisition.

[0054] The master control computer 400 is used to send a second time synchronization instruction to the controller and a third time synchronization instruction to the EEG signal acquisition module 700100.

[0055] In this embodiment, the master control computer 400 achieves multi-module clock alignment through the second time synchronization instruction (for the controller) and the third time synchronization instruction (for the EEG module).

[0056] In this embodiment, the second time synchronization instruction and the third time synchronization instruction can be the NTP protocol or a hardware trigger signal (such as the EXTI interrupt of STM32).

[0057] For example, the master control computer 400 sends a second time synchronization instruction to the binocular eye movement tracking module 600 through a network cable to achieve time synchronization between the master control computer 400 and the binocular eye movement tracking module 600, so as to ensure that the joint movement timing of the robotic arm 210 matches the eye movement data acquisition.

[0058] For example, the master control computer 400 sends a third time synchronization instruction to the EEG signal acquisition module 700100 through an IO port to achieve time synchronization between the master control computer 400 and the EEG signal acquisition module 700100, so as to ensure that the movement state of the experimental subject matches the corresponding EEG signal fluctuation trend.

[0059] The master control computer 400 is further used to generate a second instruction according to at least one of the eye movement data and the EEG signals, so as to control the controller to control the pose state of the robotic arm 210 according to the second instruction.

[0060] In this embodiment, the master control computer 400 obtains the change of the fixation point position of the monkey during the movement according to the eye movement data, and determines the action intention of the monkey during the movement according to the EEG signals; when both the eye movement data and the EEG signals are valid, the master control computer 400 can comprehensively generate a second instruction by using a dynamic weighted fusion algorithm.

[0061] For example, the weight of eye movement data focuses on spatial positioning accuracy, and the weight of EEG signal focuses on action intention confidence; if the eye movement data indicates that the target is offset but the EEG signal maintains the original instruction, the second instruction triggers path replanning (such as an arc obstacle avoidance trajectory) to prevent the robot arm 210 from colliding with obstacles; when an emergency stop intention is detected in the EEG signal, the second instruction can be used to immediately interrupt the current task and activate the safety shutdown protocol.

[0062] The multi-degree-of-freedom robotic arm interaction device provided in the embodiment of the present invention collects eye movement data of the experimental subject through a binocular movement tracking module, collects EEG signals of the experimental subject through an EEG signal collection module, and then generates a second instruction according to at least one of the eye movement data and the EEG signal through a master control computer, so as to control the controller to control the posture state of the robotic arm according to the second instruction, integrates behavioral data and eye movement data to realize synchronous recording of hand-eye coordination data, and synchronously records invasive neural signals (i.e., EEG signals) of multiple brain regions, thereby realizing high-quality multi-channel and multi-dimensional biological data recording.

[0063] In some embodiments, the multi-DOF robotic arm 210 interaction further includes: a scenario indicator light group 800 .

[0064] The scenario indicator light group 800 is deployed within the field of vision of the experimental subject and is electrically connected to the acquisition card 500; the scenario indicator light group 800 includes a plurality of scenario indicator lights, and different scenario indicator lights correspond to different operation tasks.

[0065] In this embodiment, at least one of the shape, color or brightness of the situation indicator light etc. can be used as a condition for triggering stress behavior of the experimental subject.

[0066] For example, three scene indicator lights of different colors (red, green, and blue) can be set within the monkey's field of vision. When the monkey sees the red scene indicator light up, it performs action A; when the monkey sees the green scene indicator light up, it performs action B; and when the monkey sees the blue scene indicator light up, it performs action C.

[0067] It should be noted that the monkeys can automatically trigger corresponding emergency behaviors when seeing lights of different colors turn on, which can be achieved through pre-training of the monkeys; for example, through multiple manual training in the past, the monkeys can be helped to establish conditioned reflexes to perform different actions when lights of different colors turn on.

[0068] The master control computer 400 is used to send a fourth time synchronization instruction to the scenario indicator light group 800 through the acquisition card 500 .

[0069] In this embodiment, the master control computer 400 sends a hardware trigger pulse signal (such as TTL level) through the acquisition card 500 as the fourth time synchronization instruction to ensure that the flashing frequency of the scene indicator light group 800 is strictly synchronized with modules such as the movement of the robotic arm 210 and EEG signal acquisition.

[0070] The master control computer 400 is also used to send a third instruction to the scene indicator light group 800 through the acquisition card 500 to control multiple scene indicator lights to change their working states.

[0071] In this embodiment, the master control computer 400 can send a third instruction to the scene indicator light group 800 through the PWM dimming protocol to dynamically adjust the brightness (adjustable from 0 to 100%) of each indicator light, the flashing mode (constant on / 1 Hz / 5 Hz), and the color combination (RGB color mixing) to match the operation requirements of different experimental stages.

[0072] The multi-degree-of-freedom robotic arm interaction device provided by the embodiment of the present invention sets different scene indicator lights of the scene indicator light group to correspond to different operation tasks, establishes time synchronization between the master control computer and the scene indicator light group, and finally sends a third instruction to the scene indicator light group through the acquisition card by the master control computer to control multiple scene indicator lights to change their working states, which can use the scene indicator lights to assist in training the experimental object, improve the monkey's attention to the target, and thus improve the interaction efficiency between the experimental object and the multi-degree-of-freedom robotic arm interaction device.

[0073] In some embodiments, the multi-degree-of-freedom robotic arm 210 interaction device further includes: a reward module 900.

[0074] The reward module 900 is electrically connected to the master control computer 400, and the reward module 900 is used to provide water source or food for the experimental object.

[0075] In this embodiment, the reward module 900 is deployed in the area accessible to the experimental object and is connected to the master control computer 400 through the CAN bus or RS-485 interface to receive instructions for adjusting the reward content.

[0076] In this embodiment, the reward module 900 can adopt a quantitative output mechanism driven by a micro servo motor, support water droplet injection or food particle release, and realize dose feedback control through a high-precision flow sensor and a photoelectric switch to ensure that the reward amount error is lower than the set error threshold.

[0077] In this embodiment, the master control computer 400 is used to send a fifth time synchronization instruction to the reward module 900.

[0078] In this embodiment, the master computer 400 can send a hardware-level synchronization pulse signal (such as TTL level), that is, the fifth time synchronization instruction, to the reward module 900 through ROS node communication, to ensure that the timing deviation between the reward trigger and modules such as the movement of the robotic arm 210 and EEG signal acquisition is within a controllable range.

[0079] In this embodiment, the master computer 400 strictly aligns the completion event of the joint movement of the robotic arm 210 with the reward release instruction through the fifth time synchronization instruction, to avoid the attenuation of the training effect caused by behavioral feedback delay.

[0080] The master computer 400 is also used to send a fourth instruction to the reward module 900 to control the working state of the reward module 900.

[0081] In this embodiment, the working states of the reward module 900 include: (1) opening the water source slot and closing the food slot; (2) closing the water source slot and opening the food slot; (3) opening the water source slot and the food slot; (4) closing the water source slot and the food slot.

[0082] In this embodiment, the master computer 400 generates a fourth instruction according to the experimental task state (such as successful target grasping, path planning timeout), and sets the following reward modes for the reward module 900: (1) Trigger mode: Immediately release a preset dose of reward after the task of the robotic arm 210 is completed; (2) Cumulative mode: Increase the reward amount according to the number of consecutive task completions to strengthen the long-term learning effect.

[0083] In this embodiment, the reward module 900 dynamically adjusts the reward type (water source / food) and dose according to the generated fourth instruction to strengthen the positive behavioral feedback of the experimental subject.

[0084] The multi-degree-of-freedom robotic arm interaction device provided by the embodiment of the present invention can provide water source or food for the experimental subject by setting the reward module, establish time synchronization between the master computer and the reward module, and set the master computer to send a fourth instruction to the reward module to control the working state of the reward module, and can be flexibly adjusted according to experimental needs, improving the learning efficiency and task participation of the experimental subject.

[0085] In some embodiments, the end effector 220 includes: a photoelectric sensor 221.

[0086] The photoelectric sensor 221 is installed at the end of the end effector 220. The photoelectric sensor 221 is used to detect the contact information between the experimental subject and the end effector 220. The contact information includes at least one of contact area, contact time, and pressure information.

[0087] In this embodiment, the master computer 400 can determine the contact situation between the experimental subject and the end effector 220 based on the contact information, so as to decide whether to start this interaction training or dynamically adjust the grasping strategy of the experimental subject acting on the end effector 220.

[0088] For example, if the contact area between the experimental subject and the end effector 220 exceeds the area threshold, it is determined that the experimental subject is ready to perform the target operation task, and then this interaction detection process is started.

[0089] For example, the master computer 400 dynamically adjusts the contact information of the experimental subject with the end effector 220 based on the signal of the optoelectronic sensor 221: ‌(1) Contact area control‌: When the contact area is lower than the preset threshold (such as 50% of the surface area of the target object), adjust the posture of the end effector 220 to increase the above-mentioned contact area. ‌(2) Contact time control‌: If the contact time is lower than the time threshold (such as 10 seconds), adjust the posture of the end effector 220 to increase the above-mentioned contact area.

[0090] ‌(3) Contact pressure control‌: If the contact pressure is lower than the pressure threshold (set according to user requirements, not lower than the force for the experimental subject to normally grasp an object), adjust the posture of the end effector 220 to increase the above-mentioned contact pressure.

[0091] The multi-degree-of-freedom robotic arm interaction device provided by the embodiment of the present invention can ensure the safety of the experimental subject operating the end effector and the stability of the interaction experiment by setting an optoelectronic sensor at the end of the end effector to detect the contact information between the experimental subject and the end effector.

[0092] Figure 2 It is the second structural schematic diagram of the multi-degree-of-freedom robotic arm interaction device provided by the present invention. In the Figure 2 shown embodiment, the multi-region invasive electroencephalogram signal acquisition system (corresponding to the electroencephalogram signal acquisition module) is used to acquire the electroencephalogram signals of the experimental animal, and the binocular eye movement tracking module is used to acquire the eye movement data of the experimental animal; the controller (see the Figure 2 lower right side corresponding to the computer logo) obtains the behavior data of the experimental animal through the motion capture system (corresponding to the acquisition module). The controller generates a motion instruction according to the behavior data and controls the six-degree-of-freedom robotic arm (corresponding to the robotic arm) to drive the modular end effector (corresponding to the end effector) to move, so as to drive the experimental animal to perform motion training; the master computer is electrically connected to the controller and controls the working states of the scene indicator light system and the feeding / water replacement device (corresponding to the reward system) through the acquisition card to assist the experimental animal in reinforcement learning training.

[0093] Figure 3This is the third schematic structural diagram of the multi-degree-of-freedom robotic arm interaction device provided by the present invention. In Figure 3 the illustrated embodiment, the computer 1 (corresponding to the controller) is connected to the camera via USB, to the robotic arm via network cable, and to the master computer via IO ports; the master computer is connected to the feeding / water system (corresponding to the reward system) via USB, to the electroencephalogram signal acquisition module via IO ports, and to the binocular eye movement tracking module via network cable. The master computer can send time synchronization commands to the directly connected modules above to achieve time alignment. The master computer is also electrically connected to the acquisition card; the end effector and the scene indicator group are respectively connected to the acquisition card via IO ports, that is, the master computer can send time synchronization commands to the end effector and the scene indicator group respectively through the acquisition card to achieve time alignment; the robotic arm is physically connected to the end effector; the camera can collect images in real time and send them to the computer for storage; the computer 1 can generate motion commands according to the received images and send them to the robotic arm to execute the motion commands and record the real-time pose of the robotic arm; In this embodiment, when the photoelectric sensor on the end effector detects that the experimental object touches the end effector, it generates a photoelectric signal and sends it to the master computer through the acquisition card. The master computer confirms that the experimental object is ready to perform the target operation task according to the photoelectric signal and sends behavior data to the computer 1. The computer 1 generates motion commands according to the behavior data and controls the robotic arm to execute the motion commands, thereby driving the experimental object at the end effector to perform motion learning.

[0094] Next, the control method of the multi-degree-of-freedom robotic arm interaction device provided by the present invention will be described. The control method of the multi-degree-of-freedom robotic arm interaction device described below can be mutually referred to with the multi-degree-of-freedom robotic arm interaction device described above.

[0095] Figure 4 This is the flowchart of the control method of the multi-degree-of-freedom robotic arm interaction device provided by the present invention. As Figure 4 shown, the control method of the multi-degree-of-freedom robotic arm interaction device includes the following steps: Step 410: Collect the behavior data of the experimental object based on the acquisition module.

[0096] In this step, the acquisition module can be an image or video acquisition device. For example, the acquisition module is a camera or an image recognition device with a camera function.

[0097] In this step, the behavior data includes the posture and movement information of the experimental object.

[0098] Taking the experimental object as a monkey as an example, record the upper limb movement and interaction behavior of the monkey through the acquisition module, and obtain depth information through the collected behavior data to real-time identify the position of the monkey's hand and arm in three-dimensional space, which is convenient for supporting subsequent data analysis.

[0099] Step 420: Based on the control module, generate a motion instruction using at least one of the behavior data and the pose feedback data of the robotic arm, and control the robotic arm to drive the end effector to move according to the motion instruction, so as to drive the experimental subject to perform the target operation task; wherein, the end effector is detachably mounted at the end of the robotic arm, and the end effector is used for the experimental subject to perform different operation tasks; the robotic arm includes at least 6 degrees of freedom, and the robotic arm is used to move in three-dimensional space; the target operation task belongs to different operation tasks.

[0100] In this step, the robotic arm includes a multi-joint link structure. For example, the robotic arm includes structures such as the upper arm, forearm, wrist, and base, supports at least 6 degrees of freedom, and realizes precise positioning and flexible movement in three-dimensional space through the coordinated movement of the joints.

[0101] In this step, the end effector is replaceable to adapt to different task requirements; for example, the end effector can adopt a standardized adaptation interface (mechanical buckle or electromagnetic lock), support quick installation and disassembly, and be compatible with different experimental task requirements (such as grasping, lifting, touch feedback).

[0102] In this embodiment, the end effector can also be installed at the end of the robotic arm by screws.

[0103] In this embodiment, the end effector can also integrate different functional modules. For example, the end effector integrates a tactile sensor or a flexible clamping structure, which is used to monitor the contact force of the monkey in real time, avoid damage caused by excessive force, and feedback to the control system through a data interface.

[0104] In this embodiment, the end effector can be configured with a customized actuator (such as a sphere grasping module, a lever trigger device) to adapt to different training scenarios (such as grasping a target object, pressing a switch).

[0105] The control module is electrically connected to the robotic arm. The control module generates a motion instruction using at least one of the behavior data and the pose feedback data of the robotic arm, and controls the robotic arm to drive the end effector to move according to the motion instruction, so as to drive the experimental subject to perform the target operation task; the target operation task belongs to different operation tasks.

[0106] In this embodiment, the control module can be connected to the robotic arm through an electrical signal interface (such as UART, Bluetooth), receive task parameters from the host computer (such as target pose, operation type), and generate a motion instruction based on the behavior data (experimental subject action trajectory) or the real-time pose feedback of the robotic arm (joint angle, end coordinate).

[0107] In this embodiment, the control module can receive the real-time pose feedback of the robotic arm through a network cable and send motion commands to the robotic arm.

[0108] Specifically, when it is detected that the monkey touches the end effector to trigger an action, the control module dynamically adjusts the traction force according to the behavior data of the monkey and the pose feedback of the robotic arm to strengthen the motion training. In this embodiment, a six-degree-of-freedom robotic arm is paired with a replaceable physical operation module, which supports the monkey to perform actions such as grasping, pushing, and pulling in three-dimensional space, highly conforming to the natural motion behavior pattern of primates.

[0109] The control method of the multi-degree-of-freedom robotic arm interaction device provided by the embodiment of the present invention collects the behavior data of the experimental object through the acquisition module, generates a motion command based on at least one of the behavior data and the pose feedback data of the robotic arm by the control module, and controls the robotic arm to drive the end effector to move according to the motion command, so as to drive the experimental object to perform the target operation task, improving the flexibility of the robotic arm to move in three-dimensional space and the grasping ability of the system for different tasks, and thus improving the interaction performance of the mechanical interaction system.

[0110] Figure 5 An example of the physical structure diagram of an electronic device is shown as Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540. Among them, the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logical instructions in the memory 530 to execute the control method of the multi-degree-of-freedom robotic arm interaction device. The method includes: collecting the behavior data of the experimental object based on the acquisition module; generating a motion command based on at least one of the behavior data and the pose feedback data of the robotic arm by the control module, and controlling the robotic arm to drive the end effector to move according to the motion command, so as to drive the experimental object to perform the target operation task; wherein, the end effector is detachably installed at the end of the robotic arm, and the end effector is used for the experimental object to perform different operation tasks; the robotic arm includes at least 6 degrees of freedom, and the robotic arm is used to move in three-dimensional space; the target operation task belongs to different operation tasks.

[0111] In addition, when the logical instructions in the above-mentioned memory 530 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0112] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the control method of the multi-degree-of-freedom robotic arm interaction device provided by the above-mentioned various methods. The method includes: collecting the behavior data of the experimental object based on the acquisition module; generating a motion instruction based on the control module through at least one of the behavior data and the pose feedback data of the robotic arm, and controlling the robotic arm to drive the end effector to move according to the motion instruction to drive the experimental object to perform a target operation task; wherein, the end effector is detachably installed at the end of the robotic arm, and the end effector is used for the experimental object to perform different operation tasks; the robotic arm includes at least 6 degrees of freedom, and the robotic arm is used to move in three-dimensional space; the target operation task belongs to different operation tasks.

[0113] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the control method of the multi-degree-of-freedom robotic arm interaction device provided by the above-mentioned various methods. The method includes: collecting the behavior data of the experimental object based on the acquisition module; generating a motion instruction based on the control module through at least one of the behavior data and the pose feedback data of the robotic arm, and controlling the robotic arm to drive the end effector to move according to the motion instruction to drive the experimental object to perform a target operation task; wherein, the end effector is detachably installed at the end of the robotic arm, and the end effector is used for the experimental object to perform different operation tasks; the robotic arm includes at least 6 degrees of freedom, and the robotic arm is used to move in three-dimensional space; the target operation task belongs to different operation tasks.

[0114] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.

[0115] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-degree-of-freedom robotic arm interaction device, characterized in that: include: A collection module, used to collect behavioral data of experimental subjects; A robotic arm mechanism, including a robotic arm and an end effector; The end effector is detachably mounted on the end of the robotic arm, and the end effector is used for the experimental subject to perform different operation tasks; The robotic arm comprises at least 6 degrees of freedom, and the robotic arm is used to move in three-dimensional space; A control module is electrically connected to the robotic arm, and the control module generates a motion instruction through at least one of the behavior data and the posture feedback data of the robotic arm, and controls the robotic arm to drive the end effector to move according to the motion instruction, so as to drive the experimental object to perform a target operation task; the target operation task belongs to the different operation tasks.

2. The multi-degree-of-freedom robot arm interaction device according to claim 1, characterized in that: The device also includes: A master control computer, connected to the controller for communication, and configured to send a first time synchronization instruction to the controller; The master control computer is also used to send a first instruction to the robot arm through the controller, wherein the first instruction is used to edit different operation tasks; An acquisition card, the master control computer and the end effector are electrically connected to the acquisition card respectively, and the acquisition card is used to realize data interaction and time synchronization between different modules.

3. The multi-degree-of-freedom robot arm interaction device according to claim 2, characterized in that: The device also includes: A binocular movement tracking module, which is in communication with the master control computer and collects eye movement data of the experimental subject; An EEG signal acquisition module, which is in communication with the master control computer and is used to acquire EEG signals of the experimental subject; The master control computer is used to send a second time synchronization instruction to the controller and send a third time synchronization instruction to the EEG signal acquisition module; The master control computer is also used to generate a second instruction based on at least one of the eye movement data and the EEG signal to control the controller to control the posture state of the robotic arm according to the second instruction.

4. The multi-degree-of-freedom robot arm interaction device according to claim 2, characterized in that: The device also includes: A scenario indicator light group is deployed within the field of vision of the experimental subject and is electrically connected to the acquisition card; the scenario indicator light group includes a plurality of scenario indicator lights, and different scenario indicator lights correspond to different operation tasks; The master control computer is used to send a fourth time synchronization instruction to the scene indicator light group through the acquisition card; The master control computer is also used to send a third instruction to the scene indicator light group through the acquisition card to control the multiple scene indicator lights to change their working states.

5. The multi-degree-of-freedom robot arm interaction device according to claim 2, characterized in that: The device also includes: A reward module, the reward module is electrically connected to the master control computer, and the reward module is used to provide water or food for the experimental subject; The master control computer is used to send a fifth time synchronization instruction to the reward module; The master control computer is also used to send a fourth instruction to the reward module to control the working state of the reward module.

6. The multi-degree-of-freedom robot arm interaction device according to claim 1 or 2, characterized in that: The end effector comprises: A photoelectric sensor is installed at the end of the end effector, and is used to detect contact information between the experimental object and the end effector, wherein the contact information includes at least one of contact area, contact time and pressure information.

7. A control method for a multi-degree-of-freedom robotic arm interaction device, characterized in that: include: Collecting behavioral data of experimental subjects based on the collection module; Based on the control module, a motion instruction is generated through at least one of the behavior data and the posture feedback data of the robot arm, and the robot arm is controlled to drive the end effector to move according to the motion instruction, so as to drive the experimental subject to perform the target operation task; Among them, the end effector can be detachably installed on the end of the robotic arm, and the end effector is used for the experimental subject to perform different operation tasks; the robotic arm includes at least 6 degrees of freedom, and the robotic arm is used to move in three-dimensional space; the target operation task belongs to the different operation tasks.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the control method of the multi-degree-of-freedom robotic arm interaction device as claimed in claim 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the control method of the multi-degree-of-freedom robotic arm interaction device as claimed in claim 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the control method of the multi-degree-of-freedom robotic arm interaction device as claimed in claim 7 is implemented.

Images