A Shared Control Obstacle Avoidance Method Based on Brain-Machine Perception Fusion

Abstract

The present invention provides a shared control obstacle avoidance method based on brain-machine perception fusion, which includes an environmental perception module, a perception fusion module, and a path planning module. The environmental perception module uses a brain-computer interface and a lidar to perceive the environment and obtain the environmental information where the robot is located. The perception fusion module fuses the perception of the brain-computer interface and the lidar according to weights to obtain the environmental information required for the robot path planning. The path planning control module completes the intelligent obstacle avoidance of the robot. The beneficial effects of the present invention: The shared control obstacle avoidance method based on brain-machine perception fusion fuses the biological perception of humans and the machine perception of robots in the environment to complete multi-modal environmental perception, effectively improving the ability of humans and robots to jointly perceive and make decisions, and at the same time avoiding the cumbersome control of the brain-computer interface in various scenarios.

Description

Technical Field

[0001] The present invention belongs to the field of brain-computer interfaces in artificial intelligence, which is a cutting-edge human-computer interaction technology that combines biological decision-making intelligence and computer perception intelligence, and is a shared control obstacle avoidance method for brain-computer perception fusion. Background Art

[0002] Human-computer interaction is an important means of transmitting and exchanging information between humans, computers, and robots. Among them, the brain-computer interface, as a new type of human-computer interaction technology, has attracted much attention. The brain-computer interface decodes electroencephalogram signals to obtain the control intention of a person, and converts it into a control instruction for an external device, ultimately realizing direct communication between the brain and the robot. The brain-computer interface has broad prospects in the fields of medical rehabilitation and intelligent mobility, and can help amputee and paralyzed patients recover their motor abilities, take care of themselves in daily life, and achieve coordinated operations between humans and robots or robot substitution work in industries or high-risk industries. Shared control combines human biological intelligence and machine intelligence, that is, by decoding high-level instructions of a person through a brain-computer interface, and then fusing low-level instructions and fine instructions made by machine intelligence through environmental perception, complex environment control tasks can be completed.

[0003] When controlling the movement of an intelligent vehicle in a complex environment, direct control is limited by the accuracy and information transmission rate of the brain-computer interface, bringing great pressure and mental burden to the subject. Although shared control reduces the control freedom of the subject, it has better control effects, fewer control instructions and time, and is easier to complete movement control in complex environments.

[0004] Therefore, many scholars are committed to the research on brain-computer shared control technology. However, it is still a huge challenge to establish a reasonable and effective brain-computer shared control robot intelligent obstacle avoidance method in the problem of movement control in complex environments. Summary of the Invention

[0005] In view of this, the present invention aims to propose a shared control obstacle avoidance method based on brain-computer perception fusion, aiming at the cumbersome control problem of existing brain-controlled robots in complex scenarios, which brings great pressure and mental burden to the subject.

[0006] To achieve the above object, the technical solution of the present invention is realized as follows:

[0007] A shared control obstacle avoidance method based on brain-computer perception fusion includes an environmental perception module, a perception fusion module, and a path planning module. The environmental perception module perceives the environment through a brain-computer interface and a lidar to obtain the environmental information where the robot is located. The perception fusion module fuses the perception of the brain-computer interface and the lidar according to weights to obtain the environmental information required for the robot path planning. The path planning control module completes the intelligent obstacle avoidance of the robot.

[0008] The environmental perception module includes a brain-computer interface perception module and a lidar perception module. The brain-computer interface perception module is used to perceive high-level information of the environment where the robot is located, including a road extraction module, a sub-region segmentation module, and a brain-computer interface selection module. The road extraction module is used to extract road information of the environment where the robot is located. The sub-region segmentation module is used to divide the road area into multiple sub-regions and map the centroids of the sub-regions into the brain-computer interface paradigm. The brain-computer interface selection module is used to select the centroid points required for biological perception. The lidar perception module is used to perceive basic information of the environment where the robot is located;

[0009] The perception fusion module includes a weight analysis module and a perception fusion module. The weight analysis module is used to allocate the fusion weights of the brain-computer interface perception layer and the lidar perception layer. The perception fusion module is used to fuse the above two perception layers and generate a grid map;

[0010] The path planning and control module includes local path planning and control and global path planning and control.

[0011] According to the above technical solution, the shared control obstacle avoidance method based on brain-machine perception fusion includes the following steps:

[0012] Step S1: The perception module perceives the environmental information of the robot in real time;

[0013] Step S2: The perception fusion module receives the information from the perception module, fuses it according to the weights, and generates a grid map;

[0014] Step S3: The path planning and control module plans the global path and the local path according to the obtained grid map.

[0015] According to the above technical solution, step S1 further includes the following steps:

[0016] Step S11: The robot starts, and the lidar perceives the environmental information of the robot;

[0017] Step S12: The person judges whether more advanced environmental information is needed according to the environmental information of the robot;

[0018] Step S13: Analyze the myoelectric signal in real time. If it is greater than the set threshold, the brain-computer interface perception module is started. If it is less than the set threshold, the brain-computer interface perception module is not started.

[0019] According to the above technical solution, step S13 further includes the following steps:

[0020] Step S131: Start the road extraction module to extract the road area of the environment where the robot is located;

[0021] Step S132: Activate the promoter region segmentation module to segment the road area into multiple sub-regions and map the centroids of the sub-regions into the brain-computer interface paradigm;

[0022] Step S133: Activate the brain-computer interface selection module to select the centroid points required to sense the environmental information and connect the centroids in sequence to obtain the sensing area.

[0023] According to the above technical solution, the experimental shared control obstacle avoidance system based on brain-machine perception fusion includes an electroencephalogram electrode cap, an electroencephalogram signal collector, three computers, and a robot;

[0024] The electroencephalogram electrode cap is worn on the head of the subject, and the electroencephalogram signal collector is used to acquire electroencephalogram signals;

[0025] The robot is equipped with a lidar and a camera to complete the perception of basic environmental information;

[0026] The three computers are respectively used to present the brain-computer interface paradigm, process electroencephalogram signals in real time, and control the robot in real time. The three communicate with each other through TCP / IP for data transmission.

[0027] According to the above technical solution, a terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the shared control obstacle avoidance method based on brain-machine perception fusion.

[0028] Compared with the prior art, the shared control obstacle avoidance method based on brain-machine perception fusion of the present invention has the following advantages:

[0029] The construction method of the present invention, which is a shared control obstacle avoidance method based on brain-machine perception fusion, fuses human biological perception and robot machine perception in the environment to complete multi-modal environmental perception, effectively improving the ability of humans and robots to jointly perceive and make decisions, and avoiding the cumbersome brain-computer interface control in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings that form a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0031] Figure 1 It is a system diagram of the shared control system based on brain-machine perception fusion according to an embodiment of the present invention;

[0032] Figure 2 It is a visual instruction-induced interface diagram according to an embodiment of the present invention;

[0033] Figure 3This is the flowchart of shared control based on brain-machine perception fusion according to the embodiments of the present invention. Detailed implementation manners

[0034] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0035] The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0036] The present invention aims to propose a shared control obstacle avoidance method based on brain-machine perception fusion, aiming at the cumbersome control problem of existing brain-controlled robots in complex scenarios, which brings great pressure and mental burden to the subjects. The technical solutions for achieving the above invention purposes include an environmental perception module, a perception fusion module, and a path planning module. The environmental perception module uses a brain-computer interface and a lidar to perceive the environment and obtain the environmental information where the robot is located. The perception fusion module fuses the perception of the brain-computer interface and the lidar according to weights to obtain the environmental information required for the robot path planning. The path planning control module completes the intelligent obstacle avoidance of the robot.

[0037] This experiment was carried out in a quiet laboratory with good sound insulation. The subject sat on a comfortable chair, 70 cm away from the computer screen. The subject was in good mental state, with neat hair, and the experimental impedance was required to be below 5 KΩ. Before the experiment started, the subject wore an electroencephalogram electrode cap on the head, and the experimental process was explained to the subject until the subject fully understood the experimental process before starting the experiment. The robot site to be controlled consisted of a 225 cm × 460 cm rectangular baffle and two 35 cm × 85 cm rectangular baffles.

[0038] This experiment requires two parts: offline training and online control. In the first part of offline training, the subject annotates 12 targets to obtain labeled electroencephalogram data and complete the parameter training of the linear classifier. The experimental process of the second part of online control is as follows: when there is no electroencephalogram signal of biting teeth, the robot will automatically navigate from the lower right corner to the preset target (the upper left corner of the site) and perceive environmental obstacles in real time through the lidar to complete automatic obstacle avoidance. When driving near the obstacles that cannot be perceived by the lidar (two 35 cm × 80 cm rectangular obstacles), the electroencephalogram signal of biting teeth to stop is triggered. After the perception of the brain-computer interface is completed, it is fused with the lidar perception to obtain a fusion layer, and path planning and control are performed according to the fusion layer.

[0039] This experiment analyzes the results such as the number of experimental instructions, control time, number of collisions, path length, and subjective feelings of the subjects by comparing the direct control method of the brain-computer interface, the common brain-machine shared control method, and the shared control method of brain-machine perception fusion, and proves the effectiveness of the current shared control method of brain-machine perception fusion.

[0040] Those of ordinary skill in the art will appreciate that the units and method steps of the examples described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered by the scope of the claims and the description of the present invention.

[0042] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A shared control obstacle avoidance method based on brain-machine perception fusion, including an environmental perception module, a perception fusion module, and a path planning module, characterized in that: The environmental perception module uses a brain-computer interface and a lidar to perceive the environment, obtains the environmental information where the robot is located. The perception fusion module fuses the brain-computer interface perception and the lidar perception according to weights to obtain the environmental information required for the robot path planning. The path planning control module completes the intelligent obstacle avoidance of the robot; The environmental perception module includes a brain-computer interface perception module and a lidar perception module. The brain-computer interface perception module is used to perceive the high-level information of the environment where the robot is located, including a road extraction module, a sub-region segmentation module, and a brain-computer interface selection module. The road extraction module is used to extract the road information of the environment where the robot is located. The sub-region segmentation module is used to divide the road region into multiple sub-regions and map the centroid of the sub-region into the brain-computer interface paradigm. The brain-computer interface selection module is used to select the centroid points required for biological perception. The lidar perception module is used to perceive the basic information of the environment where the robot is located; The perception fusion module includes a weight analysis module and a perception fusion module. The weight analysis module is used to allocate the fusion weights of the brain-computer interface perception layer and the lidar perception layer. The perception fusion module is used to fuse the above two perception layers and generate a grid map; The path planning control module includes local path planning control and global path planning control.

2. A shared control obstacle avoidance method based on brain-machine perception fusion according to claim 1, characterized in that: The shared control obstacle avoidance method based on brain-machine perception fusion includes the following steps: Step S1: The perception module perceives the environmental information where the robot is located in real time; Step S2: The perception fusion module receives the information of the perception module, fuses it according to weights, and generates a grid map; Step S3: The path planning control module plans and controls the global path and the local path according to the obtained grid map.

3. A shared control obstacle avoidance method based on brain-machine perception fusion according to claim 2, characterized in that: The step S1 further includes the following steps: Step S11: The robot starts, and the lidar perceives the environmental information where the robot is located; Step S12: People judge whether high-level environmental information is needed according to the environmental information where the robot is located; Step S13: Analyze the myoelectric signal in real time. If it is greater than the set threshold, the brain-computer interface perception module is started. If it is less than the set threshold, the brain-computer interface perception module is not started.

4. A shared control obstacle avoidance method based on brain-machine perception fusion according to claim 3, characterized in that: The step S13 further includes the following steps: Step S131: Start the road extraction module to extract the road region of the environment where the robot is located; Step S132: Start the sub-region segmentation module to divide the road region into multiple sub-regions and map the centroid of the sub-region into the brain-computer interface paradigm; Step S133: Start the brain-computer interface selection module to select the centroid points required for perceiving environmental information, and connect the centroids in sequence to obtain the perception region.

5. A shared control obstacle avoidance method based on brain-machine perception fusion according to claim 1, characterized in that: The experiment is based on a shared control obstacle avoidance system for brain-machine perception fusion, including an EEG electrode cap, an EEG signal collector, three computers, and a robot; The EEG electrode cap is worn on the head of the subject, and the EEG signal collector obtains EEG signals; The robot is equipped with a lidar and a camera to complete the perception of basic environmental information; The three computers are respectively used to present the brain-computer interface paradigm, process EEG signals in real time, and control the robot in real time, and the three transmit data through TCP / IP.

6. A terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by the processor, the steps of the shared control obstacle avoidance method for brain-machine perception fusion described in claim 1 are implemented.

Images