Intelligent robot system based on multi-brain-domain collaborative architecture and control method

Abstract

The invention discloses an intelligent robot system based on a multi-brain-domain collaborative architecture and a control method, and belongs to the technical field of intelligent robots. A navigation branch brain carries out path planning according to an instruction of a brain center to obtain path data, and the path data is fed back to the brain center; the voice branch brain converts the collected voice data into a voice instruction and feeds back the voice instruction to the brain center; the obstacle avoidance branch brain obtains a safe distance through obstacle detection and feeds back the safe distance to the brain center; the vision branch brain performs image analysis through the collected image data to obtain an image state, and feeds back the image state to the brain center; the telephone branch brain obtains a telephone signal and feeds the telephone signal back to the brain center; and the brain center performs multi-modal data fusion operation on data fed back by the navigation branch brain, the voice branch brain, the obstacle avoidance branch brain, the visual branch brain and the telephone branch brain, and then issues an execution instruction to a corresponding execution component, so that the purpose of realizing calculation power and real-time decoupling through a brain-branch brain distributed architecture and realizing distributed cooperative control is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent robots, and specifically relates to an intelligent robot system and control method based on a multi-brain domain collaborative architecture. Background Art

[0002] An intelligent robot is a mechanical device that integrates multiple advanced technologies. It has the ability to perceive, make decisions, and execute. It can imitate or perform human behaviors and tasks. It is the product of a high degree of combination of physical labor and intellectual labor to build an artificial machine that can "think".

[0003] Intelligent robots have a fairly developed "brain", where the central processing unit works. This computer has a direct connection with the person operating it. Such a computer can perform purposeful actions and has internal information sensors, external information sensors, and effectors that act on the surrounding environment. It must have at least three elements: sensory elements, motor elements, and thinking elements. According to their intelligence levels, they can be divided into industrial robots, primary intelligent robots, intelligent agricultural robots, home intelligent care robots, and advanced intelligent robots. They mainly involve key technologies such as multi-sensor information fusion, navigation and positioning, path planning, robot vision, intelligent control, and human-computer interface technology, and can replace humans in dangerous and complex labor in unstructured environments.

[0004] Existing traditional robots have the following main defects:

[0005] (1) Centralized architecture bottleneck: Traditional robots all use a single master control, which makes it difficult to balance computing power and real-time performance for complex tasks, such as SLAM+voice+vision processing;

[0006] (2) Sensor isolation: There is a problem of asynchronous multi-sensor data, resulting in obstacle avoidance delays greater than 200ms and failure in dynamic environments;

[0007] (3) Energy consumption imbalance: Hardware-level dynamic power consumption management is not implemented, resulting in low battery life. Summary of the Invention

[0008] In view of this, in order to solve the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide an intelligent robot system and control method based on a multi-brain domain collaborative architecture to achieve the decoupling of computing power and real-time performance through a "brain-brain" distributed architecture, and based on a multi-sensor fusion technology of a hardware-level time synchronization protocol (PTP μs-level alignment) to achieve the purpose of distributed collaborative control of intelligent robots.

[0009] The technical solution adopted by the present invention is: an intelligent robot system based on a multi-brain domain collaborative architecture, the system comprising: a brain center, a navigation sub-brain, a voice sub-brain, an obstacle avoidance sub-brain, a vision sub-brain, and a telephone sub-brain respectively connected to the brain center in communication;

[0010] The navigation sub-brain performs path planning according to the instructions of the brain center to obtain path data, and feeds the path data back to the brain center;

[0011] The speech sub-brain performs semantic understanding on the collected speech data and converts it into speech instructions, and feeds the speech instructions back to the brain center;

[0012] The obstacle avoidance brain obtains the safe distance through obstacle detection, and feeds back the safe distance judgment to the brain center;

[0013] The visual brain performs image analysis on the collected image data to obtain an image state, and feeds the image state back to the brain center;

[0014] The telephone support brain obtains the telephone signal by detecting the SIM card signal and feeds the telephone signal back to the brain center;

[0015] Among them, the brain center performs multimodal data fusion operations on the data fed back by the navigation brain, voice brain, obstacle avoidance brain, vision brain and telephone brain, and then sends execution instructions to the corresponding execution components.

[0016] Furthermore, the navigation brain, voice brain, obstacle avoidance brain, vision brain and telephone brain are all connected to the brain center through hot plugging.

[0017] Furthermore, it also includes a power control module, which supplies power to the brain center, navigation branch brain, voice branch brain, obstacle avoidance branch brain, vision branch brain, telephone branch brain and execution components respectively.

[0018] Furthermore, the execution component includes a motor, and the brain center is electrically connected to the motor through a PWM control circuit.

[0019] Furthermore, the brain center is connected to a human-computer interaction module, which includes: a display screen, a buzzer and a remote control handle. The display screen and the buzzer are electrically connected to the brain center, and the remote control handle is wirelessly connected to the brain center via Bluetooth or WiFi.

[0020] Furthermore, the brain center is electrically connected to an ultrasonic sensor, a radar sensor and an infrared sensor, and obstacle detection data is collected through the ultrasonic sensor, the radar sensor and the infrared sensor.

[0021] Furthermore, the brain center is connected to a camera module, through which image data is collected.

[0022] The present invention also discloses a method for controlling an intelligent robot based on a multi-brain domain collaborative architecture, the method comprising:

[0023] S1: Receive external user commands, analyze the type of the user commands, and the brain center determines the priority through multimodal data fusion calculation. If it is a voice command, it will run S2; if it is a visual command, it will run S3; if it is a telephone command, it will run S4;

[0024] S2: Call the voice support brain to perform semantic understanding on the voice command after voice recognition. If it is a movement command, it will execute A1; if it is an emergency command, it will execute A2;

[0025] S3: Call the visual sub-brain to perform target detection after image analysis of the visual command. If it is gesture control, execute A1; if it is an abnormal state, execute A2;

[0026] S4: The telephone support brain processes the telephone command through communication and then feeds it back to the brain center. After the brain center determines that the telephone command is dialing or answering, it issues a command to start the telephone interaction module;

[0027] Wherein, the A1 includes:

[0028] A101: The brain center calls the navigation sub-brain and obstacle avoidance sub-brain to obtain path planning and obstacle avoidance data respectively;

[0029] A102: Issues execution instructions to execution components based on path planning and obstacle avoidance data;

[0030] A103: Target verification is performed on the executed instruction. If the target is achieved, the system enters the standby state. If the target is not achieved, the system returns to A101.

[0031] The A2 includes: starting an alarm by triggering an alarm component.

[0032] Furthermore, the telephone interaction module includes a microphone array and a speaker, and the microphone array and the speaker are respectively connected to the brain center for communication.

[0033] Furthermore, the alarm component includes a buzzer and a SIM alarm, and the buzzer and the SIM alarm are electrically connected to the brain center respectively.

[0034] The beneficial effects of the present invention are:

[0035] 1. The intelligent robot system based on the multi-brain domain collaborative architecture provided by the present invention is mainly composed of a brain center, a navigation branch brain, a voice branch brain, an obstacle avoidance branch brain, a visual branch brain and a telephone branch brain. The brain center is mainly the core module of the robot, which is mainly responsible for integrating perception, decision-making and executing instructions. Through the communication between the brain center and each branch brain, a closed-loop system from sensor data processing to motion control is realized.

[0036] 2. The control method of the intelligent robot based on the multi-brain domain collaborative architecture provided by the present invention is adopted. The control method adopts the brain center + multiple branch brains for logical control. The brain center combines data from different sensors, different times and different spaces through a multimodal data fusion algorithm to improve the integrity and accuracy of the information. Each branch brain cooperates with the brain center to form a multi-brain domain collaborative architecture, which can greatly expand the functional richness and practicality of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is the overall architecture diagram of the intelligent robot system based on the multi-brain domain collaborative architecture provided by the present invention;

[0038] Figure 2 This is a logic flow chart of the intelligent robot control method based on the multi-brain domain collaborative architecture provided by the present invention;

[0039] Figure 3 This is the main control flow chart of the brain center in the intelligent robot based on the multi-brain domain collaborative architecture provided by the present invention. DETAILED DESCRIPTION

[0040] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar modules or modules with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. On the contrary, the embodiments of the present application include all changes, modifications, and equivalents that fall within the spirit and scope of the appended claims.

[0041] Example 1

[0042] The present invention provides an intelligent robot system based on a multi-brain domain collaborative architecture, which includes: a brain center, a navigation branch brain, a voice branch brain, an obstacle avoidance branch brain, a visual branch brain and a telephone branch brain respectively connected to the brain center for communication, so that the multi-brain domain architecture of the intelligent robot is jointly constituted by the above-mentioned brain center, navigation branch brain, voice branch brain, obstacle avoidance branch brain, visual branch brain and telephone branch brain.

[0043] As for the brain center, it is a heterogeneous computing center composed of Jetson AGX Orin + Xilinx Zynq MPSoC, which can achieve 32TOPS AI computing power and SLAM / NLP parallel processing functions. In actual application scenarios, specifically, the main control chip NVIDIA Jetson AGX Orin (32TOPS AI computing power) runs complex algorithms such as SLAM / NLP, and the real-time coprocessor Xilinx Zynq UltraScale + MPSoC hard real-time control can achieve <1ms delay control. The above-mentioned navigation brain, voice brain, obstacle avoidance brain, vision brain and telephone brain are all connected to the brain center in a hot-swappable manner. This modular design can realize the future functional scalability of the intelligent robot. At the same time, there is no need to restart the system of the intelligent robot when replacing the brain.

[0044] The designs for each brain branch are as follows:

[0045] (1) Navigation support

[0046] The navigation sub-brain communicates bidirectionally with the brain center. The navigation sub-brain calculates the SLAM mapping algorithm according to the mapping instructions issued by the brain center to perform path planning to obtain path data, and then feeds the path data back to the brain center.

[0047] (2) Voice-based brain

[0048] The brain center is electrically connected to a microphone array, which collects voice data. The voice sub-brain performs noise reduction and feature extraction on the collected voice data, converts the voice into text, and then interprets the semantic meaning of the voice text into voice commands. The voice commands are then fed back to the brain center for feedback and execution. Voice commands include movement commands, interaction commands, and emergency commands.

[0049] (3) Obstacle avoidance support

[0050] The ultrasonic sensor, radar sensor and infrared sensor are electrically connected by the brain center. The ultrasonic sensor, radar sensor and infrared sensor perform ultrasonic detection, radar detection and infrared detection respectively. According to the path data fed back by the navigation branch brain, obstacle detection data is collected by the ultrasonic sensor, radar sensor and infrared sensor on the planned path. The obstacle avoidance branch brain obtains the safe distance through obstacle detection, and feeds back the safe distance judgment to the brain center.

[0051] (4) Visual support of the brain

[0052] The brain center is connected to a camera module, which collects image data. The visual brain analyzes the collected image data to obtain the image status and feeds the image status back to the brain center.

[0053] (5) Telephone support

[0054] The telephone support brain obtains telephone signals after detecting SIM card signals. The telephone signals are divided into dialing instructions or incoming call requests, and feeds back the telephone signals to the brain center, which passes different telephone signals.

[0055] Among them, the brain center performs multimodal data fusion operations on the data fed back by the navigation branch brain, voice branch brain, obstacle avoidance branch brain, visual branch brain and telephone branch brain, and then sends execution instructions to the corresponding execution components, wherein the execution components include motors. The brain center is electrically connected to the motor through a PWM control circuit, that is, the operation of the motor is controlled by a PWM signal.

[0056] The brain center is also connected to a human-computer interaction module, which includes: a display screen, a buzzer and a remote control handle. The display screen and the buzzer are electrically connected to the brain center, and the remote control handle is wirelessly connected to the brain center via Bluetooth or WiFi. The display screen can display the status information of the intelligent robot to realize human-computer interaction; the buzzer transmits warning information of the intelligent robot to realize human-computer interaction; the remote control handle realizes manual control interaction between the user and the intelligent robot to realize human-computer interaction.

[0057] To ensure the normal operation of this intelligent robot, the robot is also designed with a power control module, which supplies power to the brain center, navigation sub-brain, voice sub-brain, obstacle avoidance sub-brain, vision sub-brain, telephone sub-brain, and execution components. Specifically, the power control module adopts a multi-source power supply topology mode as follows:

[0058] The power control module includes a lithium battery pack and a supercapacitor pack, which are connected via a bidirectional DC-DC connection. The other end of the lithium battery pack is connected to a charging management IC, which is connected to an AC / DC adapter. The other end of the supercapacitor pack is connected to a load distributor, which is electrically connected to the brain center, navigation sub-brain, voice sub-brain, obstacle avoidance sub-brain, vision sub-brain, telephone sub-brain, and execution components. The charging management IC can be either a wired or wireless charging module.

[0059] Example 2

[0060] Based on the intelligent robot with a multi-brain domain collaborative architecture provided in Example 1, this embodiment further discloses a control method for the intelligent robot, which includes:

[0061] S1: The robot is powered on and starts up. The intelligent robot receives user commands from the outside in real time. At the same time, the intelligent robot initializes various sensors, motors (motors), and various communication interfaces. The type of user commands received is analyzed through the voice collected by the microphone array, the images collected by the camera group, and the data collected by the sensor group. The brain center determines the priority through multimodal data fusion calculation. The priority is obstacle avoidance > navigation > interaction. If it is a voice command, it runs S2; if it is a visual command, it runs S3; if it is a telephone command, it runs S4;

[0062] S2: Call the voice support brain to perform noise reduction and feature extraction on the voice collected by the microphone array, and perform semantic understanding on the extracted voice data. If it is a movement / motion instruction, run A1; if it is an emergency instruction / emergency event, run A2; if it is an interactive instruction, run A3;

[0063] Wherein, the A1 includes:

[0064] A101: The brain center calls the navigation sub-brain and obstacle avoidance sub-brain to obtain path planning and obstacle avoidance data respectively; the details are as follows:

[0065] A1011: The brain center calls the navigation support brain and issues mapping instructions;

[0066] A1012: Mapping using SLAM algorithms;

[0067] A1013: Perform real-time positioning;

[0068] A1014: Perform path planning;

[0069] A1015: Send the planned path data to the brain center.

[0070] A102: Sends execution instructions to the execution component based on path planning and obstacle avoidance data. That is, the brain sends PWM signals to the motor / motor to control the motor / motor movement. During execution, the obstacle avoidance brain is called as follows:

[0071] A1021: Performing real-time scanning using a sensor group, wherein the sensor group includes at least an ultrasonic sensor and a radar sensor;

[0072] A1022: The obstacle avoidance sub-brain detects obstacles and determines whether they are within a safe distance. If the distance is below the threshold, an emergency command is sent to the brain center, and the brain waits for re-planning the path. If normal, continuous monitoring and execution of commands are continued.

[0073] A103: Real-time monitoring feedback is performed through the sensor group to verify the target of the execution instruction. If the target position is reached, it enters the standby state; if the target position is not reached, it returns to run A101;

[0074] Wherein, the A2 includes:

[0075] The alarm is activated by triggering an alarm component, wherein the alarm component includes a buzzer and a SIM alarm, and the buzzer and the SIM alarm are electrically connected to the brain center respectively, so as to transmit the alarm signal to the user through the buzzer and the SIM alarm to achieve the purpose of real-time interaction.

[0076] Among them, the A3 includes: the brain center calls the local corresponding library, and then transmits the information that needs to be interacted to the user by controlling the display screen or speaker to realize interaction with the user.

[0077] S3: Call the visual brain to perform target detection after image analysis of visual instructions, specifically:

[0078] Perform target detection on the image information collected by the camera group. If facial expressions / gestures are detected, recognize the facial expressions / gestures; if not, return to continue collecting images;

[0079] If it is recognized as gesture control, the above A1 is executed; if it is recognized as an abnormal state (fall / painful expression), the above A2 is executed.

[0080] S4: The telephone support brain processes the telephone command through communication and then feeds it back to the brain center. After the brain center determines that the telephone command is dialing or answering, it issues a command to start the telephone interaction module. Specifically:

[0081] If it is recognized as a dialing instruction, the brain center activates the radio frequency module to dial and establish a call connection, and starts the telephone interaction module;

[0082] If it is recognized as an answering command, the speaker rings through the brain center and decides whether to answer or reject the call. If the decision is to answer, the telephone interaction module is activated;

[0083] The telephone interaction module includes a microphone array and a speaker, and the microphone array and the speaker are respectively connected to the brain center for communication.

[0084] To further reduce the power consumption of the intelligent robot, a module power consumption monitoring algorithm is also loaded into the power management chip, as follows:

[0085] If the current of the corresponding branch is monitored to be overloaded, the corresponding branch will be cut off;

[0086] If the current of the corresponding branch is normal, the branch will continue to be powered normally.

[0087] It should be noted that any process or method description in the flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and that the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may not be performed in the order shown or discussed, including performing the functions in substantially the same manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0088] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0089] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0090] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0091] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0092] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An intelligent robot system based on a multi-brain domain collaborative architecture, characterized in that: The system includes: a brain center, a navigation sub-brain, a voice sub-brain, an obstacle avoidance sub-brain, a vision sub-brain, and a telephone sub-brain respectively connected to the brain center in communication; The navigation sub-brain performs path planning according to the instructions of the brain center to obtain path data, and feeds the path data back to the brain center; The speech sub-brain performs semantic understanding on the collected speech data and converts it into speech instructions, and feeds the speech instructions back to the brain center; The obstacle avoidance brain obtains the safe distance through obstacle detection, and feeds back the safe distance judgment to the brain center; The visual brain performs image analysis on the collected image data to obtain an image state, and feeds the image state back to the brain center; The telephone support brain obtains the telephone signal by detecting the SIM card signal and feeds the telephone signal back to the brain center; Among them, the brain center performs multimodal data fusion operations on the data fed back by the navigation brain, voice brain, obstacle avoidance brain, vision brain and telephone brain, and then sends execution instructions to the corresponding execution components.

2. The intelligent robot system based on multi-brain domain collaborative architecture according to claim 1 is characterized in that: The navigation brain, voice brain, obstacle avoidance brain, vision brain and telephone brain are all connected to the brain center through hot plugging.

3. The intelligent robot system based on multi-brain domain collaborative architecture according to claim 1 is characterized in that: It also includes a power control module, which supplies power to the brain center, navigation branch brain, voice branch brain, obstacle avoidance branch brain, vision branch brain, telephone branch brain and execution components respectively.

4. The intelligent robot system based on multi-brain domain collaborative architecture according to claim 1 is characterized in that: The execution component includes a motor, and the brain center is electrically connected to the motor through a PWM control circuit.

5. The intelligent robot system based on multi-brain domain collaborative architecture according to claim 1 is characterized in that: The brain center is connected to a human-computer interaction module, which includes: a display screen, a buzzer and a remote control handle. The display screen and the buzzer are electrically connected to the brain center, and the remote control handle is wirelessly connected to the brain center via Bluetooth or WiFi.

6. The intelligent robot system based on multi-brain domain collaborative architecture according to claim 1 is characterized in that: The brain center is electrically connected to an ultrasonic sensor, a radar sensor and an infrared sensor, and obstacle detection data is collected through the ultrasonic sensor, the radar sensor and the infrared sensor.

7. The intelligent robot system based on multi-brain domain collaborative architecture according to claim 1 is characterized in that: The brain center is connected to a camera module, through which image data is collected.

8. An intelligent robot control method based on a multi-brain domain collaborative architecture, characterized in that: The control method includes: S1: Receive external user commands, analyze the type of the user commands, and the brain center determines the priority through multimodal data fusion calculation. If it is a voice command, it will run S2; if it is a visual command, it will run S3; if it is a telephone command, it will run S4; S2: Call the voice support brain to perform semantic understanding on the voice command after voice recognition. If it is a movement command, it will execute A1; if it is an emergency command, it will execute A2; S3: Call the visual sub-brain to perform target detection after image analysis of the visual command. If it is gesture control, execute A1; if it is an abnormal state, execute A2; S4: The telephone support brain processes the telephone command through communication and then feeds it back to the brain center. After the brain center determines that the telephone command is dialing or answering, it issues a command to start the telephone interaction module; Wherein, the A1 includes: A101: The brain center calls the navigation sub-brain and obstacle avoidance sub-brain to obtain path planning and obstacle avoidance data respectively; A102: Issues execution instructions to execution components based on path planning and obstacle avoidance data; A103: Target verification is performed on the executed instruction. If the target is achieved, the system enters the standby state. If the target is not achieved, the system returns to A101. The A2 includes: starting an alarm by triggering an alarm component.

9. The intelligent robot control method based on multi-brain domain collaborative architecture according to claim 8 is characterized in that: The telephone interaction module includes a microphone array and a speaker, and the microphone array and the speaker are respectively connected to the brain center for communication.

10. The intelligent robot control method based on multi-brain domain collaborative architecture according to claim 8, characterized in that: The alarm component includes a buzzer and a SIM alarm, and the buzzer and the SIM alarm are electrically connected to the brain center respectively.

Images