Stinel robot based on Robomaster multi-sensor fusion decision

Abstract

The invention relates to the technical field of robots, and provides a Robomaster multi-sensor fusion decision-making-based sentry robot which comprises a chassis, a holder, an actuator module, a sensing module, a lower computer and an upper computer. The lower computer comprises a receiving module, a decision module, a distribution module and a motion control module; the upper computer comprises a laser radar navigation module and an image recognition processing module; the decision-making module is used for performing corresponding processing on the received data fusion robot according to the situation faced by the robot; and the distribution module is used for directly sending the received sensing data of the sensing module and the data of the judgment system to the laser radar navigation module and the image recognition processing module. According to the scheme, through multi-sensor fusion and a low-computing-power control system, a complex environment can be sensed more comprehensively, low-delay image processing and path planning are achieved, multiple independent and mutually decoupled modules are integrated, and high expansibility and stability are achieved.

Description

Technical Field

[0001] The invention relates to the technical field of robots, and in particular to a sentinel robot based on Robomaster multi-sensor fusion decision-making. Background Art

[0002] In robot competitions, such as Robomaster events, the competition scenes have extremely high real-time requirements. Robots must not only respond quickly to various dynamic changes, but also have the ability to move flexibly in areas with dense obstacles, identify teammates and opponents, and complete mission objectives. All of these require the robot system to have strong environmental perception, low-latency control, reliable navigation and processing capabilities. Traditional robot systems are often limited by computing power requirements and response speed, and it is difficult to meet the high-frequency task processing and complex scene response in competitions. Therefore, improving the adaptability and responsiveness of robots in real-time environments through multi-sensor fusion and efficient modular design has become a key research direction. In addition, existing systems mostly rely on high-computing "end-to-end" architectures, which are costly, insufficiently responsive in emergencies and system failures, and have limited adaptability. Summary of the invention

[0003] In order to solve the problems existing in the background technology, the present invention proposes a sentinel robot based on Robomaster multi-sensor fusion decision-making. Through multi-sensor fusion and low-computing power control system, it can perceive complex environments more comprehensively, realize low-latency image processing and path planning, and integrate multiple independent and mutually decoupled modules, with high scalability and stability.

[0004] To achieve the above-mentioned purpose, the present invention adopts the following scheme: a sentinel robot based on Robomaster multi-sensor fusion decision-making, including a chassis, a pan / tilt, an actuator module, a perception module, a lower computer and an upper computer; the lower computer includes a receiving module, a decision module, a distribution module and a control motion module; the receiving module is used to receive data perceived by the perception module, information fed back by the actuator module, information after the upper computer decision and data of the referee system; the decision module is used to make corresponding processing for the situation faced by the robot after fusing the received data; the distribution module is used to send the received perception data of the perception module and the referee system data directly to the upper computer; the control motion module is used to directly control the motor movement of the actuator module after receiving different instructions;

[0005] The upper computer includes a laser radar navigation module and an image recognition processing module; the image recognition processing module is connected to the lower computer through a USB virtual serial port, and is used to feed back the information after image recognition processing to the lower computer, and at the same time, various perception information received by the lower computer is used as one of the sources of image recognition processing data; the laser radar navigation module is connected to the lower computer after converting the USB to UART signal through the PB210 module, and is used to feed back its own decision information to the lower computer, and at the same time, various perception information received by the lower computer is used as one of the sources of its decision basis;

[0006] The perception module is used to receive external data and send the data to the lower computer through the serial bus UART. The perception module includes a nine-axis attitude sensor and a pressure sensor. The nine-axis attitude sensor is used to feed back the gimbal attitude to the lower computer through the bus IIC for data fusion and solution with the gimbal motor; the actuator module is used to receive the expected value output from the lower computer through the CAN bus.

[0007] Furthermore, the upper computer is connected to the lower computer via a communication interface based on the JSON communication protocol, so as to receive and send information in different computing languages.

[0008] Furthermore, the lower computer adopts a thread organization form to centralize modules at various levels to optimize the program structure; the lower computer optimizes the code by adopting a compiler optimization of level O3 and modularization during code writing.

[0009] Furthermore, the image recognition processing module includes an image processing computing platform and an industrial camera. The industrial camera is used to transmit the collected image data to the image processing computing platform for processing through a USB serial bus. The image processing computing platform is used to process the image data collected by the industrial camera and transmit the processed data to the lower computer through a virtual serial port module; the laser radar navigation module includes a 3D laser radar, a 2D laser radar and a navigation computing platform. The navigation computing platform is used to collect point cloud data of the 3D laser radar and the 2D laser radar, process the data, and send it to the lower computer through the PB2102 module; the decision module assigns different decision weights to the laser radar navigation module and the image recognition processing module according to the actual scenario that the robot faces.

[0010] Furthermore, the gimbal is rotatably arranged on the chassis through an electric slip ring, a plurality of omnidirectional wheels electrically connected to the chassis motor are arranged at the bottom of the chassis, the nine-axis attitude sensor is arranged at the top of the gimbal along the movement direction of the Pitch axis, the 3D laser radar is located at the front end of the top of the gimbal, the industrial camera is located on the gimbal below the 3D laser radar, the 2D laser radar is located on any side wall of the chassis, and the pressure sensor is located on the four side walls of the chassis.

[0011] Furthermore, the multi-sensor fusion is based on the information fusion from 3D laser radar, 2D laser radar, industrial camera and nine-axis attitude sensor of the laser radar navigation module, and the navigation computing platform packages the multi-sensor fusion information into function packages and nodes and transmits them to the decision module.

[0012] Furthermore, the actuator module includes a chassis motor and a gimbal motor; an omnidirectional wheel is provided at the bottom of the chassis, and the omnidirectional wheel is electrically connected to the chassis motor. The chassis motor transmits the current and coding value data to the lower computer through the CAN bus, and the lower computer returns the calculated expected current value to the chassis motor, and the chassis motor obtains a driving rotation signal for driving the omnidirectional wheel to rotate; the gimbal motor is connected to the lower computer through the CAN bus for controlling the 360° high-speed omnidirectional rotation of the gimbal.

[0013] Furthermore, the sentinel robot also includes a control method for using the sentinel robot, comprising the following steps:

[0014] Step 1: Set two independent startup modes for actual application scenarios and competition scenarios: Competition mode, when the remote control is turned to "Competition mode", the official referee system is connected. When the referee system issues the "Start the game" command, the lower computer directly receives the referee system data from the serial port, starts and executes the navigation program of the laser radar navigation module; Normal debugging mode, when the remote control is turned to "Debug mode", the robot directly starts and executes the navigation program;

[0015] Step 2: Initialization. During the process of starting the navigation program, keep the robot stationary and use a low-pass filter algorithm to filter the noise of the nine-axis attitude sensor data to ensure that the point cloud map matches the actual map.

[0016] Step 3: For the competition scenario, after the navigation program is running, the robot dynamically plans the path and avoids obstacles according to the preset target points on the map imported by the original program to reach the target point;

[0017] Step 4: During operation, the lower computer receives various data information, and takes the danger signals received by the referee system and the perception module as the top-level information. The decision module directly makes decisions on the robot: when the lower computer receives the "danger warning" from the referee system, it will start the automatic return home program, and the navigation program of the lidar navigation module will execute the relevant decisions on switching the target point and return by the original route; when the lower computer receives danger information from the pressure sensor, the robot will execute the small gyroscope on the spot and the gimbal 360° rotation recognition function to achieve the robot's self-protection.

[0018] The present invention has the following beneficial effects:

[0019] First, this solution adopts a two-layer decision-making process, with the lower computer as the top-level decision core, planning and controlling the coordination of the image recognition processing module, the laser radar navigation module and the lower computer's own modules. The decision layer of the lower computer has the highest decision-making power. Under the decision control of the lower computer, different decision weights are allocated to the image recognition processing module and the laser radar navigation module in real time according to the real-time perceived environmental conditions, which improves the robot's response speed to dangerous scenes and improves the robustness of the overall control. Moreover, the robot adopts a modular control system design, making each module independent and decoupled from each other, supporting rapid expansion and upgrading. The laser radar navigation module and the image recognition processing module can be quickly deployed to other devices to meet the needs of multiple scenarios such as competition, security and logistics, and enhance the system's adaptability and application flexibility.

[0020] Moreover, this solution adopts multi-sensor fusion, that is, the information fusion from 3D laser radar, 2D laser radar, industrial camera and nine-axis attitude sensor based on the laser radar navigation module to achieve high-precision real-time environmental perception, path planning and target recognition. Compared with the traditional single sensor system, the present invention can perceive complex environments more comprehensively and improve the robot's adaptability and decision-making accuracy in dynamic scenes. The navigation computing platform packages the information fused by multiple sensors into functional packages and nodes and transmits them to the decision-making module. The high reusability of this functional module allows it to be quickly deployed in different equipment and production environments. At the same time, it adopts a unique functional coordination control concept to reduce the total time consumption of the overall intelligent computing control processing, and has a strong ability to deal with risks in complex actual scenarios.

[0021] In addition, the lower computer senses the current real-time posture state of the robot through the IMU, and judges the rotation speed threshold of each motion axis within a certain period of time as the priority source of the lower computer's decision. When the robot encounters an emergency or failure, the lower computer can automatically start the return or power-off mechanism to ensure the safety of the robot and the user. Compared with the traditional design, the safety mechanism of the present invention is more perfect and suitable for industrial and security application scenarios that require high reliability.

[0022] Furthermore, this solution uses 24 conductive slip rings to decouple the gimbal from the chassis, allowing the robot to achieve 360° full-scale rotation while maintaining a stable connection between the mechanical and electrical systems. This design improves the reliability of the system and reduces the risk of cable entanglement or damage during movement.

[0023] In addition to being suitable for competitive robot events, this solution also has broad commercial application prospects and can be used in scenarios such as security inspections and logistics sorting. Its high scalability and reliability make it suitable for applications that require complex perception and autonomous decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The overall mechanical structure diagram of the present invention;

[0025] Figure 2 Module flow chart of the overall control system of the present invention;

[0026] Figure 3 Functional flow chart of dual decision layers of the present invention;

[0027] Figure 4 The 3D radar scanning range effect diagram of the present invention;

[0028] Figure 5 The grid map effect diagram constructed by 2D laser radar scanning of the present invention;

[0029] Figure 6 A relationship diagram of each node of the navigation system of the present invention;

[0030] Figure 7 The final effect diagram of the navigation function of the present invention.

[0031] Numbers in the figure: 1. Industrial camera; 2. Nine-axis attitude sensor; 3. Chassis; 4. 2D laser radar; 5. 3D laser radar; 6. Pan / tilt. DETAILED DESCRIPTION

[0032] In order to make the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the given embodiment is only one implementation method and does not represent all embodiments.

[0033] Embodiment 1

[0034] Combination Figure 1-Figure 7 This embodiment provides a sentinel robot based on Robomaster multi-sensor fusion decision-making, including a chassis 3, a gimbal 6, an actuator module, a perception module, a lower computer and an upper computer.

[0035] The lower computer includes a receiving module, a decision module, a distribution module and a control motion module. The receiving module is used to receive the data perceived by the perception module, the information fed back by the actuator module, the information after the decision of the upper computer and the data of the referee system; the decision module is used to fuse the received data into the situation faced by the robot and make corresponding processing; the distribution module is used to send the received perception data of the perception module and the referee system data directly to the laser radar navigation module and the image recognition processing module; the control motion module is used to directly control the motor movement of the actuator module after receiving different instructions.

[0036] The upper computer includes a laser radar navigation module and an image recognition processing module. The image recognition processing module is connected to the lower computer through a USB virtual serial port, and is used to feed back the information after image recognition processing to the lower computer, and at the same time, various perception information received by the lower computer is used as one of the sources of image recognition processing data; the laser radar navigation module is connected to the lower computer after converting USB to UART signal through the PB210 module, and is used to feed back its own decision information to the lower computer, and at the same time, various perception information received by the lower computer is used as one of the sources of its decision basis.

[0037] The control core of the lower computer in this embodiment adopts the MCU of the F4 series. The robot adopts a two-layer decision-making process, with the lower computer as the top-level decision-making core, planning, controlling and coordinating the image recognition processing module, the laser radar navigation module and the modules of the lower computer itself. The decision layer of the lower computer has the highest decision-making power. Under the decision control of the lower computer, different decision weights are allocated to the image recognition processing module and the laser radar navigation module in real time according to the real-time perceived environmental conditions, so as to improve the robot's response speed to dangerous scenes and improve the robustness of the overall control. Moreover, the robot adopts a modular control system design, so that each module is independent and decoupled from each other, supporting rapid expansion and upgrading. The laser radar navigation module and the image recognition processing module can be quickly deployed to other devices to meet the needs of multiple scenarios such as competition, security and logistics, and enhance the adaptability and application flexibility of the system.

[0038] The perception module is used to receive external data and send the data to the lower computer through the serial bus UART. The perception module includes a nine-axis attitude sensor 2. The nine-axis attitude sensor 2 is used to feed back the gimbal attitude to the lower computer through the IIC bus for data fusion and solution with the gimbal motor; the actuator module is used to receive the expected value output from the lower computer through the CAN bus.

[0039] The perception module also includes other sensors such as pressure sensors, and the actuator module is mainly a motor, a super capacitor group, etc. The actuator module described in this embodiment includes a chassis motor and a pan-tilt motor; an omnidirectional wheel is provided at the bottom of the chassis 3, and the omnidirectional wheel is electrically connected to the chassis motor. The chassis motor transmits the current and encoding value data to the lower computer through the CAN bus, and the lower computer returns the calculated expected current value to the chassis motor, and the chassis motor obtains a driving rotation signal to drive the omnidirectional wheel to rotate; the pan-tilt motor is connected to the lower computer through the CAN bus to control the 360° high-speed omnidirectional rotation of the pan-tilt 6. The pan-tilt 6 is connected to the chassis 3 through 24 conductive slip rings, which realizes the mechanical decoupling of the robot chassis 3 part and the pan-tilt 6 part, as well as the electrical connection. This design improves the reliability of the system and reduces the risk of cable entanglement or damage during movement.

[0040] Specifically, the upper computer is connected to the lower computer via a communication interface based on the JSON communication protocol, which is used to receive and send information in different computing languages. In this way, the reserved multiple sensor interfaces provide a basis for subsequent multi-sensor fusion.

[0041] Specifically, the lower computer adopts the organizational form of threads to concentrate modules at all levels to optimize the program structure; the lower computer optimizes the code by using compiler optimization and modularization during code writing. The lower computer needs to bear information from navigation, image processing, external sensors and control components, and perform a large amount of data processing, so reasonable program structure and code optimization are crucial. In terms of program structure, the organizational form of threads is adopted to concentrate modules at all levels to ensure that they do not interfere with each other during program operation, and the priority execution right of threads is determined according to the global state machine, which greatly increases the efficiency of the CPU. In terms of code optimization, one is to use compiler optimization with an optimization level of O3 to optimize invalid operations and memory usage in the compilation link, and the other is to modularize when writing code to ensure normal calls between layers, try to use local variables and dynamic variables, and release memory in time. Through the above method, both the real-time nature of control is met and the computing power of the lower control core can be utilized to the greatest extent.

[0042] Specifically, the image recognition processing module includes an image processing computing platform and an industrial camera 1. The industrial camera 1 is used to transmit the collected image data to the image processing computing platform through a USB serial bus for processing. The image processing computing platform is used to process the image data collected by the industrial camera 1, and transmit the processed data to the lower computer through a virtual serial port module; the laser radar navigation module includes a 3D laser radar 5, a 2D laser radar 4 and a navigation computing platform. The navigation computing platform is used to collect point cloud data of the 3D laser radar 5 and the 2D laser radar 4, process the data, and send it to the lower computer through the PB2102 module; the decision module assigns different decision weights to the laser radar navigation module and the image recognition processing module according to the actual scenario that the robot faces.

[0043] In this embodiment, the image processing computing platform is the NVIDIA JETSON Orin Nano computing platform. The lower computer communicates the image data received in real time by the industrial camera 1 with the Nano computing platform through the USB serial bus. The Nano computing platform can perform face detection through the face detection library in openCV, and in the competition, the armor plate is recognized by the model trained by the neural network, and finally the processed data is sent to the lower computer at a rate of 200HZ through the virtual serial port module. The navigation computing platform is the tenth generation Intel I5 computing platform, which obtains the point cloud data of the laser radar through the network cable at a frequency of 10HZ. After being processed by the computing platform, it is transferred through the PB2102 module to realize the serial communication of USB to UART and interact with the lower computer.

[0044] Specifically, the multi-sensor fusion is based on the information fusion from the 3D laser radar 5, 2D laser radar 4, industrial camera 1 and nine-axis attitude sensor 2 of the laser radar navigation module. The navigation computing platform packages the multi-sensor fusion information into function packages and nodes and transmits them to the decision module. The vertical angle of the existing MID360 laser radar scanning range is limited to 59°, such as Figure 4 The scanning range shown in the figure is that the scanning range of the laser radar is affected by the placement position, so most of the areas around the robot cannot be sensed. In actual competitive games, it cannot meet the needs of decision-making and path planning. Low and close obstacles may enter the blind spot of the MID360 laser radar, resulting in wrong local path planning, affecting the global path planning and the overall control decision of the robot, thus entering the dead zone. Therefore, in this embodiment, the existing 3D laser radar 5 is integrated with the 2D laser radar 4, the industrial camera 1, and the nine-axis attitude sensor 2 to meet more accurate decision-making information and other specific functions, and realize high-precision real-time environmental perception, path planning and target recognition. Compared with the traditional single sensor system, it can identify the surrounding environment faster and more accurately when facing complex terrain, and adjust decisions and behaviors in time, and finally adapt to the complex and changeable competition environment. The above sensor fusion is applied to the robot's navigation system, which can not only realize the precise positioning and obstacle avoidance of the robot, but also meet special needs through vision, such as the need for face detection and recognition in a specific environment, and the need to record certain biological features, which greatly enhances the scalability and high availability of the robot system.

[0045] The multi-sensor fusion information is packaged into function packages and nodes through Intel's tenth-generation I5 computing platform, and integrated into the decision-making system as the "branches and leaves" of the decision tree of the navigation system. In addition, the independent laser radar navigation module reserves a specially designed interface that can be coupled with different embedded systems as the source of perception information for the embedded lower computer of the navigation system. In this embodiment, the robot lower computer uses a program written in C language, and most of the upper computer navigation systems are written in Python language. Some nodes are written in C++ language, which also mixes different file languages. Therefore, it is of utmost importance to use a communication protocol with rich interfaces. This part uses the JSON protocol, and uses its huge language library to write communication interfaces suitable for more than three languages, so that the entire navigation system has a high degree of independence and can be generally applied to different types of embedded systems.

[0046] In the Robomaster competition, the different information provided by the referee system belongs to temporary and sudden real-life situations. In addition, there are other data except the referee system. In the face of the sudden attack of the opponent's robot, real-time and fast response is very important. The lower computer in this embodiment senses the real-time posture state of the current robot through IMU, and judges the rotation speed threshold on each motion axis within a certain period of time as the priority source of the lower computer decision. Once the IMU data is abnormal, and the storage IMU data in the single-chip microcomputer is in a large number of disordered situations, the output of each motor of the robot will be stopped immediately, and the robot will stop in place and wait for professional maintenance. IMU sends data to the lower computer at a high speed of 1000HZ through the serial port for processing, which can predict whether the navigation program is abnormal at the top level and terminate in time, improving the overall safety performance. And the lower computer will directly receive data from the referee system. After the lower computer makes a corresponding response to the referee system data, the decision-making authority will be delegated to the upper computer. Ensure the safety of the robot and the user. Compared with the traditional design, the safety mechanism of the present invention is more perfect and suitable for industrial and security application scenarios that require high reliability.

[0047] Specifically, the pan-tilt platform 6 is rotatably arranged on the chassis 3 through an electric slip ring. A plurality of omnidirectional wheels electrically connected to the chassis motor are arranged at the bottom of the chassis 3. The nine-axis attitude sensor 2 is arranged at the top of the pan-tilt platform 6 along the direction of motion of the Pitch axis. The 3D laser radar 5 is located at the front end of the top of the pan-tilt platform 6. The industrial camera 1 is located on the pan-tilt platform 6 below the 3D laser radar 5. The 2D laser radar 4 is located on any side wall of the chassis 3. The pressure sensor is located on the four side walls of the chassis 3. The use of an omnidirectional mobile chassis 3 enables the robot to move flexibly and accurately position in complex terrains, thereby improving the maneuverability and flexibility of the robot in competitions or practical applications. The main mapping activity of the 3D laser radar 5 for navigation is the source of information for path planning. However, it is limited in that it cannot scan in all directions without dead angles. Some terrains, such as steps and low tables, will appear. They are roughly the same height as the robot chassis 3, resulting in a mapping dead zone. At this time, the 2D radar placed on the robot chassis 3 will play an auxiliary role. The mapping effect is as follows: Figure 5 When the robot is trapped in a dead angle, that is, the task is not executed for a period of time, the message of the 2D laser radar 4 nodes of the navigation system will be assisted by the computing platform, and the navigation uses 2D radar for detection to realize obstacle detection and help the robot get out of trouble. In addition, the industrial camera 1 located in the middle of the robot can not only detect specific targets, but also integrate the IMU data, and locate the position information of specific targets in space based on a certain algorithm, thereby affecting the decision-making of the lower computer and realizing the tracking of specific targets.

[0048] Embodiment 2

[0049] The present invention also provides a control method for a sentinel robot based on Robomaster multi-sensor fusion decision-making, comprising the following steps:

[0050] Step 1: Startup. Use DJI's DT7&DR16 2.4GHZ remote controller to set two independent startup modes for actual application scenarios and competition scenarios: Competition mode. When the remote controller is turned to "Competition mode", the official referee system is connected. When the referee system issues the "Start the game" command, the lower computer directly receives the referee system data from the serial port, starts and executes the navigation program of the lidar navigation module; Normal debugging mode. When the remote controller is turned to "Debug mode", the robot directly starts and executes the navigation program.

[0051] Step 2: Initialization. To ensure the stability of the laser radar point cloud data and the nine-axis attitude sensor 2 data acquisition, and to ensure that the odometer of the navigation program works properly, during the process of starting the navigation program, the robot is kept still within the range visible to the naked eye. At the same time, a low-pass filtering algorithm is used to filter the noise of the nine-axis attitude sensor 2 data to ensure that the point cloud map matches the actual map.

[0052] Step 3: For the competition scenario, after the navigation program is run, the robot is initially in the "safe state" in the actual decision tree. The robot dynamically plans the path and avoids obstacles according to the preset target points on the map imported by the original program to reach the target point.

[0053] Step 4: During operation, the lower computer receives various data information, and takes the danger signals received by the referee system and the perception module as the top-level information. The decision module directly makes decisions on the robot. The lower computer perceives the data from the nine-axis attitude sensor 2, the referee system, etc. in advance. When encountering dangerous situations "notified" by the referee system and the nine-axis attitude sensor 2, such as rollover, fall, loss of control, etc., which can be judged by the threshold of the nine-axis attitude sensor 2, the lower computer first directly makes decisions on the robot, instead of letting the decision layer of the lower-level navigation computing platform determine what state the robot is in and then make corresponding strategies. When the lower computer receives a "danger warning" from the referee system, such as "its own health is too low", that is, the battery is too low, or "the base's health is too low", that is, the remote terminal informs that there is a more critical task, the automatic return home program will be started, and the navigation program of the lidar navigation module will execute the relevant decisions on switching the target point and return by the original route; when the lower computer receives danger information from the pressure sensor, that is, it is maliciously hit and hurt, the robot will execute the small gyroscope on the spot according to the decision module of the lower computer, and the 360° rotation recognition function of the gimbal 6 is used to realize the self-protection of the robot.

[0054] The lower computer decision module is the "brain" of the robot's overall control system. From the perspective of the lower computer decision layer, the lower computer decision layer will hierarchically determine which information can be executed in the current robot state and which data will be directly discarded. The priority of the perception information from the referee system and sensors will be higher than other information sources. By adding such a scheduling layer, different computing platforms are scheduled according to the weight of the real-time situation of the robot, which will greatly improve the response speed of the robot, rather than completely delegating the decision-making power to the navigation computing platform. Because the navigation system is limited by the scanning frequency of the laser radar, its data volume and data rate are much lower than the image recognition computing platform and the lower computer control core; on the other hand, after receiving all the message sources, the navigation system needs to be scheduled into the computing platform for planning and control, and then sent to the lower computer, which is also one of the reasons for the low overall control response rate. In this embodiment, part of the function is directly executed by the lower computer, and the image processing computing platform with a higher scheduling rate is used for pre-perception prediction, which reduces computing resources while also improving the overall control response rate of the robot.

[0055] The specific embodiments of the present invention are described in detail above with reference to the drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. The sentinel robot based on Robomaster multi-sensor fusion decision-making is characterized by: It includes a chassis (3), a pan / tilt platform (6), an actuator module, a sensing module, a lower computer and an upper computer; The lower computer includes a receiving module, a decision module, a distribution module and a control motion module; the receiving module is used to receive the data perceived by the perception module, the information fed back by the actuator module, the information after the decision of the upper computer and the data of the referee system; the decision module is used to fuse the received data with the situation faced by the robot and make corresponding processing; The distribution module is used to send the received perception module perception data and referee system data directly to the host computer; the control motion module is used to directly control the motor motion of the actuator module after receiving different instructions; The upper computer includes a laser radar navigation module and an image recognition processing module; the image recognition processing module is connected to the lower computer through a USB virtual serial port, and is used to feed back the information after image recognition processing to the lower computer, and at the same time, various perception information received by the lower computer is used as one of the sources of image recognition processing data; the laser radar navigation module is connected to the lower computer after converting the USB to UART signal through the PB210 module, and is used to feed back its own decision information to the lower computer, and at the same time, various perception information received by the lower computer is used as one of the sources of its decision basis; The perception module comprises a nine-axis attitude sensor (2) and a pressure sensor. The nine-axis attitude sensor (2) is used to feed back the attitude of the gimbal to a lower computer via a bus IIC for data fusion and solution with the gimbal motor. The actuator module is used to receive an expected value outputted from the lower computer via a CAN bus.

2. The sentinel robot based on Robomaster multi-sensor fusion decision-making according to claim 1, characterized in that: The upper computer is connected to the lower computer via a communication interface based on the JSON communication protocol, which is used for receiving and sending information in different computing languages.

3. The sentinel robot based on Robomaster multi-sensor fusion decision-making according to claim 1, characterized in that: The lower computer adopts the thread organization form to concentrate modules at various levels to optimize the program structure; the lower computer optimizes the code by adopting the compiler optimization of level O3 and modularization during code writing.

4. The sentinel robot based on Robomaster multi-sensor fusion decision-making according to claim 1, characterized in that: The image recognition processing module comprises an image processing computing platform and an industrial camera (1), wherein the industrial camera (1) is used to transmit the collected image data to the image processing computing platform via a USB serial bus for processing, and the image processing computing platform is used to process the image data collected by the industrial camera (1), and transmit the processed data to a lower computer via a virtual serial port module; the laser radar navigation module comprises a 3D laser radar (5), a 2D laser radar (4) and a navigation computing platform, and the navigation computing platform is used to collect point cloud data of the 3D laser radar (5) and the 2D laser radar (4), process the data, and send the data to the lower computer via a PB2102 module; the decision module assigns different decision weights to the laser radar navigation module and the image recognition processing module according to the actual scene that the robot faces.

5. The sentinel robot based on Robomaster multi-sensor fusion decision-making according to claim 4, characterized in that: The gimbal (6) is rotatably arranged on the chassis (3) via an electric slip ring; a plurality of omnidirectional wheels electrically connected to the chassis motor are arranged at the bottom of the chassis (3); the nine-axis attitude sensor (2) is arranged on the upper part of the gimbal (6) along the direction of motion of the pitch axis; the 3D laser radar (5) is located at the front end of the top of the gimbal (6); the industrial camera (1) is located at the front end of the gimbal (6) below the 3D laser radar (5); the 2D laser radar (4) is located on any side wall of the chassis (3); and the pressure sensor is located on the four side walls of the chassis (3).

6. The sentinel robot based on Robomaster multi-sensor fusion decision-making according to claim 4, characterized in that: The multi-sensor fusion is based on the information fusion of the laser radar navigation module from the 3D laser radar (5), the 2D laser radar (4), the industrial camera (1) and the nine-axis attitude sensor (2). The navigation calculation platform packages the multi-sensor fusion information into function packages and nodes and transmits them to the decision module.

7. The sentinel robot based on Robomaster multi-sensor fusion decision-making according to claim 1, characterized in that: The actuator module comprises a chassis motor and a pan / tilt motor; an omnidirectional wheel is provided at the bottom of the chassis (3), the omnidirectional wheel is electrically connected to the chassis motor, the chassis motor transmits current and code value data to a lower computer via a CAN bus, and the lower computer returns the calculated expected current value to the chassis motor, and the chassis motor obtains a driving rotation signal for driving the omnidirectional wheel to rotate; The pan / tilt motor is connected to a lower computer via a CAN bus and is used to control the 360° high-speed omnidirectional rotation of the pan / tilt (6).

8. The sentinel robot based on Robomaster multi-sensor fusion decision-making according to claim 1, characterized in that: Also included is a control method for using the sentinel robot, comprising the following steps: Step 1: Set two independent startup modes for actual application scenarios and competition scenarios: Competition mode, when the remote control is turned to "Competition mode", the official referee system is connected. When the referee system issues the "Start the game" command, the lower computer directly receives the referee system data from the serial port, starts and executes the navigation program of the laser radar navigation module; Normal debugging mode, when the remote control is turned to "Debug mode", the robot directly starts and executes the navigation program; Step 2: Initialization: During the process of starting the navigation program, the robot is kept stationary and a low-pass filter algorithm is used to filter the noise of the nine-axis attitude sensor (2) data to ensure that the point cloud map matches the actual map. Step 3: For the competition scenario, after the navigation program is running, the robot dynamically plans the path and avoids obstacles according to the preset target points on the map imported by the original program to reach the target point; Step 4: During operation, the lower computer receives various data information, and takes the danger signals received by the referee system and the perception module as the top-level information. The decision module directly makes decisions on the robot: when the lower computer receives the "danger warning" from the referee system, it will start the automatic return program, and the navigation program of the laser radar navigation module will execute the relevant decisions of switching the target point and return to the original route; when the lower computer receives the danger information from the pressure sensor, the robot will execute the small gyroscope on the spot according to the decision module of the lower computer, and the gimbal (6) 360° rotation recognition function is used to realize the self-protection of the robot.

Images