Multi-sensor fusion inspection robot

Abstract

The invention discloses a multi-sensor fusion inspection robot, and relates to the field of inspection robots. Comprising a vehicle body and an information terminal used for receiving collected information and controlling the vehicle body to move, an information processing unit and a signal transmission unit are arranged in the vehicle body, the vehicle body is further provided with an information collection unit, and the information collection unit comprises a route information collection unit and an environment information collection unit; wherein the vehicle body moves along the inspection moving route, and the inspection environment information is collected in the moving process; the control mode of the moving process of the vehicle body comprises the steps that control is conducted through the Raspberry Pi with pre-stored routing inspection moving route map information; manual remote control; active control; in the active control mode, the route information collecting unit collects inspection moving route information and sends the inspection moving route information to the information processing unit, the information processing unit generates a control instruction for driving the vehicle body to move, and the vehicle body moves along the inspection moving route in response to the control instruction.

Description

Technical Field

[0001] The present invention relates to the field of inspection robots, and in particular to a multi-sensor fusion inspection robot. Background Art

[0002] Using machinery and automation technology to replace manpower has become an inevitable trend in inspection management. Autonomous inspection robots with multi-data fusion can realize intelligent, digital and unmanned factory inspections, greatly improving the timeliness of inspection work.

[0003] There are two main control methods for patrol robot products on the market. One is manual remote control based on wireless information transmission; the other is automatic control of the patrol process based on simultaneous localization and mapping (SLAM) technology, which analyzes and processes the sensor information obtained by the robot's own sensor equipment. The current SLAM mainly includes methods based on sensors such as laser radar, camera, ultrasound, drone tilt photography, inertial measurement unit (IMU) and odometer. The premise of multi-sensor fusion SLAM is the accurate description of the same identification target by multiple sensors. It requires that the description content of the same target by each sensor device should be highly consistent in both spatial distribution and temporal distribution. However, due to the different data rates of sensors of different qualities and the differences in data nodes, it is necessary to perform joint calibration and data fusion of multiple sensors to ensure that multiple sensors describe the same object at the same time.

[0004] The mainstream multi-sensor fusion methods currently used are based on loose coupling or one sensor as the main sensor and the others as the auxiliary sensor. The degree of data coupling is low, and in actual operation, in order to ensure the integrity of the fused data, the fusion process is usually carried out mainly with sensors with slower upload rates. As a result, in the automatic control mode, the control timeliness of the inspection robot in an unknown environment is poor, and it is difficult to effectively control the inspection route. To this end, we propose a multi-sensor fusion inspection robot. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a multi-sensor fusion inspection robot, which solves the problem that in the automatic control mode in the background technology, the inspection robot has poor control timeliness in an unknown environment and is difficult to effectively control the inspection route.

[0006] Technical Solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a multi-sensor fusion inspection robot, including a movable body, an information terminal for receiving and collecting information and controlling the movement of the body, an information processing unit is arranged inside the body, and a signal transmission unit is also arranged for transmitting the processing signal of the information processing unit, and a communication connection is established between the signal transmission unit and the information terminal; the body is also provided with an information collection unit, and the information collection unit includes a route information collection unit for collecting inspection movement route information and an environment information collection unit for collecting inspection environment information; wherein, the body moves along the inspection movement route, and collects the inspection environment information during the movement; the movement process of the body along the inspection movement route is controlled by any of the following modes:

[0008] Mode 1: Controlled by Raspberry Pi with pre-stored inspection route map information;

[0009] Mode 2: Manual remote control;

[0010] Mode 3: Active control;

[0011] In the active control mode, the route information collection unit collects the inspection movement route information and sends it to the information processing unit, and the information processing unit generates a control instruction to drive the vehicle body to move. In response to the control instruction, the vehicle body moves along the inspection movement route.

[0012] The control process of the active control comprises the following steps:

[0013] S1, performing self-calibration on all sensors in the route information collection unit, and performing joint calibration among multiple sensors according to the transformation relationship among the sensors;

[0014] S2, collect k time nodes t1~t within the time interval t k The characteristic data values ​​of all sensors below It is represented as the characteristic data value of the jth sensor of the i-th category at the k-th time node;

[0015] S3, the characteristic data values ​​of the collected sensors Perform preprocessing, build a feature data evolution model, and obtain the end time node t at the time interval t through the feature data evolution model k The characteristic data values ​​of all sensors at this moment

[0016] S4, obtain the sensor characteristic data value Perform registration and convert the characteristic data values ​​of all sensors Perform data fusion and generate current inspection mobile route map information based on the data fusion results;

[0017] S5, obtaining the initial position information of the vehicle body in the inspection environment, iterating the initial position information according to the generated current inspection movement route map information, and obtaining the iterative position information of the vehicle body in the inspection environment;

[0018] S6, repeating steps S2-S5, generating the inspection movement route according to the iterative update of the position information, and generating the control instruction according to the inspection movement route.

[0019] The steps for constructing the feature data evolution model are as follows:

[0020] S31, set any r-th sensor in the route information collection unit as the main fusion sensor, r∈i, n is the number of r-th sensors, and construct a main fusion sensor at t k The characteristic data value matrix at the moment Among them, the matrix The expression is:

[0021] S32, construct time nodes t1~t k The characteristic value dataset of all sensors at the moment in,

[0022] S33, Extraction Matrix Any element in the uth column and the u+1th column, where u=1,2,...,k-1, is represented by the matrix The u-th column element in is the initial vector point set P u , with the u+1th column element as the end vector point set P u+1 ,but u is a positive integer, and u+1∈k;

[0023] S34, calculate the initial vector point set P u To the end vector point set P u+1 The vector increment point set ΔP, where

[0024] S35, constructing a vector increment prediction model according to the vector increment point set ΔP, and obtaining the k+1th time node t through the vector increment prediction model k+1 The vector increment at the time, the expression of the vector increment prediction model is: Get data fusion node t k+1 The characteristic data value of the main fusion sensor at the moment Where λ is a constant coefficient, and λ∈(0,1);

[0025] S36, repeat steps S31-S35 to determine the data fusion node t k+1 The characteristic data values ​​of all sensors at the moment are obtained to obtain the characteristic value data set of the data fusion node at the moment.

[0026] The route information collection unit comprises:

[0027] A radar installed on the front side of the vehicle body to detect environmental information of the inspection moving route;

[0028] A satellite navigation terminal installed inside the vehicle body for real-time positioning of the vehicle body;

[0029] An IMU terminal is installed inside the vehicle body to detect the vehicle body posture;

[0030] The route information collection unit also includes two sets of cameras symmetrically arranged in front of the vehicle body;

[0031] The information terminal used to control the movement of the vehicle body includes an APP remote control, a PS2 handle, and a model aircraft remote control in a manual remote control mode.

[0032] The communication mode between the signal transmission unit and the information terminal is any one of CAN communication, Bluetooth communication or serial communication.

[0033] The environmental information collection unit comprises:

[0034] A PM2.5 concentration sensor for collecting PM2.5 concentration information in the inspection environment, the PM2.5 concentration sensor is arranged at the front end of the vehicle body;

[0035] A smoke sensor for collecting smoke concentration information in the inspection environment, the smoke sensor is arranged at the front end of the vehicle body;

[0036] A temperature and humidity sensor for collecting temperature and humidity information of the inspection environment, the temperature and humidity sensor is arranged above the vehicle body;

[0037] A vehicle-mounted gimbal installed above the vehicle body.

[0038] The rear end of the vehicle body is provided with a charging female connector connected to the charging interface, and is also provided with a display installed thereon, and the display is communicatively connected with the information processing unit.

[0039] The beneficial effects of the present invention are:

[0040] 1. The multi-sensor fusion inspection robot proposed in the technical solution of the present invention, by setting a movable body and a signal terminal connected by a signal transmission unit, can determine the three-dimensional state of the environment of the body, the position of the body, the posture of the body, and the environmental picture of the body according to the radar, satellite navigation terminal, IMU terminal, and camera set on the body, so as to determine the switching of remote control of the body by using APP remote control, PS handle or model aircraft remote control, or directly control the movement of the body through the Raspberry Pi with pre-stored map information, or adjust and control the movement of the body by collecting information and processing it through a processor. Such a setting can realize the switching between active control and automatic control of the body.

[0041] 2. The multi-sensor fusion inspection robot proposed in the technical solution of the present invention can switch different communication modes according to the transmission distance and the influence of the transmission signal environment on the transmission signal by setting the communication mode between the signal transmission unit and the information terminal as one of CAN communication, Bluetooth communication or serial communication.

[0042] 3. The technical solution of the present invention proposes an active control mode for vehicle body movement. In the active control mode, the route information acquisition unit acquires the inspection route information and sends it to the information processing unit. The information processing unit generates a control instruction to drive the vehicle body to move. In response to the control instruction, the vehicle body moves along the inspection route. The active control mode that integrates multi-sensor data can timely integrate data from sensors of different qualities, thereby improving the control timeliness of the inspection robot in an unknown environment, thereby effectively controlling the robot's inspection route movement process. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic diagram of the structure of the multi-sensor fusion inspection robot proposed in the technical solution of the present invention;

[0044] Figure 2 This is a structural schematic diagram of the multi-sensor fusion inspection robot from another perspective proposed by the technical solution of the present invention;

[0045] Figure 3 This is a structural block diagram of the multi-sensor fusion inspection robot proposed in the technical solution of the present invention;

[0046] Figure 4 This is a control flow chart of the active control mode proposed in the technical solution of the present invention.

[0047] In the figure: 1. Car body; 2. Information processing unit; 3. Signal transmission unit; 4. Information collection unit; 5. Information terminal; 6. Raspberry Pi; 7. Display; 8. Charging female connector;

[0048] 41. Route information collection unit;

[0049] 411. Radar; 412. Satellite navigation terminal; 413. IMU terminal; 414. Camera;

[0050] 42. Environmental information collection unit;

[0051] 421, PM2.5 concentration sensor; 422, smoke sensor; 423, temperature and humidity sensor; 424, vehicle-mounted gimbal;

[0052] 511. APP remote control; 512. PS2 controller; 513. Model aircraft remote control. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] See also Figure 1-Figure 3 As shown, a multi-sensor fusion inspection robot includes a movable body 1, an information terminal 5 for receiving and collecting information and controlling the movement of the body 1, an information processing unit 2 is arranged inside the body 1, and a signal transmission unit 3 for transmitting the processing signal of the information processing unit 2 is also arranged, and a communication connection is established between the signal transmission unit 3 and the information terminal 5; the body 1 is also provided with an information collection unit 4, and the information collection unit 4 includes a route information collection unit 41 for collecting inspection movement route information and an environment information collection unit 42 for collecting inspection environment information;

[0055] The route information collection unit 41 includes:

[0056] A radar 411 disposed at the front side of the vehicle body 1 for detecting environmental information of the patrol movement route;

[0057] A satellite navigation terminal 412 disposed inside the vehicle body 1 for real-time positioning of the vehicle body 1;

[0058] An IMU terminal 413 disposed inside the vehicle body 1 for detecting the posture of the vehicle body 1;

[0059] The route information collection unit 41 also includes two sets of cameras 414 symmetrically arranged in front of the vehicle body 1;

[0060] The information terminal 5 for controlling the movement of the vehicle body 1 includes an APP remote control 511 for use in a manual remote control mode, a PS2 handle 512 , and an aircraft model remote control 513 .

[0061] The communication mode between the signal transmission unit 3 and the information terminal 5 is any one of CAN communication, Bluetooth communication or serial communication.

[0062] The environmental information collection unit 42 includes:

[0063] A PM2.5 concentration sensor 421 for collecting PM2.5 concentration information of the inspection environment, wherein the PM2.5 concentration sensor 421 is disposed at the front end of the vehicle body 1;

[0064] A smoke sensor 422 for collecting smoke concentration information in the inspection environment, the smoke sensor 422 is arranged at the front end of the vehicle body 1;

[0065] A temperature and humidity sensor 423 for collecting temperature and humidity information of the inspection environment, the temperature and humidity sensor 423 is arranged above the vehicle body 1;

[0066] A vehicle-mounted pan / tilt platform 424 is provided above the vehicle body 1 .

[0067] The rear end of the vehicle body 1 is provided with a charging female connector 8 connected to the charging interface, and is also provided with a display 7 installed thereon, and the display 7 is communicatively connected with the information processing unit 2 .

[0068] The vehicle body 1 moves along the inspection moving route and collects inspection environment information during the movement; the movement process of the vehicle body 1 along the inspection moving route is controlled by any of the following modes:

[0069] Mode 1: Controlled by Raspberry Pi 6 with pre-stored inspection route map information;

[0070] Mode 2: Manual remote control;

[0071] Mode 3: Active control;

[0072] When the movement process of the vehicle body 1 along the inspection moving route is controlled by mode 1, the Raspberry Pi 6 generates a control instruction according to the pre-stored inspection moving route map information, and controls the vehicle body 1 to move along the inspection moving route;

[0073] When the moving process of the vehicle body 1 along the inspection moving route is controlled by using the second mode, the three-dimensional coordinates of the robot can be obtained in real time through the satellite navigation terminal 412, and the APP is connected to the Bluetooth module interface of the vehicle body 1 through the signal transmission unit 3 for remote control, so as to debug the robot online, and the vehicle body 1 is moved by remote control using the APP remote control 511, PS2 handle 512 or model aircraft remote control 513;

[0074] When the moving process of the vehicle body 1 along the inspection moving route is controlled by mode 3, in the active control mode, the route information collection unit 41 collects the inspection moving route information and sends it to the information processing unit 2, and the information processing unit 2 generates a control instruction to drive the vehicle body 1 to move. In response to the control instruction, the vehicle body 1 moves along the inspection moving route, such as Figure 4 As shown, the specific control process includes the following steps:

[0075] S1, self-calibrate all sensors in the route information collection unit 41, and perform joint calibration among multiple sensors according to the transformation relationship among the sensors;

[0076] S2, collect k time nodes t1~t within the time interval t k The characteristic data values ​​of all sensors below It is represented as the characteristic data value of the jth sensor of the i-th category at the k-th time node;

[0077] S3, the characteristic data values ​​of the collected sensors Perform preprocessing, build a feature data evolution model, and obtain the end time node t at the time interval t through the feature data evolution model k The characteristic data values ​​of all sensors at this moment Among them, the steps for constructing the feature data evolution model are as follows:

[0078] S31, set any r-th sensor in the route information collection unit 41 as the main fusion sensor, r∈i, n is the number of r-th sensors, and construct the main fusion sensor at t k The characteristic data value matrix at the moment Among them, the matrix The expression is:

[0079] S32, construct time nodes t1~t k The characteristic value dataset of all sensors at the moment in,

[0080] S33, Extraction Matrix Any element in the uth column and the u+1th column, where u=1,2,...,k-1, is represented by the matrix The u-th column element in is the initial vector point set P u , with the u+1th column element as the end vector point set P u+1 ,but u is a positive integer, and u+1∈k;

[0081] S34, calculate the initial vector point set P uTo the end vector point set P u+1 The vector increment point set ΔP, where

[0082] S35, constructing a vector increment prediction model according to the vector increment point set ΔP, and obtaining the k+1th time node t through the vector increment prediction model k+1 The vector increment at time t, obtain the data fusion node t k+1 The characteristic data value of the main fusion sensor at the moment The expression of the vector incremental prediction model is:

[0083]

[0084] Where λ is a constant coefficient, and λ∈(0,1);

[0085] S36, repeat steps S31-S35 to determine the data fusion node t k+1 The characteristic data values ​​of all sensors at the moment, and the characteristic value data set at the moment of the data fusion node;

[0086] S4, obtain the sensor characteristic data value Perform registration and convert the characteristic data values ​​of all sensors Perform data fusion and generate current inspection mobile route map information based on the data fusion results;

[0087] S5, obtaining the initial position information of the vehicle body 1 in the inspection environment, iterating the initial position information according to the generated current inspection movement route map information, and obtaining the iterative position information of the vehicle body 1 in the inspection environment;

[0088] S6, repeating steps S2-S5, generating an inspection movement route according to the iterative update of the location information, and generating a control instruction according to the inspection movement route.

[0089] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A multi-sensor fusion inspection robot, comprising a movable body (1), and an information terminal (5) for receiving and collecting information and controlling the movement of the body (1), characterized in that: The vehicle body (1) is provided with an information processing unit (2) inside, and is also provided with a signal transmission unit (3) for transmitting a processing signal of the information processing unit (2), and a communication connection is established between the signal transmission unit (3) and the information terminal (5); the vehicle body (1) is also provided with an information collection unit (4), and the information collection unit (4) includes a route information collection unit (41) for collecting patrol movement route information and an environment information collection unit (42) for collecting patrol environment information; wherein the vehicle body (1) moves along the patrol movement route, and collects the patrol environment information during the movement; the movement process of the vehicle body (1) along the patrol movement route is controlled by any of the following modes: Mode 1: Controlled by Raspberry Pi (6) with pre-stored inspection route map information; Mode 2: Manual remote control; Mode 3: Active control; In the active control mode, the route information collection unit (41) collects the inspection movement route information and sends it to the information processing unit (2); the information processing unit (2) generates a control instruction for driving the vehicle body (1) to move; in response to the control instruction, the vehicle body (1) moves along the inspection movement route.

2. A multi-sensor fusion inspection robot according to claim 1, characterized in that: The control process of the active control comprises the following steps: S1, self-calibrating all sensors in the route information collection unit (41), and performing joint calibration among multiple sensors according to the transformation relationship among the sensors; S2, collect k time nodes t1~t within the time interval t k The characteristic data values ​​of all sensors below It is represented as the characteristic data value of the jth sensor of the i-th category at the k-th time node; S3, the characteristic data values ​​of the collected sensors Perform preprocessing, build a feature data evolution model, and obtain the end time node t at the time interval t through the feature data evolution model k The characteristic data values ​​of all sensors at this moment S4, obtain the sensor characteristic data value Perform registration and convert the characteristic data values ​​of all sensors Perform data fusion and generate current inspection mobile route map information based on the data fusion results; S5, obtaining initial position information of the vehicle body (1) in the inspection environment, iterating the initial position information according to the generated current inspection movement route map information, and obtaining iterative position information of the vehicle body (1) in the inspection environment; S6, repeating steps S2-S5, generating the inspection movement route according to the iterative update of the position information, and generating the control instruction according to the inspection movement route.

3. The multi-sensor fusion inspection robot according to claim 2, characterized in that: The steps for constructing the feature data evolution model are as follows: S31, assuming that any r-th sensor in the route information collection unit (41) is a main fusion sensor, r∈i, n is the number of r-th sensors, and constructs a main fusion sensor at t k The characteristic data value matrix at the moment Among them, the matrix The expression is: S32, construct time nodes t1~t k The characteristic value dataset of all sensors at the moment in, S33, Extraction Matrix Any element in the uth column and the u+1th column, where u=1,2,...,k-1, is represented by the matrix The u-th column element in is the initial vector point set P u , with the u+1th column element as the end vector point set P u+1 ,but u is a positive integer, and u+1∈k; S34, calculate the initial vector point set P u To the end vector point set P u+1 The vector increment point set ΔP, where S35, constructing a vector increment prediction model according to the vector increment point set ΔP, and obtaining the k+1th time node t through the vector increment prediction model k+1 The vector increment at the time, the expression of the vector increment prediction model is: Get data fusion node t k+1 The characteristic data value of the main fusion sensor at the moment Where λ is a constant coefficient, and λ∈(0,1); S36, repeat steps S31-S35 to determine the data fusion node t k+1 The characteristic data values ​​of all sensors at the moment are obtained to obtain the characteristic value data set of the data fusion node at the moment.

4. The multi-sensor fusion inspection robot according to claim 1, characterized in that: The route information collection unit (41) comprises: A radar (411) disposed on the front side of the vehicle body (1) for detecting environmental information of the inspection moving route; A satellite navigation terminal (412) disposed inside the vehicle body (1) and used for real-time positioning of the vehicle body (1); An IMU terminal (413) is arranged inside the vehicle body (1) and is used to detect the posture of the vehicle body (1); The route information collection unit (41) also includes two groups of cameras (414) symmetrically arranged in front of the vehicle body (1).

5. The multi-sensor fusion inspection robot according to claim 1, characterized in that: The communication mode between the signal transmission unit (3) and the information terminal (5) is any one of CAN communication, Bluetooth communication or serial communication.

6. The multi-sensor fusion inspection robot according to claim 1, characterized in that: The environmental information collection unit (42) comprises: A PM2.5 concentration sensor (421) for collecting PM2.5 concentration information of the inspection environment, wherein the PM2.5 concentration sensor (421) is arranged at the front end of the vehicle body (1); A smoke sensor (422) for collecting smoke concentration information in an inspection environment, wherein the smoke sensor (422) is arranged at the front end of the vehicle body (1); A temperature and humidity sensor (423) for collecting temperature and humidity information of the inspection environment, wherein the temperature and humidity sensor (423) is arranged above the vehicle body (1); A vehicle-mounted pan / tilt platform (424) is arranged above the vehicle body (1).

7. The multi-sensor fusion inspection robot according to claim 1, characterized in that: The rear end of the vehicle body (1) is provided with a charging female connector (8) connected to a charging interface, and is also provided with a display (7) installed thereon, and the display (7) is communicatively connected to the information processing unit (2).

8. The multi-sensor fusion inspection robot according to claim 1, characterized in that: The information terminal (5) used for controlling the movement of the vehicle body (1) comprises an APP remote control (511) used in a manual remote control mode, a PS2 handle (512), and a model aircraft remote control (513).

Images