A target tracking device and method based on multi-sensor

Abstract

The present invention relates to a target tracking device based on multiple sensors, including a housing, a pan-tilt, a camera module, an MCU, a GPS positioning module, a posture acquisition module, a power supply, a motor and a motion execution module; the pan-tilt is installed on the housing, the camera module is installed on the pan-tilt, the MCU, the GPS positioning module, the posture acquisition module, the power supply and the motor are installed in the housing respectively, the motion execution module is connected to the motor and installed at the bottom of the housing, the pan-tilt, the camera module, the GPS positioning module, the posture acquisition module, the power supply and the motor are connected to the MCU respectively, and the present invention also relates to a target tracking method based on multiple sensors, which is used for the above target tracking device. Compared with the prior art, the present invention has the advantages of compensating for angle lag and high precision.

Description

Technical Field

[0001] The present invention relates to the technical field of target image tracking, and in particular to a target tracking device and method based on multiple sensors. Background Art

[0002] With the development of image technology, face recognition, target detection, etc. have been applied in many industries and fields. In some applications, it is necessary to mark each target in the image and track the marked part, so as to accurately know the real-time movement of the tracked target. At present, this kind of technology mainly starts with neural network training, learns the required tracking target, and then in actual use, performs image recognition and marking, determines its position in the current image, and adjusts the angle according to the deviation between the position and the center of the image to achieve tracking of the target object.

[0003] Chinese patent CN109389093A discloses a directional tracking method based on face recognition, which tracks the face by adjusting the steering gear by calculating the deviation between the face target and the center position of the image. The patent achieves real-time tracking of the target and has strong platform flexibility. However, the target tracking method in the patent still has the problem of lag in the rotation angle of the tracking camera, poor accuracy and poor stability. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a multi-sensor target tracking device and method that compensates for the angle lag and has high accuracy.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A target tracking device based on multiple sensors comprises a housing, a pan-tilt, a camera module, an MCU, a GPS positioning module, a posture acquisition module, a power supply, a motor and a motion execution module; the pan-tilt is mounted on the housing; the camera module is mounted on the pan-tilt; the MCU, the GPS positioning module, the posture acquisition module, the power supply and the motor are respectively mounted in the housing; the motion execution module is connected to the motor and mounted at the bottom of the housing; the pan-tilt, the camera module, the GPS positioning module, the posture acquisition module, the power supply and the motor are respectively connected to the MCU.

[0007] Preferably, the gimbal is a two-axis gimbal.

[0008] Preferably, the camera module includes a binocular camera for measuring the distance between the tracked target and the tracking device and a visible light camera for image recognition; the binocular camera and the visible light camera are both installed on the gimbal; the binocular camera and the visible light camera are respectively connected to the MCU.

[0009] Preferably, the GPS positioning module includes two GPS modules respectively connected to the MCU.

[0010] Preferably, the posture acquisition module is a nine-axis sensor.

[0011] Preferably, the motion execution module is a track or a wheel.

[0012] A target tracking method based on multiple sensors, characterized by comprising:

[0013] Step 1: Lock the target;

[0014] Step 2: Determine the position coordinates of the target tracking device at time t and time t-Δt;

[0015] Step 3: Obtain the motion direction, acceleration and angle data of the target tracking device at time t and time t-Δt through the attitude acquisition module;

[0016] Step 4: Obtain the distance between the target tracking device and the tracked target at time t and time t-Δt through the camera module;

[0017] Step 5: Obtain the gimbal angle parameters θ(α, β) at time t and time t-Δt, where α is the gimbal lateral angle parameter and β is the gimbal longitudinal angle parameter;

[0018] Step 6: Predict the position coordinates of the target tracking device and the tracked target at time t+Δt based on the movement direction, acceleration and position coordinate information of the target tracking device and the tracked target at time t and time t-Δt;

[0019] Step 7: Calculate the angle parameter θ of the gimbal at time t+Δt based on the position coordinates of the target tracking device and the tracked target at time t+Δt and the movement direction of the target tracking device p (α,β);

[0020] Step 8: Calculate the angle change through the gimbal angle parameters θ(α, β) at time t and time t-Δt, and predict the angle parameter θ of the gimbal at time t+Δt q (α,β);

[0021] Step 9: Integrate θ p (α,β) and θ q (α, β), obtain the angle parameter data θ of the gimbal at time t+Δt.

[0022] Preferably, the step 1 specifically includes: locking the tracked target through the RCNN target detection algorithm.

[0023] Preferably, the step 2 specifically includes: using the GPS positioning module (5) to determine the position coordinates of the target tracking device at time t and time t-Δt by a differential positioning method.

[0024] Preferably, the calculation method of θ in step 9 is:

[0025]

[0026] Where L is the distance between the target tracking device and the tracked target at time t; L min is the minimum distance that can be measured by the binocular camera; L max is the maximum distance that can be measured by the binocular camera.

[0027] Compared with the prior art, the present invention has the following advantages:

[0028] 1. High positioning accuracy: The target tracking device and the target tracking method of the present invention predict the spatial position coordinates of the target tracking device and the tracked target, and predict the rotation angle of the gimbal from time t to time t-Δt to improve the target tracking accuracy. The two prediction results are then integrated and the gimbal is controlled in advance to compensate for the angle lag.

[0029] 2. Safe and reliable: The target tracking device in the present invention is provided with a bumper, and the selected nine-axis sensor is provided with an industrial-grade waterproof connector, which improves the reliability of the device.

[0030] 3. Strong applicability: The target tracking device of the present invention can use wheels or tracks as motion actuators and can be applied to complex road conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the structure of the target tracking device in the present invention;

[0032] Figure 2 It is a schematic diagram of the structure of a target tracking device in an embodiment of the present invention;

[0033] Figure 3 Schematic diagram of the flow of the target tracking method in the present invention.

[0034] The numbers in the figure show:

[0035] 1. Housing, 2. Pan / tilt, 3. Camera module, 4. MCU, 5. GPS positioning module, 6. Posture acquisition module, 7. Power supply, 8. Motor, 9. Motion execution module, 301. Binocular camera, 302. Visible light camera. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0037] A target tracking device based on multiple sensors, the structure of which is as follows Figure 1 and Figure 2 As shown, it includes a housing 1, a pan-tilt 2, a camera module 3, an MCU4, a GPS positioning module 5, a posture acquisition module 6, a power supply 7, a motor 8 and a motion execution module 9. The pan-tilt 2 is mounted on the housing 1, and the camera module 3 is mounted on the pan-tilt 2. The MCU4, the GPS positioning module 5, the posture acquisition module 6, the power supply 7 and the motor 8 are all mounted in the housing 1, and the GPS positioning module 5, the posture acquisition module 6, the power supply 7 and the motor 8 are respectively connected to the MCU4. The motion execution module 9 is connected to the motor 8 and is mounted at the bottom of the housing 1 to realize the motion of the target tracking device.

[0038] The gimbal 2 in this embodiment is a two-axis gimbal.

[0039] The camera module 3 includes a binocular camera 301 for measuring the distance between the tracked target and the tracking device and a visible light camera 302 for image recognition. Both the binocular camera 301 and the visible light camera 302 are installed on the pan / tilt 2. The binocular camera 301 and the visible light camera 302 are respectively connected to the MCU 4. The camera module 3 in this embodiment uses Intel D435i.

[0040] MCU4 uses STM32 microcontroller.

[0041] The GPS positioning module 5 uses two RTK Multi-GNSS positioning modules.

[0042] The attitude acquisition module 6 is a nine-axis sensor. This embodiment uses the LPMS-RS232AL2 attitude sensor, which integrates a three-axis gyroscope, a three-axis accelerometer and a three-axis magnetometer. It can calculate the attitude direction, linear acceleration and altitude of the sensor in real time. RS232 is used as a communication means, and an industrial-grade waterproof connector is used to make the target tracking device more reliable.

[0043] The motion execution module 9 is a crawler track or a wheel. In this embodiment, a wheel is selected as the motion execution module.

[0044] The target tracking device in this embodiment is also provided with a wireless communication module, and the target tracking device communicates with an existing terminal through the module.

[0045] This embodiment also relates to a target tracking method based on multiple sensors, and its process is as follows: Figure 3 As shown, including:

[0046] Step 1: Lock the tracked target through the RCNN target detection algorithm;

[0047] Step 2: Determine the position coordinates of the target tracking device at time t and time t-Δt by differential positioning method;

[0048] Step 3: Obtain the motion direction, acceleration and angle data of the target tracking device at time t and time t-Δt through the posture acquisition module 6;

[0049] Step 4: Obtain the distance between the target tracking device and the tracked target at time t and time t-Δt through the camera module 3;

[0050] Step 5: Obtain the gimbal angle parameters θ(α, β) of gimbal 2 at time t and time t-Δt, where α is the gimbal lateral angle parameter and β is the gimbal longitudinal angle parameter;

[0051] Step 6: Predict the position coordinates of the target tracking device and the tracked target at time t+Δt based on the movement direction, acceleration and position coordinate information of the target tracking device and the tracked target at time t and time t-Δt;

[0052] Step 7: Calculate the angle parameter θ of the pan / tilt 2 at time t+Δt based on the position coordinates of the target tracking device and the tracked target at time t+Δt and the movement direction of the target tracking device p (α,β);

[0053] Step 8: Calculate the angle change by using the angle parameters θ(α, β) of the gimbal 2 at time t and time t-Δt, and predict the angle parameter θ of the gimbal 2 at time t+Δt q (α,β);

[0054] Step 9: Integrate θ p (α,β) and θ q (α, β), obtain the angle parameter data θ of the gimbal 2 at time t+Δt, the calculation method is:

[0055]

[0056] Where L is the distance between the target tracking device and the tracked target at time t; L min is the minimum distance that can be measured by the binocular camera; L max is the maximum distance that can be measured by the binocular camera.

[0057] In theory, the smaller the Δt in this embodiment, the better, which is determined by the performance of the MCU. However, it is limited by the accuracy of the sensor and the speed of the object. The appropriate value range of Δt is 10ms to 1s.

[0058] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A target tracking method using a target tracking device based on a multi-sensor, characterized in that: include: Step 1: Lock the target; Step 2: Determine the position coordinates of the target tracking device at time t and time t-Δt; Step 3: Obtain the movement direction, acceleration and angle data of the target tracking device at time t and time t-Δt through the posture acquisition module (6); Step 4: Obtain the distance between the target tracking device and the tracked target at time t and time t-Δt through the camera module (3); Step 5: Obtain the gimbal angle parameters θ(α, β) of the gimbal (2) at time t and time t-Δt, where α is the gimbal lateral angle parameter and β is the gimbal longitudinal angle parameter; Step 6: Predict the position coordinates of the target tracking device and the tracked target at time t+Δt based on the movement direction, acceleration and position coordinate information of the target tracking device and the tracked target at time t and time t-Δt; Step 7: Calculate the angle parameter θ of the pan / tilt (2) at time t+Δt based on the position coordinates of the target tracking device and the tracked target at time t+Δt and the movement direction of the target tracking device p (α,β); Step 8: Calculate the angle change through the gimbal angle parameters θ(α, β) of the gimbal (2) at time t and time t-Δt, and predict the angle parameter θ of the gimbal (2) at time t+Δt q (α,β); Step 9: Integrate θ p (α,β) and θ q (α, β), obtain the angle parameter data θ of the gimbal (2) at time t+Δt; The calculation method of θ in step 9 is: Where L is the distance between the target tracking device and the tracked target at time t; L min is the minimum distance that can be measured by the binocular camera; L max is the maximum distance that can be measured by the binocular camera.

2. The target tracking method using a multi-sensor based target tracking device according to claim 1, characterized in that: The step 1 is specifically as follows: locking the tracked target through the RCNN target detection algorithm.

3. The target tracking method using a multi-sensor based target tracking device according to claim 1, characterized in that: The step 2 specifically includes: using the GPS positioning module (5) to determine the position coordinates of the target tracking device at time t and time t-Δt by using a differential positioning method.

4. The target tracking method using a multi-sensor based target tracking device according to claim 1, characterized in that: The target tracking device comprises a housing (1), a pan-tilt platform (2), a camera module (3), an MCU (4), a GPS positioning module (5), a posture acquisition module (6), a power supply (7), a motor (8) and a motion execution module (9); the pan-tilt platform (2) is mounted on the housing (1); the camera module (3) is mounted on the pan-tilt platform (2); the MCU (4), the GPS positioning module (5), the posture acquisition module (6), the power supply (7) and the motor (8) are respectively mounted in the housing (1); the motion execution module (9) is connected to the motor (8) and is mounted at the bottom of the housing (1); the pan-tilt platform (2), the camera module (3), the GPS positioning module (5), the posture acquisition module (6), the power supply (7) and the motor (8) are respectively connected to the MCU (4).

5. The target tracking method using a multi-sensor based target tracking device according to claim 1, characterized in that: The pan / tilt head (2) is a two-axis pan / tilt head.

6. The target tracking method using a multi-sensor based target tracking device according to claim 1, characterized in that: The camera module (3) comprises a binocular camera (301) for measuring the distance between the tracked target and the tracking device and a visible light camera (302) for performing image recognition; the binocular camera (301) and the visible light camera (302) are both installed on the pan-tilt platform (2); the binocular camera (301) and the visible light camera (302) are respectively connected to the MCU (4).

7. The target tracking method using a multi-sensor based target tracking device according to claim 4, characterized in that: The GPS positioning module (5) comprises two GPS modules respectively connected to the MCU (4).

8. The target tracking method using a multi-sensor based target tracking device according to claim 1, characterized in that: The posture acquisition module (6) is a nine-axis sensor.

9. The target tracking method using a multi-sensor based target tracking device according to claim 4, characterized in that: The motion execution module (9) is a crawler track or a wheel.

Images