Target tracking method and device based on multi-sensor fusion

Abstract

The invention discloses a target tracking method and device based on multi-sensor fusion. The method comprises the following steps: acquiring to-be-tracked target area information; performing registration processing on the camera, the laser radar and the millimeter wave radar; and performing fusion tracking processing on the target by using the camera, the laser radar and the millimeter wave radar to obtain target position information. According to the method, the problems of difficulty in realizing stable tracking of a target vehicle in an unstructured environment and how to effectively fuse multiple types of sensors by a single-system target tracking means are solved.

Description

Technical Field

[0001] The present invention relates to the fields of autonomous driving and target tracking, and particularly to a target tracking method and device based on multi-sensor fusion. Background Art

[0002] In the autonomous driving scenario, it is often necessary to track target vehicles or typical moving targets in space. The target tracking technology in structured scenarios has been relatively mature. For complex unstructured environments, such as environments with a large amount of dust or thick fog caused by the movement of the vehicle in front, it will bring great difficulties to vehicle target tracking.

[0003] For optical means such as cameras, existing target tracking methods can achieve good performance on public datasets. However, in unstructured scenarios, the complex road surface environment leads to unstable images. For example, rough terrain, changing light, and image jitter caused by partial occlusion of surrounding obstacles will seriously affect the tracking results. In addition, due to the limited field of view of the camera, when the leading vehicle turns, the target is easily lost from the field of view, resulting in its loss. At the same time, it is necessary to convert the position of the target in the image frame to the local frame of the vehicle through perspective transformation for motion planning. There are inevitable conversion errors in the conversion process, so it is difficult to provide accurate and stable positions for the follower. The leading vehicle is the target vehicle to be tracked.

[0004] For 3D lidar, compared with cameras, the position of the leading vehicle can be directly obtained from the point cloud space without perspective transformation, and the horizontal field of view is 360° without dead angles. Deep learning techniques widely used in image processing are also applied to 3D target detection. These methods are mainly used to solve tracking problems in urban environments. In the case of bumpy roads with dust in unstructured environments, due to the poor anti-interference ability of 3D lidar to dust, the performance of these methods will drop sharply. In addition, as the distance increases, the point cloud becomes sparse, resulting in a rapid decrease in detection accuracy with the increase of distance.

[0005] Millimeter wave radar has strong anti-interference ability to dust, rain, fog and snow. At present, millimeter wave radar is widely used in vehicle safety and adaptive cruise. Its disadvantage is that it cannot distinguish static vehicles and surrounding obstacles, which is important for the initialization of target tracking. When the leading vehicle starts statically, it needs other sensors to provide the initial tracking target. In addition, due to the limited field of view when turning, it is also easy to lose the tracking target.

[0006] In summary, a single target tracking method is difficult to achieve stable tracking of target vehicles in unstructured environments. How to effectively fuse multiple types of sensors is an urgent problem to be solved currently. Summary of the Invention

[0007] The present invention mainly solves the problem that it is difficult to stably track a target vehicle in an unstructured environment by using the target tracking means of a single system and how to effectively fuse multiple types of sensors. The present invention discloses a target tracking method and device based on multi-sensor fusion.

[0008] In the first aspect of the embodiments of the present application, a target tracking method based on multi-sensor fusion is disclosed, including:

[0009] S1, obtaining target area information to be tracked;

[0010] S2, performing registration processing on the target auxiliary tracking position information and the millimeter-wave radar;

[0011] S3, using the camera, lidar, and millimeter-wave radar to perform fusion tracking processing on the target to obtain target position information;

[0012] The performing registration processing on the camera, lidar, and millimeter-wave radar includes:

[0013] S21, performing systematic error registration processing on the camera, lidar, and millimeter-wave radar;

[0014] S22, performing time registration processing and space registration processing on the camera, lidar, and millimeter-wave radar;

[0015] S23, initializing space gate information; the space gate is a cube in space; the space gate information includes the x-axis distance threshold X1, y-axis distance threshold Y1, and z-axis distance threshold Z1 in a three-dimensional rectangular coordinate system.

[0016] The using the camera, lidar, and millimeter-wave radar to perform fusion tracking processing on the target to obtain target position information includes:

[0017] S31, using the camera to perform image acquisition processing on the target area information to be tracked to obtain target area image information; using the lidar to perform detection processing on the target area information to be tracked to obtain target area point cloud data;

[0018] S32, performing joint monitoring processing on the target area image information and the target area point cloud data to obtain target auxiliary tracking position information;

[0019] S33, using the millimeter-wave radar to perform tracking filtering processing on the target with the target auxiliary tracking position information as the guiding information to obtain target position sequence information; the target position sequence information includes a plurality of target positions;

[0020] S34. Perform a validity determination on the target position sequence information to obtain a first determination result. If the first determination result is valid, use the target position sequence information as the target position information. If the first determination result is invalid, execute S32.

[0021] The joint monitoring and processing of the target area image information and the target area point cloud data to obtain the target auxiliary tracking position information includes:

[0022] S321. Perform image target detection processing on the target area image information to obtain first target position area information.

[0023] S322. Perform target point cloud matching processing on the target area point cloud data to obtain second target position area information.

[0024] S323. Perform spatial gate detection on the first target position area information and the second target position area information to obtain the target auxiliary tracking position information.

[0025] S324. Perform a consistency determination on the target auxiliary tracking position information obtained at several times to obtain a second determination result. If the second determination result is valid, execute S33. If the second determination result is invalid, execute S31.

[0026] The performing of spatial gate detection on the first target position area information and the second target position area information to obtain the target auxiliary tracking position information includes:

[0027] S3231. Obtain the overlapping area information of the first target position area information and the second target position area information. The overlapping area is a three-dimensional cube. The overlapping area information is the side length X2 in the x-axis direction, the side length Y2 in the y-axis direction, and the side length Z2 in the z-axis direction of the three-dimensional cube.

[0028] S3232. Perform joint discrimination processing on the overlapping area information and the spatial gate information to obtain a third determination result.

[0029] If the third determination result is satisfied, perform fusion and weighting processing on the first target position area information and the second target position area information to obtain the target auxiliary tracking position information.

[0030] If the third determination result is not satisfied, confirm that the target area image information and the target area point cloud data do not contain the same target, and execute S321.

[0031] The expression of the joint discrimination processing is:

[0032] ∣X1 - X2∣≤ a1,

[0033] |Y1 – Y2| ≤ a2,

[0034] |Z1 – Z2| ≤ a3,

[0035] |(X2 - X1)(Y2 - Y1)| ≤ a4,

[0036] |(Z2 - Z1)(Y2 - Y1)| ≤ a5,

[0037] Wherein, a1 to a5 are all preset discrimination thresholds; when the above five inequalities are all satisfied by the overlapping region information, it is confirmed that the third discrimination result is satisfied, otherwise, it is confirmed that the third discrimination result is not satisfied;

[0038] The expression of the fusion weighted processing is:

[0039]

[0040] Wherein, [x1, y1, z1] and [x2, y2, z2] are the center point coordinates of the first target position region information and the second target position region information respectively, is the target auxiliary tracking position information, α1 is the ratio of the area of the overlapping region information to the area of the first target position region information, and α2 is the ratio of the area of the overlapping region information to the area of the second target position region information.

[0041] The validity discrimination includes:

[0042] The tracking filtering process is implemented by using Kalman filtering;

[0043] Obtain the measurement matrix of each target position in the target position sequence information during the corresponding Kalman filtering process;

[0044] Perform a mean value processing on all the obtained measurement matrices to obtain a mean matrix;

[0045] Find the difference between each measurement matrix and the mean matrix to obtain a corresponding difference matrix;

[0046] Perform a mean value processing on the difference matrices of all the measurement matrices to obtain a difference mean matrix;

[0047] Perform a decomposition process on the difference mean matrix to obtain a feature matrix;

[0048] Extract the diagonal elements of the feature matrix to obtain a feature vector;

[0049] Perform a linear fitting process on the elements and element serial number values of the feature vector to obtain a difference discrimination polynomial;

[0050] Calculate the mean of all row vectors of the obtained difference mean matrix;

[0051] Substitute the row vector serial number corresponding to the maximum value in the means of all the row vectors into the difference discrimination polynomial to obtain a first discrimination value;

[0052] Judge whether the first discrimination value is greater than a preset trajectory threshold value. If it is not greater, confirm that the first discrimination result is valid; if it is greater, confirm that the first discrimination result is invalid.

[0053] The consistency discrimination includes:

[0054] Obtain the target auxiliary tracking position information of the same target obtained at several times;

[0055] Express the target auxiliary tracking position information of the same target obtained at the several times as an auxiliary tracking position matrix; the row vector of the auxiliary tracking position matrix is the target auxiliary tracking position information obtained at one time;

[0056] Perform a consistency value calculation process on the auxiliary tracking position matrix to obtain a consistency value;

[0057] The expression of the consistency value calculation process is:

[0058]

[0059] where q ki is the element in the k-th row and the i-th column of the auxiliary tracking position matrix, m and n are respectively the row dimension and the column dimension of the auxiliary tracking position matrix, and h is the consistency value;

[0060] Judge whether the consistency value is greater than a set consistency threshold value. If it is not greater, confirm that the second discrimination result is valid; if it is greater, confirm that the second discrimination result is invalid.

[0061] In the second aspect of the embodiments of the present invention, a target tracking device based on multi-sensor fusion is disclosed, and the device includes:

[0062] A memory storing executable program code;

[0063] A processor coupled to the memory;

[0064] The processor calls the executable program code stored in the memory and executes the above-mentioned target tracking method based on multi-sensor fusion.

[0065] In the third aspect of the embodiments of the present invention, a computer-readable storage medium is disclosed. The computer-readable storage medium stores computer instructions, and when the computer instructions are called by the computer, they are used to execute the above-mentioned target tracking method based on multi-sensor fusion.

[0066] In the fourth aspect of the embodiments of the present invention, an information data processing terminal is disclosed, and the information data processing terminal is used to implement the target tracking method based on multi-sensor fusion described above.

[0067] The beneficial effects of the present invention are as follows:

[0068] The present invention solves the problems that it is difficult to stably track a target vehicle in an unstructured environment by using a single-system target tracking method and how to effectively fuse multiple types of sensors.

[0069] The present invention first uses the camera and lidar to perform preliminary tracking on the target to obtain target auxiliary tracking position information, so as to overcome the tracking problem of the leading vehicle starting statically by the millimeter-wave radar. After obtaining the target auxiliary tracking position information, a consistency discrimination model is established to perform consistency discrimination on the target auxiliary tracking position information obtained at several times, ensuring the accuracy of the preliminary tracking result.

[0070] After the preliminary tracking is completed, the present invention uses the obtained target auxiliary tracking position information as the initial value and uses the millimeter-wave radar to achieve high-precision tracking of the target. To ensure the effectiveness of the tracking result, the present invention introduces an effectiveness discrimination algorithm, selects the measurement matrix of the system as the effectiveness variable, and ensures the accuracy of the fusion tracking result by performing effectiveness discrimination on the measurement matrix. Description of the Drawings

[0071] Figure 1 It is a flowchart of the implementation of the method of the present invention. Detailed Embodiments

[0072] To better understand the content of the present invention, an embodiment is given here.

[0073] Figure 1 It is a flowchart of the implementation of the method of the present invention.

[0074] Aiming at the problems that it is difficult to stably track a target vehicle in an unstructured environment by using a single-system target tracking method and how to effectively fuse multiple types of sensors, the present invention discloses a target tracking method and device based on multi-sensor fusion.

[0075] In the first aspect of the embodiments of the present application, a target tracking method based on multi-sensor fusion is disclosed, including:

[0076] S1. Obtain the information of the target area to be tracked;

[0077] S2. Perform registration processing on the camera, lidar, and millimeter-wave radar;

[0078] S3. Use the camera, lidar, and millimeter-wave radar to perform fusion tracking processing on the target to obtain target position information;

[0079] The registration processing of the camera, lidar, and millimeter-wave radar includes:

[0080] S21. Perform systematic error registration processing on the camera, lidar, and millimeter-wave radar;

[0081] S22. Perform time registration processing and spatial registration processing on the camera, lidar, and millimeter-wave radar;

[0082] S23. Initialize the spatial wavegate information; the spatial wavegate is a cube in space; the spatial wavegate information includes the x-axis distance threshold X1, y-axis distance threshold Y1, and z-axis distance threshold Z1 in a three-dimensional rectangular coordinate system;

[0083] The use of the camera, lidar, and millimeter-wave radar to perform fusion tracking processing on the target to obtain target position information includes:

[0084] S31. Use the camera to perform image acquisition processing on the target area information to be tracked to obtain target area image information; use the lidar to perform detection processing on the target area information to be tracked to obtain target area point cloud data;

[0085] S32. Perform joint monitoring processing on the target area image information and the target area point cloud data to obtain target auxiliary tracking position information;

[0086] S33. Use the millimeter-wave radar to perform tracking filtering processing on the target with the target auxiliary tracking position information as the guiding information to obtain target position sequence information; the target position sequence information includes a number of target positions;

[0087] S34. Perform validity discrimination on the target position sequence information to obtain a first discrimination result; if the first discrimination result is valid, confirm the target position sequence information as the target position information; if the first discrimination result is invalid, execute S32;

[0088] The performance of joint monitoring processing on the target area image information and the target area point cloud data to obtain target auxiliary tracking position information includes:

[0089] S321. Perform image target detection processing on the target area image information to obtain first target position area information;

[0090] S322. Perform target point cloud matching processing on the target area point cloud data to obtain second target position area information;

[0091] S323. Perform spatial gating detection on the first target position area information and the second target position area information to obtain target assisted tracking position information;

[0092] S324. Perform consistency discrimination on the target assisted tracking position information obtained at several times to obtain a second discrimination result; if the second discrimination result is valid, execute S33; if the second discrimination result is invalid, execute S31;

[0093] The performing spatial gating detection on the first target position area information and the second target position area information to obtain target assisted tracking position information includes:

[0094] S3231. Obtain the overlapping area information of the first target position area information and the second target position area information; the overlapping area is a three-dimensional cube; the overlapping area information is the side length X2 in the x-axis direction, the side length Y2 in the y-axis direction, and the side length Z2 in the z-axis direction of the three-dimensional cube;

[0095] S3232. Perform joint discrimination processing on the overlapping area information and the spatial gating information to obtain a third discrimination result;

[0096] If the third discrimination result is satisfied, perform fusion weighted processing on the first target position area information and the second target position area information to obtain target assisted tracking position information;

[0097] If the third discrimination result is not satisfied, confirm that the target area image information and the target area point cloud data do not contain the same target, and execute S321;

[0098] The expression of the joint discrimination processing is:

[0099] ∣X1 - X2∣≤ a1,

[0100] ∣Y1 – Y2∣≤ a2,

[0101] ∣Z1 – Z2∣≤ a3,

[0102] ∣(X2 - X1)(Y2 - Y1)∣≤ a4,

[0103] ∣(Z2 - Z1)(Y2 - Y1)∣≤ a5,

[0104] where a1 to a5 are all preset discrimination thresholds, and the values can be 0.6, 0.5, 0.2, 0.4, and 0.8. When the overlapping area information all satisfies the above five inequalities, confirm that the third discrimination result is satisfied; otherwise, confirm that the third discrimination result is not satisfied.

[0105] The expression of the fusion weighted processing is:

[0106]

[0107] Among them, [x1, y1, z1] and [x2, y2, z2] are the center point coordinates of the first target position area information and the center point coordinates of the second target position area information respectively, is the target auxiliary tracking position information, α1 is the ratio of the area of the overlapping area information to the area of the first target position area information, and α2 is the ratio of the area of the overlapping area information to the area of the second target position area information;

[0108] The validity discrimination includes:

[0109] The tracking filtering process is implemented by using Kalman filtering;

[0110] Obtain the measurement matrix of each target position in the target position sequence information during the corresponding Kalman filtering process;

[0111] Perform a mean value processing on all the obtained measurement matrices to obtain a mean matrix;

[0112] Find the difference between each measurement matrix and the mean matrix to obtain a corresponding difference matrix;

[0113] Perform a mean value processing on the difference matrices of all the measurement matrices to obtain a difference mean matrix;

[0114] Perform a decomposition processing on the difference mean matrix to obtain a feature matrix;

[0115] Extract the diagonal elements of the feature matrix to obtain a feature vector;

[0116] Perform a linear fitting process on the elements and element serial number values of the feature vector to obtain a difference discrimination polynomial;

[0117] Calculate the mean value of all row vectors of the difference mean matrix;

[0118] Substitute the row vector serial number corresponding to the maximum value in the mean value of all the row vectors into the difference discrimination polynomial to obtain a first discrimination value;

[0119] Judge whether the first discrimination value is greater than a preset trajectory threshold value. If it is not greater, confirm that the first discrimination result is valid; if it is greater, confirm that the first discrimination result is invalid.

[0120] The consistency discrimination includes:

[0121] Obtain the target auxiliary tracking position information of the same target obtained at several times;

[0122] The target auxiliary tracking position information of the same target obtained at the several times is expressed as an auxiliary tracking position matrix; the row vector of the auxiliary tracking position matrix is the target auxiliary tracking position information obtained at one time.

[0123] Perform a consistent value calculation process on the auxiliary tracking position matrix to obtain a consistent value.

[0124] The expression of the consistent value calculation process is:

[0125]

[0126] where q ki is the element in the k-th row and the i-th column of the auxiliary tracking position matrix, m and n are respectively the row dimension and the column dimension of the auxiliary tracking position matrix, and h is the consistent value.

[0127] Judge whether the consistent value is greater than a set consistent threshold. If it is not greater, confirm that the second discrimination result is valid; if it is greater, confirm that the second discrimination result is invalid.

[0128] Using the target auxiliary tracking position information as the guiding information and using a millimeter-wave radar to perform tracking processing on the target means using the target auxiliary tracking position information as the initial position information of the target in the Kalman filter.

[0129] The linear fitting process uses the characteristic vector element serial number value Ix as the known independent variable and the characteristic vector element value as the known dependent variable, constructs a curve to be approximated using the known independent variable and the known dependent variable, and performs curve fitting on the curve to be approximated using the function approximation method to obtain a difference discrimination polynomial f(Ix).

[0130] Using the function approximation method to perform curve fitting on the curve to be approximated can adopt the best uniform linear approximation method. The difference discrimination polynomial f(Ix), its expression is:

[0131] f(Ix) = α P1 (Ix) P1 + α P1-1 (Ix) P1-1 + … + α2(Ix) 2 + α1(Ix) + α0,

[0132] where P1 is the order of the difference discrimination polynomial f(Ix), and α0, α1, α2, …, α P1 are the coefficients of the difference discrimination polynomial f(Ix);

[0133] The characteristic vector is expressed as I a , I a = [λ1, λ2, …, λ N1, N1 is the number of elements included in the feature vector;

[0134] For the decomposition process, its calculation expression is:

[0135] Y = UAV,

[0136] where U is the left decomposition matrix, A is the feature matrix, V is the right decomposition matrix, both U and V are orthogonal matrices, A is a diagonal matrix, and Y is the difference mean matrix;

[0137] For the element in the i-th row and j-th column of the mean matrix obtained by the mean calculation process, it is the mean of the elements in the i-th row and j-th column of all measurement matrices;

[0138] For the system error registration process, it is to separately obtain the measurement system errors of the camera, lidar, and millimeter-wave radar, and use the measurement system errors of each measurement device to calibrate the errors of the collected data.

[0139] For the time registration process, it is to unify the measurement data of different measurement devices to the same time reference; for the time registration process, interpolation / extrapolation method, Lagrange three-point interpolation method, etc. can be used;

[0140] For the tracking and filtering process, it is to use the millimeter-wave radar to detect the target to obtain the target point trajectory, and use the Kalman filtering algorithm to process the target point trajectory.

[0141] For the spatial registration process, it is to perform coordinate transformation on the measurement data of different measurement devices and unify them to the same coordinate system.

[0142] For the image target detection process, it can be based on the template image of the target to perform template matching on the target area image information to obtain the target position area information and target position center information; for the template matching process, it can be the MAD algorithm or the SSD algorithm.

[0143] For the target point cloud matching process, the point cloud feature extraction algorithm in the lidar or the deep learning algorithm for 3D target detection of the point cloud can be used; specifically, the point cloud template data of the target can be obtained first, and then the global feature extraction algorithm based on the three-dimensional Hough transform is collected to perform matching on the point cloud template data to obtain the second target position area information.

[0144] The target tracked in this method can be a vehicle target;

[0145] The camera, lidar, and millimeter-wave radar are installed on the same vehicle platform.

[0146] In the second aspect of the embodiments of the present invention, a target tracking device based on multi-sensor fusion is disclosed. The device includes:

[0147] A memory storing executable program code;

[0148] A processor coupled to the memory;

[0149] The processor calls the executable program code stored in the memory and executes the target tracking method based on multi-sensor fusion.

[0150] In a third aspect of the embodiments of the present invention, a computer-readable storage medium is disclosed. The computer-readable storage medium stores computer instructions, which are used to execute the target tracking method based on multi-sensor fusion when called by a computer.

[0151] In a fourth aspect of the embodiments of the present invention, an information data processing terminal is disclosed. The information data processing terminal is used to implement the target tracking method based on multi-sensor fusion.

[0152] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A target tracking method based on multi-sensor fusion, characterized in that Including: S1. Obtain the target area information to be tracked; S2. Perform registration processing on the camera, lidar, and millimeter-wave radar; S3. Use the camera, lidar, and millimeter-wave radar to perform fusion tracking processing on the target to obtain the target position information.

2. The target tracking method based on multi-sensor fusion according to claim 1, characterized in that, The performing registration processing on the camera, lidar, and millimeter-wave radar includes: S21. Perform system error registration processing on the camera, lidar, and millimeter-wave radar; S22. Perform time registration processing and space registration processing on the camera, lidar, and millimeter-wave radar; S23. Initialize the space wavegate information; the space wavegate is a cube in space; the space wavegate information includes the x-axis distance threshold X1, y-axis distance threshold Y1, and z-axis distance threshold Z1 in the three-dimensional rectangular coordinate system.

3. The target tracking method based on multi-sensor fusion according to claim 2, characterized in that, The using the camera, lidar, and millimeter-wave radar to perform fusion tracking processing on the target to obtain the target position information includes: S31. Use the camera to perform image acquisition processing on the target area information to be tracked to obtain the target area image information; use the lidar to perform detection processing on the target area information to be tracked to obtain the target area point cloud data; S32. Perform joint monitoring processing on the target area image information and the target area point cloud data to obtain the target auxiliary tracking position information; S33. Use the millimeter-wave radar to perform tracking filtering processing on the target with the target auxiliary tracking position information as the guiding information to obtain the target position sequence information; the target position sequence information includes several target positions; S34. Perform validity discrimination on the target position sequence information to obtain a first discrimination result; if the first discrimination result is valid, use the target position sequence information as the target position information; if the first discrimination result is invalid, execute S32.

4. The target tracking method based on multi-sensor fusion according to claim 3, wherein, The performing joint monitoring processing on the target area image information and the target area point cloud data to obtain the target auxiliary tracking position information includes: S321. Perform image target detection processing on the target area image information to obtain the first target position area information; S322. Perform target point cloud matching processing on the target area point cloud data to obtain the second target position area information; S323. Perform space wavegate detection on the first target position area information and the second target position area information to obtain the target auxiliary tracking position information; S324. Perform consistency discrimination on the target auxiliary tracking position information obtained at several times to obtain a second discrimination result; if the second discrimination result is valid, execute S33; if the second discrimination result is invalid, execute S31.

5. The target tracking method based on multi-sensor fusion according to claim 4, characterized in that The performing space wavegate detection on the first target position area information and the second target position area information to obtain the target auxiliary tracking position information includes: S3231. Obtain the overlapping area information of the first target position area information and the second target position area information; the overlapping area is a three-dimensional cube; the overlapping area information is the side length X2 in the x-axis direction, side length Y2 in the y-axis direction, and side length Z2 in the z-axis direction of the three-dimensional cube; S3232, perform joint discrimination processing on the overlapping region information and the spatial wavegate information to obtain a third discrimination result; If the third discrimination result is satisfied, perform fusion weighting processing on the first target position region information and the second target position region information to obtain target auxiliary tracking position information; If the third discrimination result is not satisfied, confirm that the same target is not included in the target region image information and the target region point cloud data, and execute S321; The expression of the joint discrimination processing is: ∣X1 - X2∣≤a1, ∣Y1 – Y2∣≤a2, ∣Z1 – Z2∣≤a3, ∣(X2 - X1)(Y2 - Y1)∣≤a4, ∣(Z2 - Z1)(Y2 - Y1)∣≤a5, where a1 to a5 are all preset discrimination thresholds; when all the overlapping region information satisfies the above five inequalities, confirm that the third discrimination result is satisfied, otherwise, confirm that the third discrimination result is not satisfied; The expression of the fusion weighting processing is: Wherein, [x1, y1, z1] and [x2, y2, z2] are the center point coordinates of the first target position area information and the center point coordinates of the second target position area information respectively, is the target auxiliary tracking position information, α1 is the ratio of the area of the overlapping area information to the area of the first target position area information, and α2 is the ratio of the area of the overlapping area information to the area of the second target position area information.

6. The object tracking method based on multi-sensor fusion according to claim 4, characterized in that The validity discrimination includes: The tracking filtering processing is implemented by using Kalman filtering; Obtain the measurement matrix of each target position in the target position sequence information during the corresponding Kalman filtering processing; Perform mean value processing on all the obtained measurement matrices to obtain a mean matrix; Find the difference between each measurement matrix and the mean matrix to obtain a corresponding difference matrix; Perform mean value processing on the difference matrices of all the measurement matrices to obtain a difference mean matrix; Perform decomposition processing on the difference mean matrix to obtain a feature matrix; Extract the diagonal elements of the feature matrix to obtain a feature vector; Perform linear fitting processing on the elements and element serial number values of the feature vector to obtain a difference discrimination polynomial; Calculate the mean value of all row vectors of the difference mean matrix; Substitute the row vector serial number corresponding to the maximum value in the mean values of all the row vectors into the difference discrimination polynomial to obtain a first discrimination value; Judge whether the first discrimination value is greater than a preset trajectory threshold value. If it is not greater, confirm that the first discrimination result is valid; if it is greater, confirm that the first discrimination result is invalid.

7. The target tracking method based on multi-sensor fusion according to claim 4, characterized in that, The consistency discrimination includes: Obtain the target auxiliary tracking position information of the same target at several times; Represent the target auxiliary tracking position information of the same target obtained at the several times as an auxiliary tracking position matrix; the row vector of the auxiliary tracking position matrix is the target auxiliary tracking position information obtained at one time; Perform consistent value calculation processing on the auxiliary tracking position matrix to obtain a consistent value; The expression of the consistent value calculation processing is: where q ki is the element at the k-th row and the i-th column of the auxiliary tracking position matrix, m and n are the row dimension and the column dimension of the auxiliary tracking position matrix respectively, and h is the consistency value; Judge whether the consistent value is greater than a set consistency threshold value. If it is not greater, confirm that the second discrimination result is valid; if it is greater, confirm that the second discrimination result is invalid.

8. An object tracking device based on multi-sensor fusion, characterized in that, The device includes: A memory storing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory and executes the multi-sensor fusion based target tracking method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions which, when called by a computer, are used to execute the multi-sensor fusion-based target tracking method according to any one of claims 1 to 7.

10. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the multi-sensor fusion-based target tracking method according to any one of claims 1 to 7.

Images