A space-based distributed multi-sensor multi-target tracking and positioning method

Abstract

The application discloses a space-based distributed multi-sensor multi-target tracking and positioning method. The method first acquires a gray image of each sensor, extracts the coordinates of the target in the image, matches the serial number of the extracted target, and takes the first frame coordinates of the target and the image sequence as the tracking input. The tracking process is based on the discriminant correlation filter, adds the energy concentration threshold of small target features, constructs the double-constraint trajectory association model of the target distance and speed similarity between the continuous frames, and simultaneously uses the Kalman filtering algorithm to predict the target position. An adaptive covariance cross-fusion positioning algorithm is designed between multiple sensors, the geometric relationship between multiple detection satellites and the target is established, the sensor observation parameters are used as the basis for adjusting the fusion weight, and the three-dimensional position of the target is calculated. The method can significantly improve the target tracking accuracy and suppress the positioning deviation problem caused by the instability of part of the sensors, can adapt to the dynamic changes of the distributed detection scene, and ensures the target positioning accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a space-based distributed multi-sensor multi-target tracking and positioning method. BACKGROUND

[0002] Low-orbit infrared sensors are important means for space exploration. However, the detection range of a single satellite is limited, and there are difficulties in the task of continuous target positioning and indication. Multi-satellite joint observation is a more effective choice. Cooperative detection of target groups by distributed multi-infrared sensors can enhance the long-time and wide-range monitoring capability of the target and improve the robustness and anti-interference capability of the system. Multi-sensor fusion positioning can not only combine the advantages of different sensors to improve the positioning accuracy, but also effectively reduce the influence of single sensor error on the positioning result, and enhance the robustness and reliability of the positioning algorithm. However, in the multi-satellite cooperative positioning scene, due to the differences in the basic properties, system errors, and detection conditions of different infrared sensors, the noise levels of the measurement data are different, and there is unknown correlation between the sensors, which brings errors and instability to the fusion positioning.

[0003] The purpose of multi-sensor information fusion is to improve the tracking and positioning accuracy of the detected target, but how to handle the unknown correlation between the sensor information is a challenging problem. Researchers have proposed a simple convex combination fusion algorithm to weight each decision based on the reliability of the detector and obtain a global decision by setting a threshold. This algorithm can provide optimal fusion estimation when the estimation errors of the sensors are independent of each other. The Bar-Shalom-Campo fusion algorithm further considers the correlation between sensor estimates due to common process noise, and can provide optimal estimation in the sense of maximum likelihood. However, this algorithm requires the correlation information of the sensors, which is usually difficult to obtain in practice. To solve this problem, researchers have proposed a Covariance Intersection (CI) fusion algorithm. This algorithm only relies on the known mean and covariance for sensor measurement fusion, allowing for optimal fusion of different levels of estimation while avoiding the independence assumption required by the Kalman filter, but the convergence speed may be reduced. Researchers have proposed a fast covariance intersection filter algorithm that uses a suboptimal non-iterative algorithm to replace the optimal nonlinear cost function minimization iteration in CI fusion, reducing the computational complexity, but the algorithm is only applicable to Gaussian distributed inputs. Researchers have further extended the multi-sensor CI fusion estimation and proposed a Batch Covariance Intersection (BCI) fusion algorithm based on multi-dimensional nonlinear cost function optimization, but the batch algorithm has a large amount of calculation and requires more computational resources. SUMMARY:

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present application provides a space-based distributed multi-sensor multi-target tracking and positioning method. The method is a distributed multi-sensor common tracking and positioning method involving multiple targets, and the number of sensors is not less than two. The method can significantly improve the target tracking accuracy and suppress the positioning deviation problem caused by the instability of part of the sensors, and can adapt to the dynamic changes of the distributed detection scene to ensure the accuracy of target positioning.

[0005] The above-mentioned object of the present application is achieved by the following technical solutions:

[0006] A space-based distributed multi-sensor multi-target tracking and positioning method, comprising the following steps:

[0007] Step S1, each sensor continuously acquires a gray-scale video image;

[0008] Step S2, sub-pixel accuracy detection and extraction of the target in the image are performed to obtain a candidate target sequence, including the centroid data and observation time of the target;

[0009] Step S3, based on the candidate target sequence of the first frame of the image sequence, subsequent tracking is carried out, the candidate target sequence is taken as input, a space-time regularization discriminative correlation filter is set as a target tracker, and each candidate target is assigned a matching serial number and a tracker;

[0010] Step S4, for the target region estimated by the space-time regularization discriminative correlation filter, the energy concentration degree of each target region is calculated; the upper and lower limits [Ethl, Ethh] of the energy concentration degree threshold E are set; for the target region meeting the threshold range, the target centroid position (x c ,y c ) is calculated, and the target that does not meet the threshold range is removed as a noise point;

[0011] Step S5, trajectory association is performed on adjacent frame targets, the minimum Euclidean distance and speed similarity between each target in adjacent two frames are calculated to associate the trajectories of the same target;

[0012] Step S6, time registration is performed on the candidate target sequences of multiple sensors, and the registration method is to judge whether there is a target with an observation time difference less than a set threshold, and the candidate target sequences of multiple sensors are encapsulated into observation data groups according to time slices;

[0013] Step S7, the target position is estimated according to the candidate target sequences of multiple sensors in the observation data group;

[0014] Step S8, the target position is accurately estimated.

[0015] Further, in step S1, the sensor adopts a surface array type image detector, and the output frame frequency is not less than 1 frame per second.

[0016] Further, in step S4, the calculation formula of the energy concentration degree is as follows:

[0017]

[0018] wherein P(u, v) represents the maximum value of the pixel of the point target area, and the denominator represents the sum of the pixel gray scale of the point target area.

[0019] Further, in step S5, the calculation formula of the Euclidean distance is as follows:

[0020]

[0021] wherein (x t ,y t ) and (x t+1 ,y t+1 ) respectively represent the coordinate data of the target on the tth frame and the t+1th frame image;

[0022] The speed vector change angle θ of the target in the former and latter two frames is introduced to represent the speed similarity, and the calculation process of θ is as follows:

[0023]

[0024] wherein v t-1 and v t respectively represent the speed vector of the target in the t-1th frame and the tth frame; the similarity score S under the two types of constraints is calculated, and the target with similar scores of the former and latter frames is taken as the criterion for correlation:

[0025]

[0026] wherein a1 and a2 are weight parameters of the two types of constraints, and the parameters can be set.

[0027] Further, the method of step S7 is as follows:

[0028] Supposing that the target position is T=(x T ,y T ,z T ), the spatial position of the sensor n is O n =(x n ,y n ,z n ), and the corresponding detection line of sight is LOS n ; the target spatial position expressed based on the sensor n is as follows:

[0029] T=O n +λn LOS n

[0030] where λ n is the detection distance of the target relative to sensor n;

[0031] For sensor n, the distance of the target to the detection line of sight can be calculated by the following formula:

[0032] λ n =‖(T-O n )×LOS n ‖

[0033] The target position T is solved by least squares, and the objective function of the least squares is defined as follows:

[0034]

[0035] Solve by matrix form.

[0036] Further, the method of step S8 is as follows:

[0037] That is, λ n is the detection distance of the target relative to sensor n, and the three-dimensional position of the target calculated under the detection condition of the sensor is Initialize the estimated error covariance P n of the sensor;

[0038] Consider using sensor observation parameters such as detection distance, detection time, and camera resolution characteristics to optimize the weight ω n is the weight coefficient of sensor n, satisfying 0≤ω n ≤1 and The weight coefficient is calculated as follows:

[0039]

[0040] Where b1, b2, b3 are observation parameter weights assigned by humans; T n is the observation time length of sensor n; GSD is the ground resolution, and the GSD parameter reflects the imaging quality of the camera, which is related to the pixel size, focal length and detection distance; λ n is the detection distance of the target relative to sensor n;

[0041] Finally, the global estimate and covariance after fusion are calculated, and the calculation formula is as follows:

[0042]

[0043] Where, is the state estimate value after fusion, and P CIFor the fused estimated error variance.

[0044] Further, in step S1, the sensor is an infrared area detector, a visible area detector or an ultraviolet area detector.

[0045] Further, in step S2, the sub-pixel precision detection and extraction is obtained by target detection on the image, super-resolution processing on the target neighborhood and centroid weighted calculation of the target centroid position.

[0046] Further, in step S3, the space-time regularization discriminative correlation filter refers to a discriminative correlation filter with a regularization constraint term, and the trajectory is synchronously predicted by adding an extended Kalman filter algorithm.

[0047] Compared with the prior art, the present application has the following beneficial effects:

[0048] 1. The energy concentration threshold is set according to the characteristics of small targets after updating the discriminative correlation filter response, which is used to suppress background noise.

[0049] 2. The minimum Euclidean distance and the speed similarity between the target continuous frames are used to associate the target trajectories, which solves the association error caused by the close distance or trajectory intersection between multiple targets.

[0050] 3. The adaptive weight covariance cross-fusion algorithm is used to fuse the information of multiple sensors, which effectively suppresses the problem of low positioning accuracy caused by the instability of part of the sensors or tracking error. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The present application is a flow chart for the implementation.

[0052] Figure 2 The present application is a gray simulation image containing multiple targets.

[0053] Figure 3 The present application is a multi-star positioning geometric relationship diagram.

[0054] Figure 4 The present application is a trajectory diagram of a simulated multi-sensor cooperative detection scene.

[0055] Figure 5 The present application is a detection time period diagram of a simulated multi-sensor cooperative detection scene.

[0056] Figure 6 The present application provides a tracking and positioning result of a simulated multi-sensor tracking scene. DETAILED DESCRIPTION

[0057] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described in detail. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of the present application.

[0058] The present application provides a space-based distributed multi-sensor multi-target tracking and positioning method, comprising the following steps:

[0059] Step S1, each sensor continuously acquires a gray video image;

[0060] The sensor adopts a surface array type image detector, and the output frame frequency is not less than 1 frame per second. Specifically, the sensor is an infrared surface array detector, a visible surface array detector or an ultraviolet surface array detector.

[0061] Step S2, sub-pixel precision detection and extraction of the target in the image are performed to obtain a candidate target sequence, including the centroid data and observation time of the target.

[0062] The sub-pixel precision detection and extraction is obtained by performing target detection on the image, performing super-resolution processing on the target neighborhood, and then performing centroid weighted calculation to obtain the target centroid position.

[0063] Step S3, based on the candidate target sequence of the first frame of the image sequence, subsequent tracking is carried out, the candidate target sequence is taken as input, a space-time regularization discriminative correlation filter is set as a target tracker, and each candidate target is assigned a matching serial number and a tracker.

[0064] The space-time regularization discriminative correlation filter refers to a discriminative correlation filter with a regularization constraint term, and the trajectory is synchronously predicted by adding an extended Kalman filter algorithm.

[0065] Step S4, for the target region estimated by the space-time regularization discriminative correlation filter, the energy concentration degree of each target region is calculated; the upper and lower limits [Ethl, Ethh] of the energy concentration degree threshold E are set; for the target region meeting the threshold range, the target centroid position (x c ,y c ) is calculated, and the target region not meeting the threshold range is removed as a noise point.

[0066] The calculation formula of the energy concentration degree is as follows:

[0067]

[0068] Wherein P(u, v) represents the maximum value of the pixel of the point target region, and the denominator represents the sum of the pixel gray scale of the point target region.

[0069] Step S5, adjacent frame targets are associated with trajectories, the minimum Euclidean distance and velocity similarity between each target in adjacent two frames are calculated to associate the trajectories of the same target;

[0070] Euclidean distance The calculation formula is as follows:

[0071]

[0072] Wherein, (x t ,y t ) and (x t+1 ,y t+1 ) represent the coordinate data of the target on the tthframe and t+1thframe image respectively;

[0073] The velocity vector change angle θ of the target in the front and rear two frames is introduced to represent the velocity similarity, and the calculation process of θ is as follows:

[0074]

[0075] Wherein, v t-1 and v t represent the velocity vector of the target in the t-1thframe and the tthframe respectively; the similarity score S under two types of constraints is calculated, and the target with similar scores of the front and rear frames is taken as the association criterion:

[0076]

[0077] Wherein a1 and a2 are weight parameters of two types of constraints, and the parameters can be set.

[0078] Step S6, time registration is performed on the candidate target sequence of multiple sensors, and the registration method is to judge whether there is a target with an observation time difference less than a set threshold, and the candidate target sequence of multiple sensors is packaged into an observation data group according to a time slice;

[0079] Step S7, the target position is estimated according to the candidate target sequence of multiple sensors in the observation data group;

[0080] Let the target position be T=(x T ,y T ,z T ), the spatial position of sensor n is O n =(x n ,y n ,z n ), and the corresponding detection line of sight is LOS n ; the target spatial position expressed based on the sensor n is as follows:

[0081] T=O n +λ n LOS n

[0082] where λ n is the detection distance of the target relative to sensor n;

[0083] For sensor n, the distance of the target to the line of sight can be calculated by the following formula:

[0084] λ n =‖(T-O n )×LOS n ‖

[0085] The target position T is solved by least squares, and the objective function of the least squares is defined as follows:

[0086]

[0087] Solve by matrix form.

[0088] Step S8, accurately estimate the target position.

[0089] That is, λ n is the detection distance of the target relative to sensor n, and the three-dimensional position of the target calculated under the detection condition of the sensor is Initialize the estimation error covariance P n of the sensor;

[0090] Consider using sensor observation parameters such as detection distance, detection time, and camera resolution characteristics to optimize the weight ω n is the weight coefficient of sensor n, satisfying 0≤ω n ≤1 and The weight coefficient is calculated as follows:

[0091]

[0092] Where b1, b2, b3 are observation parameter weights assigned by humans; T n is the observation time length of sensor n; the Ground Sampling Distance (GSD) parameter reflects the imaging quality of the camera, and its numerical size is related to the pixel size, focal length and detection distance; λ n is the detection distance of the target relative to sensor n;

[0093] Finally, the global estimate and covariance after fusion are calculated, and the calculation formula is as follows:

[0094]

[0095] Where, is the state estimate value after fusion, and P CIis the fused estimated error variance.

[0096] In order to make the objects, technical solutions and advantages of the present application clearer, the following further describes the present application in detail with specific examples and in conjunction with the drawings:

[0097] The method verification is performed on a simulation space-based distributed infrared multi-observer multi-target scene, and the process of observing the same target cluster by four infrared sensors carried on four satellites is used. The scene includes a motion process of a central target releasing 10 sub-targets, the detection distance of each sensor is 4000km-5000km, and the total time length of the target is 1918 seconds, and the frame replacement frequency is 5Hz;

[0098] The energy concentration threshold Ethl is set to 0.3, the Ethh threshold is set to 0.6, the weights a1 and a2 of the two types of constraints are set to 0.6 and 0.4 respectively, and the observation parameter weights b1, b2 and b3 are set to 0.5, 0.3 and 0.2 respectively;

[0099] In the multi-satellite joint detection scene, the optimal sub-mode allocation (OSPA) and the distance error multi-target fusion positioning algorithm are used as the evaluation index, and the classical covariance fusion (CI) method, the generalized covariance fusion method, the sequential covariance fusion (SCI) method and the weighted arithmetic average (WAA) method are used as the comparative algorithm;

[0100] The multi-target positioning precision of the method is controlled within 30m, the average OSPA precision is less than 44m, and the processing speed can be controlled within 0.4ms / frame under the condition that four sensors exist at the same time.

[0101] Figure 1 It is a specific method flowchart for realizing the present application. Firstly, the gray scale image of each sensor is acquired, and the image coordinates of the target in each sensor are extracted; a matching serial number and a tracker are assigned to each candidate target extracted, and the coordinates of the target and the image sequence are taken as the tracking input. The tracking process takes the discriminant correlation filter as the tracker, sets the energy concentration threshold based on the characteristics of small targets, constructs a target distance and speed similarity double-constraint track association model between consecutive frames, and adds a Kalman filter algorithm to predict the target position synchronously; an adaptive covariance cross-fusion positioning algorithm is designed between multiple sensors, the geometric relationship between multiple detection satellites and the target is established, the sensor observation parameters are used as the basis for adjusting the fusion weight, and the three-dimensional position of the target is calculated. The method can significantly improve the target tracking precision and suppress the positioning deviation problem caused by the instability of part of the sensors, can adapt to the dynamic changes of the distributed detection scene, and ensures the target positioning accuracy.

[0102] Figure 2is the simulation image of sensor image detecting multi-target scene. Wherein, the left image (a) in the simulation trajectory intersection scene, the right image (b) in the simulation multi-target proximity scene, both represent the image examples taken by different sensors.

[0103] Figure 3 is a schematic diagram of the geometric relationship between targets and sensors in the multi-star cooperative detection scene, taking double stars as an example.

[0104] Figure 4 is a schematic diagram of the trajectory between each sensor and the target when multiple sensors cooperatively detect the same cluster target.

[0105] Figure 5 is a schematic diagram of the visible period of each sensor to the target when multiple sensors cooperatively detect the same cluster target. Figure 6 is the tracking and positioning result of simulating a multi-sensor tracking scene.

Claims

1. A space-based distributed multi-sensor multi-target tracking and localization method, characterized in that: The method comprises the following steps: Step S1, each sensor continuously acquires a gray video image; Step S2, sub-pixel precision detection and extraction are performed on the target in the image to obtain a candidate target sequence, including target centroid data and observation time; Step S3, based on the candidate target sequence of the first frame of the image sequence, subsequent tracking is carried out, the candidate target sequence is taken as input, a space-time regularization discriminative correlation filter is set as a target tracker, and each candidate target is assigned a matching serial number and a tracker; Step S4, for the target region estimated by the space-time regularization discriminative correlation filter, the energy concentration degree of each target region is calculated; Setting an energy concentration threshold upper and lower limits [Ethl, Ethh]; for target regions meeting the threshold range, calculating target centroid positions target regions not meeting the threshold range are removed as noise points; Step S5, trajectory correlation is performed on adjacent frame targets, the minimum Euclidean distance and speed similarity between each target in adjacent two frames are calculated, and the trajectories of the same target are associated; Step S6, time registration is performed on the candidate target sequences of multiple sensors, the registration method is to judge whether there is a target with an observation time difference less than a set threshold, and the candidate target sequences of multiple sensors are encapsulated into observation data groups according to time slices; Step S7, the target position is estimated according to the candidate target sequences of multiple sensors in the observation data group; Step S8, the target position is accurately estimated, and the method is as follows: is the detection range of the target relative to the sensor , the three-dimensional position of the target calculated under the detection condition of the sensor is , the estimated error covariance of the sensor is initialized ; The weight is optimized by using sensor observation parameters such as a detection distance, a detection time length, and a resolution characteristic of a camera of the sensor, is a weight coefficient of the sensor , satisfies and ; The weight coefficient is calculated as follows: wherein , , is the observation parameter weight assigned by human; is the observation duration of the sensor ; GSD is the ground resolution, and the GSD parameter reflects the imaging quality of the camera, and the numerical value is related to the pixel size, focal length and detection distance; is the detection distance of the target relative to the sensor ; Finally, the global estimation and covariance after fusion are calculated, and the calculation formula is as follows: 。 2. The method according to claim 1, wherein: In step S1, the sensor adopts a surface array image detector, and the output frame frequency is not less than 1 frame per second.

3. The method of claim 2, wherein: In step S4, the calculation formula of the energy concentration degree is as follows: wherein represents the maximum value of the pixels of the point target region, and the denominator represents the pixel gray value sum of the point target region.

4. The method of claim 3, wherein: In step S5, the Euclidean distance is calculated according to the following formula: wherein, and represent coordinate data of the target on the frame and on the frame image, respectively; frame and on frame image, respectively; The angle between the velocity vector changes of the two frames of target before and after the introduction to represent the velocity similarity, The calculation process is as follows: where, and denote the velocity vector of the target in the frame and the frame, respectively; the similarity score under both types of constraints is computed and the target with the similar score between the previous and the current frame is taken as the criterion for association. wherein and are weight parameters for the two types of constraints, which can be set.

5. The method of claim 4, wherein: The method of step S7 is as follows: The target position is set as , the spatial position of the sensor , , and the corresponding detection line of sight is ; the target spatial position expressed based on the sensor is as follows: wherein, is the detection distance of the target relative to the sensor ; For the sensor The distance of the target to the line of sight of the probe can be calculated by the following equation: The target position T is solved by using least squares, and the objective function of the least squares method is defined as follows: The solution is obtained in the form of a matrix.

6. The method of claim 1-5, wherein: In step S1, the sensor is an infrared surface array detector, a visible surface array detector or an ultraviolet surface array detector.

7. The method of claim 1-5, wherein: In step S2, the sub-pixel precision detection and extraction is obtained by performing target detection on the image, performing super-resolution processing on the target neighborhood, and then performing centroid weighted calculation to obtain the target centroid position.

8. The method of claim 1-5, wherein: In step S3, the space-time regularization discriminative correlation filter refers to a discriminative correlation filter with a regularization constraint term, and the trajectory is synchronously predicted by adding an extended Kalman filter algorithm.

Images