A multi-objective real-time rendezvous measurement method based on spatio-temporal optimal correlation matching

Abstract

The present invention provides a multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching to solve the problem that the existing multi-target real-time intersection measurement method cannot balance high precision and real-time performance. The present invention fully considers the system angle measurement error and station error, sets an inter-frame adaptive correlation threshold, establishes an optimization model for multi-target spatio-temporal inter-frame correlation, and uses the motion information of any multi-frame targets for multi-target image point correlation, improving the accuracy of multi-target inter-frame correlation; at the same time, it also sets an inter-station adaptive matching threshold and establishes an optimization model for multi-target spatio-temporal inter-station matching, ensuring the high precision and real-time performance of multi-target intersection measurement. The present invention first matches two-dimensional target image points and then calculates the three-dimensional trajectory of the target using the matching results. This method has a small amount of calculation, realizes the processing of abnormal situations where input data has missing frames, improves the robustness of the algorithm, and has few constraint conditions for algorithm input, with strong algorithm universality.

Description

Technical Field

[0001] The present invention belongs to the field of air target observation, and particularly relates to a multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching. Background Technique

[0002] Using optical theodolites to achieve intersection measurement and obtain spatio-temporal information such as the three-dimensional coordinates, speed, and acceleration of flying targets has been widely used in the field of air target observation. Such an intersection measurement system requires at least two theodolites. Each theodolite measures the azimuth angle and elevation angle of the target respectively. Based on the spatial coordinates of the theodolites, the spatial position of the flying target at a certain moment can be completely determined by using the method of two-station or multi-station intersection. This measurement system has advantages such as high measurement accuracy and the ability to observe multiple targets simultaneously. However, in multi-target intersection measurement, the inter-frame correlation and station-to-station matching of multi-target image points are difficult points, and the accuracy of correlation and matching has a very large impact on the intersection measurement accuracy. Therefore, the inter-frame correlation and station-to-station matching of multi-target image points are the core of multi-target intersection measurement.

[0003] Most current multi-target intersection measurement algorithms are post-intersection processing algorithms. The research focus of post-intersection multi-target intersection processing algorithms mainly concentrates on system accuracy analysis, coordinate transformation, formula derivation of intersection measurement, station-to-station image point matching, etc. Most existing real-time intersection measurement algorithms only calculate single targets, and there are few algorithms for multi-target real-time intersection measurement.

[0004] As Figure 1 shown, the following situations will occur when performing inter-frame correlation and station-to-station matching of multi-target image points: ① When the detector or detection processing unit fails to distinguish all targets, or there are serious occlusion and adhesion phenomena of the targets in the measurement coordinate system; ② There is a one-to-many phenomenon in the measurement data of the measurement station; ③ False targets appear in the complex imaging background; ④ Frame loss occurs during the observation process, all of which bring difficulties to the inter-frame correlation and station-to-station matching of multi-target image points. Currently, commonly used multi-target matching algorithms include the maximum likelihood method, minimum distance method, spectral correlation method, integer programming method, Lagrangian relaxation algorithm, and neural network, etc. However, these algorithms all have limitations in ensuring both intersection accuracy and real-time performance. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem that the existing multi-target real-time intersection measurement method cannot take into account both high accuracy and real-time performance, and provides a multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is:

[0007] A multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching, characterized in that it includes the following steps:

[0008] Step 1: Establish a number of measurement stations in the space to be measured. Each measurement station measures and records the target image point data at each moment in the space. The target image point data includes the imaging time of the target image point, the azimuth angle and the elevation angle of the target image point relative to each measurement station.

[0009] Step 2: Each measurement station performs spatio-temporal optimal inter-frame correlation calculation on the obtained target image point data at each moment to obtain the angular trajectory correlation result of the target image points measured by each measurement station.

[0010] Step 3: According to the angular trajectory correlation results of the target image points measured by each measurement station, perform spatio-temporal optimal inter-station matching to obtain the inter-station target image point matching result of each target image point.

[0011] Step 4: Perform intersection calculation according to the inter-station target image point matching results of each target image point to obtain the coordinates and motion speed of the target in three-dimensional space.

[0012] Further, Step 2 is specifically as follows:

[0013] 2.1. Each measurement station numbers the obtained target image points as 1, 2,..., i,..., N i ;

[0014] 2.2. Each measurement station respectively establishes a set of angular trajectories with successfully associated target image points, associates the target image points with the same number in the first two frames obtained, records them as successfully associated angular trajectories and numbers them as 1, 2,..., j,..., M j , and at the same time saves the azimuth angle vector and elevation angle vector of the angular trajectories in the set according to the target image point data in the angular trajectories.

[0015] 2.3. Each measurement station performs spatio-temporal optimal inter-frame correlation on the target image points obtained at the current moment with the angular trajectories in the set of successfully associated angular trajectories in turn. If the association is successful, assign the number of the angular trajectory successfully associated with the target image point; if the association is not successful, add the target image point as a new angular trajectory to the trajectory set and assign the angular trajectory number M j +c, and M j +c is greater than N i ; c≥1;

[0016] 2.4. According to the set angular trajectory length, fill the azimuth angle vector and elevation angle vector of all angular trajectories in the angular trajectory set in a sliding window manner. If the angular trajectory is successfully associated with a new target image point, fill the azimuth angle and elevation angle of the new target image point in the azimuth angle vector and elevation angle vector of the corresponding angular trajectory in turn; if the angular trajectory is not successfully associated with a new target image point, fill infinity inf in the azimuth angle vector and elevation angle vector of the corresponding angular trajectory.

[0017] 2.5. According to the methods in 2.1 to 2.4, the measurement station performs spatio-temporal optimal inter-frame association on the target image points of all frames obtained, and obtains the angular trajectory association result of the target image points at each moment of each measurement station.

[0018] Further, step 2.3 is specifically as follows:

[0019] 2.3.1. Select two target image points on any angular trajectory in the successfully associated angular trajectory set, and form an angular trajectory with three time points with the target image point at the current moment;

[0020] 2.3.2. Calculate the direction consistency between the selected angular trajectory and the target image point at the current moment;

[0021] Let two target image points on any angular trajectory be target image point 3, whose coordinates are (A 3 , E 3 ) and target image point 4, whose coordinates are (A 4 , E 4 ), and the target image point at the current moment is target image point 5, whose coordinates are (A 5 , E 5 ); the angular trajectory is in sequence: target image point 3, target image point 4 to target image point 5 at the current moment; where, A 3 , A 4 , A 5 represents the azimuth angle of the target image point, and E 3 , E 4 , E 5 represents the elevation angle of the target image point;

[0022] Calculate the flight direction consistency θ between the angular trajectory and the target image point at the current moment:

[0023]

[0024]

[0025]

[0026]

[0027] Among them, represents the square of the distance between target image point 3 and target image point 4, represents the square of the distance between target image point 4 and the target image point 5 at the current moment; (A 5 - A 4 )(A 4 - A 3 ) represents the distance product of the vector and the vector in the azimuth direction, (E5 -E 4 )(E 4 -E 3 ) represents a vector and the vector in the pitch direction, and the product of the distances; represents the sum of these two products; the vector is the vector formed by target image point 3 and target image point 4; the vector is the vector formed by target image point 4 and target image point 5 at the current moment;

[0028] 2.3.3. Calculate the flight direction consistency threshold ε(θ * ) between the selected angular trajectory and the target image point at the current moment:

[0029]

[0030] where ε(AE * ) is the upper error limit of the selected angular trajectory;

[0031] 2.3.4. Calculate the flight speed consistency v between the selected angular trajectory and the target image point at the current moment:

[0032]

[0033] where a is the time difference between target image point 4 and target image point 3, and b is the time difference between target image point 4 and target image point 5 at the current moment;

[0034] 2.3.5. Calculate the flight speed consistency threshold ε(v * ) between the selected angular trajectory and the target image point at the current moment:

[0035]

[0036] 2.3.6. Select two target image points from the selected angular trajectories in a permutation and combination manner, and calculate the flight direction consistency, flight direction consistency threshold, flight speed consistency, and flight speed consistency threshold between the two target image points in the selected angular trajectory and the target image point at the current moment according to the methods in steps 2.3.1 to 2.3.5;

[0037] 2.3.7. Calculate the average flight direction consistency Average flight direction consistency threshold Average flight speed consistency and the average flight speed consistency threshold

[0038] 2.3.8. Calculate the weighted value of the average flight direction - speed consistency between the selected angular trajectory and the target image point at the current moment

[0039]

[0040] where ρ is the associated weight coefficient, 0 ≤ ρ ≤ 1;

[0041] 2.3.9. Calculate the weighted threshold of the average flight direction - speed consistency between the selected angular trajectory and the target image point at the current moment

[0042]

[0043] 2.3.10. Associate each of the target image points obtained at the current moment with the angular trajectories in the angular trajectory set one by one, and obtain the weighted value of the average flight direction - speed consistency of all target image points at the current moment according to the methods in 2.3.1 - 2.3.9 Matrix and weighted threshold of average flight direction - speed consistency Matrix;

[0044] 2.3.11. Establish an optimization model for multi - target inter - frame association;

[0045]

[0046]

[0047] where, is the weighted value of the average flight direction - speed consistency between the i - th target image point and the j - th angular trajectory at the current moment;

[0048] is the weighted threshold of the average flight - speed consistency between the i - th target image point and the j - th angular trajectory;

[0049] q ij is a decision variable of 0 or 1, representing the association relationship between the i - th target image point and the j - th angular trajectory; if the association is successful, then q ij = 1, if the association is not successful, then q ij = 0;

[0050] 2.3.12. Use the Hungarian algorithm to obtain the inter - frame association result. If the association is successful, assign the number of the angular trajectory successfully associated with the target image point; if the association is not successful, add the target image point as a new angular trajectory to the trajectory set and assign the angular trajectory number M j + c, and the value of M j + c is greater than N i ; c ≥ 1.

[0051] Further, step 3 is specifically as follows:

[0052] 3.1. Obtain the angular trajectory association results of the target image points measured by each measurement station according to step 2, as well as the azimuth angles and elevation angles of all target image points of each measurement station, and calculate the angular trajectory unit vectors of the target image points within each measurement station respectively;

[0053] Define the spatial position coordinates of measurement station s 1 as (x 1 , y 1 , z 1 ), and the spatial position coordinates of measurement station s 2 as (x 2 , y 2 , z 2 ). At the current moment, for any target image point o 1 within the field of view of measurement station s 1 , the azimuth angle and elevation angle relative to measurement station s 1 are and For any target image point o 2 within the field of view of measurement station s 2 , the azimuth angle and elevation angle relative to measurement station s 2 are and Then

[0054] At the current moment, the angular trajectory unit vector of target image point o 1 relative to measurement station s 1 is:

[0055]

[0056] At the current moment, the angular trajectory unit vector of target image point o 2 relative to measurement station s 2 is:

[0057]

[0058] 3.2. Let L 1 be the plane formed by target image point o 1 and measurement stations s 1 , s 2 , and let L 2 be the plane formed by target image point o 2 and measurement stations s 1 , s 2 . Calculate the angle β between the plane L1 where target image point o 1 is located and the plane L2 where image point o 2 is located:

[0059]

[0060] 3.3. Calculate the adaptive threshold ε(β * ) of the angle between plane L1 and plane L2:

[0061]

[0062] where ε(x i * ) is the upper limit of the measurement error in the x - direction of the site coordinates, ε(y i * ) is the upper limit of the measurement error in the y - direction of the site coordinates, ε(z i * ) is the upper limit of the measurement error in the z - direction of the site coordinates, ε(A i * ) is the upper limit of the measurement error of the azimuth angle of the target image point relative to the measurement station, and ε(E i * ) is the upper limit of the measurement error of the elevation angle of the target image point relative to the measurement station;

[0063] 3.4. According to the methods in steps 3.1 - 3.3, calculate the angle matrix between the planes where all target image points of the two measurement stations are located at the current moment, and the corresponding adaptive threshold matrix of the angle;

[0064] 3.5. Combine the angle and the adaptive threshold matrix of the angle in the historical Q - frame to calculate the average angle matrix of all target image points of the two measurement stations at the current moment, and the corresponding average adaptive threshold matrix of the angle;

[0065] 3.6. Establish a multi - target spatio - temporal optimal matching model;

[0066]

[0067]

[0068] where, is the average value of the angle between the plane where the k - th target image point of the measurement station s 1 is located at the current moment and the plane where the l - th target image point of the measurement station s 2 is located at the current moment; M and N are the numbers of target image points of the measurement station s 1 and the measurement station s 2 at the current moment respectively;

[0069] is the average adaptive threshold of the angle between the plane where the k - th target image point of the measurement station s 1 is located at the current moment and the plane where the l - th target image point of the measurement station s 2 is located at the current moment;

[0070] p kl is a decision variable of 0 or 1, representing the matching relationship between the k-th target image point of measurement station s 1 and the l-th target image point of measurement station s 2 ; if the matching is successful, then p kl = 1, if the matching is not successful, then p kl = 0.

[0071] 3.7. Use the Hungarian algorithm to obtain the matching results of the inter-station target image points for each target image point.

[0072] Furthermore, in step 2.3.4, the time difference between the adjacent corner trajectory points is 1.

[0073] Furthermore, in step 2.2, the dimensions of both the azimuth vector and the elevation vector are 1×N max , where N max is the number of target image points in the fillable corner trajectory.

[0074] Furthermore, in step 2.2, N max = 3 to 7.

[0075] Compared with the prior art, the beneficial technical effects of the present invention are as follows:

[0076] 1. The multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching provided by the present invention considers the system angle measurement error, proposes an inter-frame adaptive correlation threshold, establishes an optimization model for multi-target spatio-temporal inter-frame correlation, and can use the motion information of any multi-frame targets for multi-target image point correlation, improving the accuracy of multi-target inter-frame correlation.

[0077] 2. The multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching provided by the present invention considers the system angle measurement error and measurement station error, proposes an inter-station adaptive matching threshold, eliminates false targets, and the set threshold does not need to be reset due to the change of the observation distance. Considering the multi-frame motion information of the targets, an optimization model for multi-target spatio-temporal inter-station matching is established, and the Hungarian algorithm with low complexity is used to solve the global optimal matching result, ensuring the high accuracy and real-time performance of multi-target intersection measurement.

[0078] 3. The multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching provided by the present invention has a wider applicability. When the detector or the detection and processing unit fails to resolve all targets, or there are serious occlusion and adhesion phenomena of the targets in the measurement coordinate system, or there is a one-to-many phenomenon in the measurement data of the measurement station, or false targets appear in the complex imaging background, or frame loss occurs during the observation process, high-accuracy matching of multi-targets can still be achieved.

[0079] 4. The multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching provided by the present invention first matches two-dimensional target image points, and then calculates the three-dimensional trajectory of the target using the matching results. This method has a small amount of calculation, realizes the processing of abnormal situations where input data has missing frames, improves the robustness of the algorithm, and has few constraint conditions for algorithm input, so the algorithm has strong universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 It is a schematic diagram of inter-frame correlation and inter-station matching of multi-target image points in multi-target intersection measurement; among them, (a) is a schematic diagram of inter-frame correlation of multi-target image points; (b) is a schematic diagram of inter-station matching;

[0081] Figure 2 It is a flowchart of the multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching of the present invention;

[0082] Figure 3 It is a flowchart of the spatio-temporal optimal inter-frame correlation algorithm in the measurement method of the present invention;

[0083] Figure 4 It is a calculation flowchart of the spatio-temporal optimal inter-frame correlation algorithm in the measurement method of the present invention;

[0084] Figure 5 It is a schematic diagram of the angular trajectory formed by the trajectory points and target image points in the embodiment of the present invention;

[0085] Figure 6 It is a schematic diagram of three angular trajectory point vectors in the embodiment of the present invention;

[0086] Figure 7 It is a flowchart of the spatio-temporal optimal inter-station matching algorithm in the measurement method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0087] To make the objectives, advantages and features of the present invention clearer, the following further elaborates in detail a multi-target real-time intersection measurement method based on spatio-temporal optimal correlation matching proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. Those skilled in the art should understand that these embodiments are only used to explain the technical principle of the present invention, and the purpose is not to limit the protection scope of the present invention.

[0088] As Figure 2 shown, the present invention provides the following specific steps:

[0089] Step 1. Establish several measurement stations in the space to be measured, and each measurement station measures and records the target image point data at each moment in the space; the target image point data includes the imaging time of the target image point, the azimuth angle and elevation angle of the target image point relative to each measurement station.

[0090] Step 2: Each measurement station performs spatio-temporal optimal inter-frame correlation calculation on the data of the target image points at each moment, and obtains the angular trajectory correlation results of the target image points measured by each measurement station.

[0091] Spatio-temporal optimal inter-frame correlation algorithm

[0092] The spatio-temporal optimal inter-frame correlation algorithm refers to performing inter-frame correlation on multiple targets. Specifically: according to the detected angular trajectories of multiple targets, the mathematical definitions of the consistency of the target flight direction and the consistency of the speed are deduced, and the problem of multi-target inter-frame correlation is transformed into a bi-objective programming problem with the optimal consistency of the target flight direction and the speed, and then simplified into an optimization problem with the weighted optimality of the target flight direction and the speed.

[0093] As Figure 3 shown, for the spatio-temporal optimal inter-frame correlation algorithm process, taking one measurement station as an example, according to the system angle measurement error, an adaptive correlation threshold is proposed, an optimization model for multi-target inter-frame correlation is established, and by solving the optimization mathematical model, the inter-frame multi-target correlation results are obtained.

[0094] 2.1. Each measurement station numbers the obtained target image points as 1, 2,..., i,..., N i .

[0095] 2.2. Each measurement station respectively establishes a set of angular trajectories of successfully associated target image points, correlates the target image points with the same number in the first two frames obtained, records them as the successfully associated angular trajectories and numbers them as 1, 2,..., j,..., M j , and at the same time saves the azimuth angle vector and the elevation angle vector of the angular trajectories in the set according to the target image point data in the angular trajectories;

[0096] Successful association means that at least the target flight directions of two target image points form an angular trajectory, that is, the number of frames of the angular trajectory of the associated target image points is greater than or equal to 2;

[0097] The dimensions of both the azimuth angle vector and the elevation angle vector are 1×N max , where N max is the number of target image points in the fillable angular trajectory.

[0098] 2.3. Each measurement station performs spatio-temporal optimal inter-frame correlation on the target image points obtained at the current moment with the angular trajectories in the set of successfully associated angular trajectories in turn; if the association is successful, the number of the angular trajectory successfully associated with the target image point is assigned; if the association is not successful, the target image point is added to the trajectory set as a new angular trajectory and the angular trajectory number M j +c is assigned, and the value of M j +c is greater than N i ;

[0099] It should be noted that when associating the target image point at this moment with the angular trajectories in the successfully associated trajectory set in sequence, those with the number of frames of the angular trajectories in the set being greater than or equal to 2 are preferentially selected.

[0100] As Figure 4 shown, associating the target image point at the current moment with the angular trajectories in the successfully associated trajectory set is specifically as follows:

[0101] 2.3.1. Select two target image points in any one of the angular trajectories in the successfully associated angular trajectory set, and form an angular trajectory with three time points with the target image point at the current moment;

[0102] 2.3.2. Calculate the direction consistency between the selected angular trajectory and the target image point at the current moment;

[0103] As Figure 5 shown, assume that the set angular trajectory length N max = 4, and the angular trajectory is: 1 → 2 → 3 → 4. Each time, select two points in the trajectory, select target image point 3, target image point 4 and the target image point 5 of the current frame to form three points, and calculate the included angle and distance of the two vectors formed by the three points. Let the coordinates of target image point 3 be (A3, E3), the coordinates of target image point 4 be (A4, E4), and the coordinates of the target image point 5 of the current frame be (A5, E5), where A 3 , A 4 , A 5 represents the azimuth angle of the target image point, and E 3 , E 4 , E 5 represents the pitch angle of the target image point. As Figure 6 shown, the two vectors in the angular trajectory are respectively and

[0104] Calculate the flight direction consistency θ between the angular trajectory and the target image point at the current moment:

[0105]

[0106]

[0107]

[0108]

[0109] Among them, represents the square of the distance between target image point 3 and target image point 4, represents the square of the distance between target image point 4 and the target image point 5 at the current moment; (A 5 - A 4 )(A 4 - A3 ) represents the product of the distances of the vector and the vector in the azimuth direction, (E 5 -E 4 )(E 4 -E 3 ) represents the vector and the vector in the pitch direction, represents the sum of these two products;

[0110] 2.3.3. Calculate the flight direction consistency threshold ε(θ * ) of the selected angular trajectory and the target image point at the current moment:

[0111]

[0112] where ε(AE * ) is the upper error limit of the selected angular trajectory;

[0113] 2.3.4. Calculate the flight speed consistency v of the selected angular trajectory and the target image point at the current moment:

[0114]

[0115] where a is the time difference between target image point 4 and target image point 3, and b is the time difference between target image point 4 and the target image point 5 at the current moment; it is stipulated that the time difference between adjacent angular trajectory points is 1;

[0116] 2.3.5. Calculate the flight speed consistency threshold ε(v * ) of the selected angular trajectory and the target image point at the current moment:

[0117]

[0118] 2.3.6. Select two target image points from the selected angular trajectory in a permutation and combination manner, and calculate the flight direction consistency, flight direction consistency threshold, flight speed consistency, and flight speed consistency threshold of the two target image points in the selected angular trajectory and the target image point at the current moment according to the methods in steps 2.3.1 to 2.3.5;

[0119] 2.3.7. Calculate the average flight direction consistency Average flight direction consistency threshold Average flight speed consistency and the average flight speed consistency threshold

[0120] 2.3.8. Calculate the average flight direction - speed consistency weighted value of the selected angular trajectory and the target image point at the current moment

[0121]

[0122] Among them, ρ is the weight coefficient of association, where 0 ≤ ρ ≤ 1;

[0123] 2.3.9. Calculate the weighted threshold of the average flight direction - speed consistency between the selected angular trajectory and the target image point at the current moment

[0124]

[0125] 2.3.10. Associate all the target image points obtained at the current moment with the angular trajectories in the angular trajectory set one by one. According to the above method, obtain the weighted values of the average flight direction - speed consistency of all the target image points at the current moment Matrix and the weighted threshold of the average flight direction - speed consistency Matrix;

[0126] 2.3.11. Establish an optimization model for multi - target inter - frame association;

[0127]

[0128]

[0129] Among them, is the weighted value of the average flight direction - speed consistency between the i - th target image point and the j - th angular trajectory at the current moment;

[0130] is the weighted threshold of the average flight - speed consistency between the i - th target image point and the j - th angular trajectory;

[0131] q ij is a decision variable of 0 or 1, representing the association relationship between the i - th target image point and the j - th angular trajectory; if the association is successful, then q ij = 1, if the association is unsuccessful, then q ij = 0;

[0132] This model transforms the multi - target inter - frame association problem into a bi - objective programming problem with optimal target flight direction consistency and speed consistency, and then simplifies it into an optimization problem with optimal weighted target flight direction consistency and speed consistency.

[0133] 2.3.12. Use the Hungarian algorithm to obtain the inter - frame association result. If the association is successful, assign the number of the angular trajectory successfully associated with the target image point; if the association is unsuccessful, add the target image point as a new angular trajectory to the trajectory set and assign the angular trajectory number M j +c, and M jThe value of +c is greater than N i ; c ≥ 1.

[0134] 2.4. Fill the azimuth angle vector and elevation angle vector of all the angular trajectories in the angular trajectory set in a sliding window manner according to the set angular trajectory length; if an angular trajectory is successfully associated with a new target image point, fill the azimuth angle and elevation angle of the new target image point into the azimuth angle vector and elevation angle vector of the corresponding angular trajectory in sequence; if an angular trajectory fails to be successfully associated with a new target image point, fill infinity inf into the azimuth angle vector and elevation angle vector of the corresponding angular trajectory.

[0135] 2.5. According to the methods in 2.1 - 2.4, the measurement station performs spatio-temporal optimal inter-frame association on all the target image points obtained, and obtains the angular trajectory association result of the target image points at each moment for each measurement station.

[0136] Step 3. According to the angular trajectory association result of the target image points measured by each measurement station respectively, perform spatio-temporal optimal inter-station matching to obtain the inter-station target image point matching result for each target image point.

[0137] Spatio-temporal optimal inter-station matching algorithm

[0138] As Figure 7 shown, the spatio-temporal optimal inter-station matching algorithm process fully considers the system angle measurement error and station error, proposes an adaptive matching threshold for the multi-target inter-station matching problem, eliminates false targets in complex environments, and the proposed threshold does not need to be reset due to the change of the observation distance. Based on the same-name image point matching criterion with the smallest included angle, considering the multi-frame motion information of the target, adopts the smallest average included angle of the target's multi-frames as the matching criterion, establishes an optimization model for spatio-temporal optimal inter-station matching of multi-targets, and obtains the inter-station multi-target association result by solving the optimization mathematical model.

[0139] 3.1. According to the angular trajectory association result of the target image points measured by each measurement station obtained in step 2, and the azimuth angle and elevation angle of all the target image points of each measurement station, calculate the angular trajectory unit vector of the target image points within each measurement station respectively;

[0140] Taking two measurement stations as an example, assume that the spatial position coordinates of measurement station s 1 are (x 1 , y 1 , z 1 ), the spatial position coordinates of measurement station s 2 are (x 2 , y 2 , z 2 ), and for any target image point o 1 within the field of view of measurement station s 1 at the current moment, the azimuth angle and elevation angle relative to measurement station s 1 are and measurement station s 2 any target image point o within the field of view 2 relative to the measurement station s 2 the azimuth and elevation angles are and then

[0141] the target image point o at the current moment 1 relative to the measurement station s 1 the angular trajectory unit vector is:

[0142]

[0143] the target image point o at the current moment 2 relative to the measurement station s 2 the angular trajectory unit vector is:

[0144]

[0145] 3.2. Let L 1 be the plane formed by the target image point o 1 and the measurement stations s 1 and s 2 . Let L 2 be the plane formed by the target image point o 2 and the measurement stations s 1 and s 2 . Calculate the angle β between the plane L1 where the target image point o 1 is located and the plane L2 where the target image point o 2 is located:

[0146]

[0147]

[0147] 3.3. Calculate the adaptive threshold ε(β * ) of the angle between the plane L1 and the plane L2:

[0148]

[0149] where ε(x i * ) is the upper limit of the measurement error in the x direction of the station location coordinates, ε(y i * ) is the upper limit of the measurement error in the y direction of the station location coordinates, ε(z i * ) is the upper limit of the measurement error in the z direction of the station location coordinates, ε(A i * ) is the upper limit of the measurement error of the azimuth angle of the target image point relative to the measurement station, ε(E i *) is the upper limit of the pitch angle measurement error of the target image point relative to the measurement station;

[0150] 3.4. Calculate the angle matrix between the planes where all target image points of the two measurement stations are located at the current moment, and the corresponding angle adaptive threshold matrix, according to the methods in steps 3.1 - 3.3;

[0151] 3.5. Combine the angles and angle adaptive threshold matrices of the historical Q frames to calculate the average angle matrix of all target image points of the two measurement stations at the current moment, and the corresponding average angle adaptive threshold matrix;

[0152] 3.6. Establish a multi - target spatio - temporal optimal matching model;

[0153]

[0154]

[0155] Among them, is the average value of the angle between the plane where the k - th target image point of the measurement station s is located at the current moment and the plane where the l - th target image point of the measurement station s is located at the current moment; M and N are the numbers of target image points of the measurement station s and the measurement station s at the current moment respectively; 1 the k - th target image point of the measurement station s at the current moment 2 and the plane where the l - th target image point of the measurement station s is located at the current moment 1 and the measurement station s 2 the number of target image points;

[0156] is the average angle adaptive threshold between the plane where the k - th target image point of the measurement station s is located at the current moment and the plane where the l - th target image point of the measurement station s is located at the current moment; 1 the k - th target image point of the measurement station s at the current moment 2 and the plane where the l - th target image point of the measurement station s is located at the current moment

[0157] p kl is a 0 or 1 decision variable, indicating the matching relationship between the k - th target image point of the measurement station s and the l - th target image point of the measurement station s; if the matching is successful, then p 1 = 1, if the matching is not successful, then p 2 = 0. kl kl

[0158] 3.7. Use the Hungarian algorithm to obtain the inter - station target image point matching results for each target image point.

[0159] Step 4. Perform intersection calculation to obtain the coordinates and motion speed of the target in three - dimensional space.

[0160] After the inter - station target image point matching of each target image point is completed, through intersection calculation, the coordinates and motion speed of the target image point in three - dimensional space can be obtained.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the present invention.

Claims

1. A multi-target real-time rendezvous measurement method based on spatio-temporal optimal correlation matching, characterized in that, it includes the following steps: Step 1: Establish a number of measurement stations in the space to be measured, and each measurement station measures and records the target image point data at each moment in the space; the target image point data includes the imaging time of the target image point, the azimuth angle and the elevation angle of the target image point relative to each measurement station; Step 2: Each measurement station performs spatio-temporal optimal inter-frame correlation calculation on the obtained target image point data at each moment to obtain the angular trajectory correlation result of the target image point measured by each measurement station; Step 3: According to the angular trajectory correlation result of the target image point measured by each measurement station, perform spatio-temporal optimal inter-station matching to obtain the inter-station target image point matching result of each target image point; Step 4: Perform rendezvous calculation according to the inter-station target image point matching result of each target image point to obtain the coordinates and motion speed of the target in three-dimensional space.

2. The multi-target real-time rendezvous measurement method based on spatio-temporal optimal correlation matching according to claim 1, characterized in that, Step 2 is specifically as follows: 2.

1. Each measurement station numbers the acquired target image points as 1, 2,..., i,..., N i ; 2.

2. For each measurement station, establish a set of angular trajectories with successfully associated target image points. Associate the target image points with the same number in the first two frames obtained, record them as the successfully associated angular trajectories and number them 1, 2,..., j,..., M j At the same time, save the azimuth vector and elevation vector of the angular trajectories in the set according to the target image point data in the angular trajectories; 2.

3. For each measurement station, the target image points obtained at the current moment are successively subjected to spatio-temporal optimal inter-frame association with the angular trajectories in the successfully associated angular trajectory set; if the association is successful, the number of the angular trajectory successfully associated with the target image point is assigned; if the association is not successful, the target image point is added to the trajectory set as a new angular trajectory, and the angular trajectory number M j +c is assigned, and M j +c is greater than N i ; c≥1; 2.

4. Fill the azimuth angle vector and elevation angle vector of all angular trajectories in the angular trajectory set in a sliding window manner according to the set angular trajectory length; if the angular trajectory is successfully associated with a new target image point, fill the azimuth angle and elevation angle of the new target image point in the azimuth angle vector and elevation angle vector of the corresponding angular trajectory in sequence; if the angular trajectory is not successfully associated with a new target image point, fill infinity inf in the azimuth angle vector and elevation angle vector of the corresponding angular trajectory; 2.

5. According to the method of 2.1 to 2.4, the measurement station performs spatio-temporal optimal inter-frame correlation on the obtained target image points of all frames to obtain the angular trajectory correlation result of the target image point at each moment of each measurement station.

3. The multi-target real-time rendezvous measurement method based on spatio-temporal optimal correlation matching according to claim 2, characterized in that, Step 2.3 is specifically as follows: 2.3.

1. Select two target image points in any one angular trajectory in the successfully associated angular trajectory set, and form an angular trajectory with three time points with the target image point at the current moment; 2.3.

2. Calculate the direction consistency between the selected angular trajectory and the target image point at the current moment; Let two target image points on any angular trajectory be selected as target image point 3, whose coordinates are (A 3 , E 3 ) and target image point 4, whose coordinates are (A 4 , E 4 ). The current target image point is target image point 5, whose coordinates are (A 5 , E 5 ); The angular trajectories are in sequence: target image point 3, target image point 4 to the current target image point 5; Among them, A 3 where A 4 and A 5 represent the azimuth angle of the target image point, and E 3 where E 4 and E 5 represent the elevation angle of the target image point; calculate the flight direction consistency θ between the angular trajectory and the target image point at the current moment: Among them, represents the square of the distance between target image point 3 and target image point 4, represents the square of the distance between target image point 4 and target image point 5 at the current moment; (A 5 -A 4 )(A 4 -A 3 ) represents the product of the distances in the azimuth direction of the vector and the vector ; (E 5 -E 4 )(E 4 -E 3 ) represents the product of the distances in the pitch direction of the vector and the vector ; represents the sum of these two products; the vector is the vector formed by target image point 3 and target image point 4; the vector is the vector formed by target image point 4 and target image point 5 at the current moment; 2.3.

3. Calculate the consistency threshold ε(θ * ) of the selected angular trajectory and the flight direction of the target image point at the current moment: where ε(AE * ) is the upper error limit of the selected angular trajectory; 2.3.

4. Calculate the flight speed consistency v between the selected angular trajectory and the target image point at the current moment; Among them, a is the time difference between target image point 4 and target image point 3, and b is the time difference between target image point 4 and the current target image point 5; 2.3.

5. Calculate the flight speed consistency threshold ε(v * ) between the selected angular trajectory and the target image point at the current moment: 2.3.

6. Select two target image points from the selected angular trajectory in a permutation and combination manner, and calculate the flight direction consistency, flight direction consistency threshold, flight speed consistency and flight speed consistency threshold between the two target image points in the selected angular trajectory and the target image point at the current moment according to the method of steps 2.3.1 to 2.3.5; 2.3.

7. Calculate the consistency of the average flight direction between the selected angular trajectory and the target image point at the current moment Average flight direction consistency threshold Average flight speed consistency And the average flight speed consistency threshold 2.3.

8. Calculate the average flight direction - speed consistency weighted value of the selected angular trajectory and the target image point at the current moment Among them , ρ is the associated weight coefficient, 0 ≤ ρ ≤ 1; 2.3.

9. Calculate the average flight direction - speed consistency weighted threshold between the selected angular trajectory and the target image point at the current moment 2.3.

10. Associate all the target image points obtained at the current moment with the angular trajectories in the angular trajectory set one by one, and according to the methods in 2.3.1 - 2.3.9, obtain the average flight direction - speed consistency weighted value of all the target image points at the current moment Matrix and average flight direction - speed consistency weighted threshold Matrix; 2.3.

11. Establish an optimization model for multi-target inter-frame association; Among them, is the weighted value of the average flight direction - speed consistency between the i-th target image point and the j-th angular trajectory at the current moment; is the average flight-speed consistency weighting threshold between the i-th target image point and the j-th angular trajectory; q ij is a decision variable of 0 or 1, representing the association relationship between the i-th target image point and the j-th angular trajectory; if the association is successful, then q ij = 1, if the association is unsuccessful, then q ij = 0; 2.3.

12. Use the Hungarian algorithm to obtain the inter-frame correlation results. If the association is successful, assign the number of the angular trajectory that is successfully associated with the target image point; if the association is not successful, add the target image point as a new angular trajectory to the trajectory set and assign the angular trajectory number M j +c, and M j +c has a value greater than N i ; c ≥ 1.

4. The multi-target real-time rendezvous measurement method based on spatio-temporal optimal correlation matching according to claim 3, characterized in that, Step 3 specifically includes: 3.

1. Based on the angular trajectory association results of the target image points measured by each measurement station obtained in Step 2, and the azimuth angles and elevation angles of all target image points of each measurement station, calculate the angular trajectory unit vectors of the target image points within each measurement station respectively; Define the spatial position coordinates of measurement station s 1 as (x 1 , y 1 , z 1 ). The spatial position coordinates of measurement station s 2 are (x 2 , y 2 , z 2 ). At the current moment, for any target image point o 1 within the field of view of measurement station s 1 , the azimuth angle and elevation angle relative to measurement station s 1 are and For any target image point o 2 within the field of view of measurement station s 2 , the azimuth angle and elevation angle relative to measurement station s 2 are and Then The target image point o at the current moment 1 Relative to the measurement station s 1 The angular trajectory unit vector is: The target image point o at the current moment 2 Relative to the measurement station s 2 The angular trajectory unit vector is as follows: 3.

2. Let L 1 be the plane formed by the target image point o 1 and the measurement stations s 1 and s 2 . Let L 2 be the plane formed by the target image point o 2 and the measurement stations s 1 and s 2 . Calculate the included angle β between the plane L1 where the target image point o 1 is located and the plane L2 where the image point o 2 is located: 3.

3. Calculate the adaptive threshold ε(β * ) of the angle between plane L1 and plane L2: Among them, ε(x i * ) is the upper limit of the measurement error in the x - direction of the site coordinates, ε(y i * ) is the upper limit of the measurement error in the y - direction of the site coordinates, ε(z i * ) is the upper limit of the measurement error in the z - direction of the site coordinates, ε(A i * ) is the upper limit of the measurement error of the azimuth angle of the target image point relative to the measurement station, ε(E i * ) is the upper limit of the measurement error of the elevation angle of the target image point relative to the measurement station; 3.

4. According to the methods in Steps 3.1 - 3.3, calculate the included angle matrix between the planes where all target image points of the two measurement stations are located at the current moment, and the corresponding included angle adaptive threshold matrix; 3.

5. Combining the included angles and the included angle adaptive threshold matrix of the historical Q frames, calculate the average included angle matrix of all target image points of the two measurement stations at the current moment, and the corresponding average included angle adaptive threshold matrix; 3.

6. Establish a multi - target spatio - temporal optimal matching model; Among them, is the measurement station s at the current moment 1 The average angle between the plane where the k-th target image point is located and the measurement station s at the current moment 2 and the plane where the l-th target image point is located; M and N are respectively the measurement station s at the current moment 1 and the measurement station s 2 The number of target image points; Measurement station s at the current moment 1 The average angular adaptive threshold between the plane where the k-th target image point is located and the measurement station s at the current moment 2 And the plane where the l-th target image point is located p kl is a decision variable of 0 or 1, representing the matching relationship between the k-th target image point of the measurement station s 1 and the l-th target image point of the measurement station s 2 ; if the matching is successful, then p kl = 1, if the matching is unsuccessful, then p kl = 0; 3.

7. Use the Hungarian algorithm to obtain the inter - station target image point matching results of each target image point.

5. The multi - target real - time intersection measurement method based on spatio - temporal optimal correlation matching according to claim 4, characterized in that: In Step 2.3.4, the time difference between adjacent angular trajectory points is 1.

6. The multi - target real - time intersection measurement method based on spatio - temporal optimal correlation matching according to claim 5, characterized in that: In step 2.2, the dimensions of both the azimuth vector and the elevation vector are 1×N max , where N max is the number of target image points in the fillable angular trajectory.

7. The multi - target real - time intersection measurement method based on spatio - temporal optimal correlation matching according to claim 6, characterized in that: In Step 2.2, N max = 3 to 7.

Images