Multi-passive sensor space-time and track dual-constraint multi-target identity determination method

Abstract

The application belongs to the technical field of multi-target identity determination among multiple passive sensors. The application provides a multi-passive-sensor space-time and track dual-constraint multi-target identity determination method. The embodiments of the disclosure can more accurately describe the identity determination characteristics of multi-targets in different sensors by constructing a space-time criterion based on continuous multi-frame epipolar geometry constraint and a track criterion based on track feature cosine similarity, thereby significantly improving the accuracy and robustness of multi-target identity determination. The adaptive fusion weight mechanism solves the weight optimization problem under different sensor configurations and scene characteristics, avoiding the performance degradation of the fixed weight strategy in complex scenes. It has good scene adaptive ability and excellent real-time response characteristics, fully meets the fast online processing needs in the resource-constrained airborne environment, while maintaining the flexibility and scalability of cross-platform deployment, providing reliable technical support for multi-aircraft cooperative sensing and other frontier applications.

Description

Technical Field

[0001] This disclosure relates to the field of multi-target identity determination technology among multiple passive sensors, and particularly to a method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints of multiple passive sensors. Background Technology

[0002] Passive sensors possess advantages such as strong concealment, low power consumption, and long detection range, and are widely used in civilian scenarios such as urban security monitoring, traffic management, environmental monitoring, and drone control. However, individual passive sensors have inherent drawbacks, including limited field of view, susceptibility to environmental factors, inability to directly acquire 3D position information, and limited recognition capabilities in complex backgrounds. To address these shortcomings, a multi-passive sensor collaborative sensing scheme deploys multiple passive sensors to form a distributed sensing network. This leverages multi-view collaborative observation to achieve wide-range field of view coverage, improve detection robustness, realize 3D target localization, and enhance recognition accuracy in complex environments. In multi-passive sensor collaborative sensing, multi-target identity determination is a key technical step. This involves determining the one-to-one correspondence between multiple non-cooperative targets detected in the 2D imaging plane of different sensors, assigning unique IDs to these non-cooperative targets, and enabling multiple passive sensors to recognize these unique IDs. The accuracy of this technology directly affects the effectiveness of subsequent target localization, tracking, and situational awareness.

[0003] Current methods for determining the identity of multiple targets among multiple passive sensors can be mainly categorized into two main types:

[0004] The first category is methods based on spatiotemporal geometric constraints. These methods associate targets based on the imaging geometric constraints of the target across different passive sensors. Typical constraints include epipolar geometric constraints, line-of-sight intersection constraints, and spatiotemporal consistency constraints. For example, using the epipolar geometric constraints of the primary and secondary sensors, target identity is determined by calculating the matching relationship of the target's direction-finding lines in the imaging planes of the two sensors; or the principle of multi-sensor line-of-sight intersection is used to determine target identity by minimizing reprojection errors. This type of method performs well when the sensor positions are precisely known and the target detection accuracy is high, but it is easily limited by the spatial layout constraints of multiple sensors. When the baseline distance between sensors is too small, the difference in viewing angle is not significant, or the target position does not meet the geometric constraints, the associativity conditions may not be met, leading to failure in target identity determination or a decrease in accuracy.

[0005] The second category is feature similarity methods based on deep learning. This method utilizes deep learning networks to extract common features of a target across different passive sensors, determining target identity through feature similarity matching. Typical approaches include using deep learning networks to extract target appearance features, employing recurrent neural networks to learn trajectory motion pattern features, and using attention mechanisms to fuse multimodal features. By calculating the similarity of target features across different sensors, the feature pairs with the highest similarity are identified as the same target. This method performs well in scenarios with high target feature discriminative power, but it is limited by the learning effect and feature extraction capabilities of specific scenarios. It is less adaptable to multi-target scenarios with complex motion patterns and similar target features. Furthermore, training deep learning models requires a large amount of labeled data, making it difficult to adapt to large-scale, diverse real-world application scenarios.

[0006] The aforementioned multi-target identity determination method also has the following shortcomings: First, it is limited by the single constraint criterion, relying solely on one criterion from geometric constraints or feature similarity, which is insufficient in fault tolerance under complex scenarios; second, it lacks an adaptive weight adjustment mechanism, making it impossible to dynamically optimize the identity determination strategy according to different sensor configurations and scenario characteristics; third, it has high computational complexity, making it difficult to meet the requirements of real-time airborne online processing; and fourth, it has stringent requirements for sensor calibration accuracy, limiting its applicability in actual deployment.

[0007] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0008] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0009] The purpose of this disclosure is to provide a method for determining the identity of multiple targets under the dual constraints of spatiotemporal and trajectory constraints of multiple passive sensors, thereby overcoming, to at least to some extent, one or more problems caused by the limitations and defects of related technologies.

[0010] According to embodiments of this disclosure, a method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors is provided, including:

[0011] Step S1: Construct a multi-layer data pool. Each layer of the data pool stores observation data from multiple engine platforms in frames. The multi-layer data pool is dynamically updated using a first-in-first-out strategy. Once the multi-layer data pool is filled with a set number of frames at a set depth, the identity determination task is started.

[0012] Step S2: Based on all the observation data in the multi-layer data pool, spline interpolation fitting is performed on the pixel coordinates of each target in the imaging plane of each passive sensor for multiple consecutive frames to generate a two-dimensional time-series track of each target in each passive sensor. The two-dimensional time-series tracks of each target are sorted based on the distance from the pixel coordinates of each target in the first frame of the multi-layer data pool to the center of the imaging plane of the passive sensor to obtain an ordered track set.

[0013] Step S3: Set one passive sensor as the main sensor and the rest as auxiliary sensors, and decompose the multi-target identity determination problem among multiple passive sensors into multiple sub-problems between main and auxiliary sensors.

[0014] Step S4: For each pair of primary and secondary sensors, construct the spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix respectively.

[0015] Step S5: Calculate the geometric configuration relationship between the primary and secondary sensors based on the observation data of the first layer in the multi-layer data pool, and adaptively determine the first fusion weight of the spatiotemporal geometric constraint identity determination matrix and the second fusion weight of the track feature constraint identity determination matrix.

[0016] Step S6: Using the first fusion weight and the second fusion weight, the spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix are fused to obtain the dual-criteria identity determination matrix between the main and auxiliary sensors. The Hungarian algorithm is then used to solve the dual-criteria identity determination matrix between the main and auxiliary sensors to obtain the multi-target identity determination result between a pair of main and auxiliary sensors.

[0017] Step S7: Statistically analyze the multi-target identity determination results of all primary and secondary sensors to obtain the global multi-target identity determination results of multiple passive sensors.

[0018] Furthermore, step S1 specifically includes:

[0019] A multi-layer data pool is constructed, with each layer storing observation data from multiple onboard platforms frame by frame. The observation data includes the timestamps, 3D positions, attitude angles, orientation angles of the passive sensors mounted on the onboard platforms, and the pixel coordinates of all targets obtained by them.

[0020] The depth of each data pool represents the number of frames of stored data. A first-in-first-out strategy is adopted, and the update of multiple data pools is completed after reading the next frame of observation data.

[0021] If the observation data in the data pool has not reached the preset depth, then the identity determination preparation stage will begin.

[0022] If the observation data in the data pool is full at the set depth, the multi-target identity determination task will be executed.

[0023] Furthermore, step S2 specifically includes:

[0024] Based on all observation data within the multi-layer data pool, extract continuous pixel coordinate sequences for all targets;

[0025] Based on the continuous pixel coordinate sequence of all targets, cubic spline interpolation is used to fit and generate two-dimensional time-series tracks for each target.

[0026] Using the multi-target pixel coordinates of the first frame of the multi-layer data pool as a reference, the Euclidean distance from the pixel coordinates of each target to the center of the imaging plane of the passive sensor is calculated. The two-dimensional time-series tracks of each target are sorted and numbered according to the Euclidean distance from smallest to largest to obtain an ordered track set.

[0027] Furthermore, the construction of the spatiotemporal geometric constraint identity determination matrix specifically includes:

[0028] Based on the observation and attitude angles of the main sensor and the auxiliary sensor in the passive sensor imaging plane, as well as the internal and external parameters of the main sensor and the auxiliary sensor, the unit direction finding vector pointing from the main sensor to its corresponding observation and the three-dimensional direction finding vector pointing from the auxiliary sensor to its corresponding observation in the world coordinate system are obtained.

[0029] Based on the unit direction finding vector of the main sensor pointing to its corresponding observation in the world coordinate system and the three-dimensional direction finding vector of the auxiliary sensor pointing to its corresponding observation, the common perpendicular distance between the two direction finding lines of the main sensor and the auxiliary sensor pointing to the target is obtained.

[0030] The mean constraint of the common perpendicular distance between the two direction finding lines pointed to the target by the main sensor and the auxiliary sensor within a continuous time interval is obtained.

[0031] The mean value constraint of the distance between the common perpendiculars of the direction finding lines is used as a special criterion for spatiotemporal geometric constraints to construct a spatiotemporal geometric constraint identity determination matrix.

[0032] Furthermore, the construction of the track feature constraint identity determination matrix specifically includes:

[0033] Extract the instantaneous motion vector of the target within a continuous time interval;

[0034] Decompose the motion direction characteristics and motion displacement characteristics from the instantaneous motion vector;

[0035] By integrating the target motion characteristics and motion direction characteristics within a continuous time interval, the temporal trajectory characteristic constraint criteria of the target are represented.

[0036] Based on the temporal trajectory feature constraint criterion, a trajectory feature constraint identity determination matrix is ​​constructed.

[0037] Furthermore, step S5 specifically includes:

[0038] Based on the observation data of the first layer in the multi-layer data pool, the baseline direction vectors of the main and auxiliary sensors are calculated;

[0039] The midpoint of the common perpendicular of the line of sight from the optical centers of the primary and secondary sensors to their respective field-of-view centers at that moment is taken as the overlap point of the field of view of the primary and secondary sensors. Combined with the baseline direction vectors of the primary and secondary sensors, the foot of the perpendicular and the midpoint of the baseline at this point are calculated.

[0040] Calculate the geometric configuration parameters based on the baseline length and the distance between the perpendicular foot and the midpoint of the baseline. ;

[0041] When geometry configuration parameters At that time, the first fusion weight of the spatiotemporal geometric constraint identity determination matrix The second fusion weight of the identity determination matrix of track feature constraints The calculation method is as follows:

[0042]

[0043] When geometry configuration parameters At that time, the first fusion weight of the spatiotemporal geometric constraint identity determination matrix The second fusion weight of the identity determination matrix of track feature constraints The calculation method is as follows:

[0044]

[0045] When geometry configuration parameters At that time, the first fusion weight of the spatiotemporal geometric constraint identity determination matrix The second fusion weight of the identity determination matrix of track feature constraints The calculation method is as follows:

[0046] .

[0047] Furthermore, step S6 specifically includes:

[0048] The spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix are normalized.

[0049] The normalized spatiotemporal geometric constraint identity determination matrix and the normalized track feature constraint identity determination matrix are fused using the first fusion weight and the second fusion weight to obtain the dual-criteria identity determination matrix between the main and auxiliary sensors.

[0050] Based on the dual-criteria identity determination matrix between the main and auxiliary sensors, the Hungarian algorithm is used to solve the minimum weight maximum matching problem to obtain the multi-target identity determination result between the main and auxiliary sensors.

[0051] Furthermore, the expression for the dual-criteria identity determination matrix between the primary and secondary sensors is as follows:

[0052]

[0053] in, This is the normalized spatiotemporal geometric constraint identity determination matrix. This is the normalized track feature constraint identity determination matrix;

[0054] Solving for the identity determination matrix between the primary and secondary sensors based on dual criteria, the multi-target identity determination problem can be formulated as follows:

[0055]

[0056] in, The number of targets observed within the main sensor. Indicates the first observation within the main sensor One goal, To increase the number of targets observed within the auxiliary sensor, The first observation within the auxiliary sensor One goal, The weights for determining the identity of multiple targets between primary and secondary sensors are between 0 and 1. For identity determination decision variables, , This indicates that the target is not the same. This indicates that they are considered to be the same target.

[0057] Furthermore, step S7 specifically includes:

[0058] Statistically analyze the multi-target identity determination results of all primary and secondary sensors, and use the average accuracy. The evaluation metrics assess the effectiveness of multi-target identity determination among multiple sensors to obtain a global multi-target identity determination result; the average accuracy rate is:

[0059]

[0060] In the formula, This represents the number of correct results in the multi-objective identity determination. This represents the number of errors in the multi-objective identity determination result.

[0061] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0062] In the embodiments of this disclosure, the multi-target identity determination method with spatiotemporal and track dual constraints of multiple passive sensors, on the one hand, by constructing spatiotemporal criteria based on continuous multi-frame epipolar geometric constraints and track criteria based on track feature cosine similarity, can more accurately describe the identity determination characteristics of multiple targets in different sensors, thereby significantly improving the accuracy and robustness of multi-target identity determination. On the other hand, the method effectively solves the weight optimization problem under different sensor configurations and scene characteristics through an adaptive fusion weight mechanism, avoiding the performance degradation of fixed weight strategies in complex scenes. This method has good scene adaptability and excellent real-time response characteristics, fully meeting the rapid online processing requirements of resource-constrained airborne environments, while maintaining the flexibility and scalability of cross-platform deployment, providing reliable technical support for cutting-edge applications such as multi-aircraft collaborative perception. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0064] Figure 1 This diagram illustrates the steps of a multi-passive sensor spatiotemporal and trajectory dual-constraint multi-target identity determination method in an exemplary embodiment of this disclosure.

[0065] Figure 2 This diagram illustrates a typical scenario of multi-target identity determination in an exemplary embodiment of this disclosure, where multiple motor-mounted platforms equipped with multiple passive sensors collaboratively perceive multiple moving targets.

[0066] Figure 3 This diagram illustrates the data storage formats of multiple engine-borne platforms in an exemplary embodiment of this disclosure.

[0067] Figure 4 This diagram illustrates a data processing and storage method for multiple engine-based platforms in an exemplary embodiment of this disclosure.

[0068] Figure 5 This illustration shows a method for decomposing the problem of determining the identity of multiple targets among multiple sensors into a problem of determining the unity among multiple main and auxiliary sensors in an exemplary embodiment of this disclosure;

[0069] Figure 6 This diagram illustrates a method for determining the spatiotemporal geometric constraint identity determination matrix based on spatiotemporal geometric constraint criteria in an exemplary embodiment of this disclosure.

[0070] Figure 7 This diagram illustrates a method for determining the identity determination matrix of track feature constraints based on temporal track feature criteria in an exemplary embodiment of this disclosure.

[0071] Figure 8 This diagram illustrates a method for determining adaptive fusion weights of the identity determination matrix in conjunction with geometric configuration parameters in an exemplary embodiment of this disclosure.

[0072] Figure 9 This diagram illustrates a method for obtaining the identity determination result of multiple targets among multiple sensors based on the identity determination matrix of multiple targets between primary and secondary sensors in an exemplary embodiment of this disclosure. Detailed Implementation

[0073] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0074] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0075] This example implementation provides a method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors. (Reference) Figure 1 As shown, the method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors may include:

[0076] Step S1: Construct a multi-layer data pool. Each layer of the data pool stores observation data from multiple engine platforms in frames. The multi-layer data pool is dynamically updated using a first-in-first-out strategy. Once the multi-layer data pool is filled with a set number of frames at a set depth, the identity determination task is started.

[0077] Step S2: Based on all the observation data in the multi-layer data pool, spline interpolation fitting is performed on the pixel coordinates of each target in the imaging plane of each passive sensor for multiple consecutive frames to generate a two-dimensional time-series track of each target in each passive sensor. The two-dimensional time-series tracks of each target are sorted based on the distance from the pixel coordinates of each target in the first frame of the multi-layer data pool to the center of the imaging plane of the passive sensor to obtain an ordered track set.

[0078] Step S3: Set one passive sensor as the main sensor and the rest as auxiliary sensors, and decompose the multi-target identity determination problem among multiple passive sensors into multiple sub-problems between main and auxiliary sensors.

[0079] Step S4: For each pair of primary and secondary sensors, construct the spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix respectively.

[0080] Step S5: Calculate the geometric configuration relationship between the primary and secondary sensors based on the observation data of the first layer in the multi-layer data pool, and adaptively determine the first fusion weight of the spatiotemporal geometric constraint identity determination matrix and the second fusion weight of the track feature constraint identity determination matrix.

[0081] Step S6: Using the first fusion weight and the second fusion weight, the spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix are fused to obtain the dual-criteria identity determination matrix between the main and auxiliary sensors. The Hungarian algorithm is then used to solve the matrix to obtain the multi-target identity determination result between a pair of main and auxiliary sensors.

[0082] Step S7: Statistically analyze the multi-target identity determination results of all primary and secondary sensors to obtain the global multi-target identity determination results of multiple passive sensors.

[0083] The aforementioned method for determining the identity of multiple targets using a multi-passive sensor approach with both spatiotemporal and trajectory constraints achieves several advantages. Firstly, by constructing spatiotemporal criteria based on continuous multi-frame epipolar geometric constraints and trajectory criteria based on cosine similarity of trajectory features, it can more accurately describe the identity determination characteristics of multiple targets across different sensors, thus significantly improving the accuracy and robustness of multi-target identity determination. Secondly, this method effectively solves the weight optimization problem under different sensor configurations and scene characteristics through an adaptive fusion weight mechanism, avoiding the performance degradation of fixed weight strategies in complex scenarios. This method possesses excellent scene adaptability and real-time response characteristics, fully meeting the rapid online processing requirements of resource-constrained airborne environments, while maintaining the flexibility and scalability of cross-platform deployment, providing reliable technical support for cutting-edge applications such as multi-aircraft collaborative sensing.

[0084] Below, we will refer to Figures 1 to 9 The steps of the multi-passive sensor spatiotemporal and track dual-constraint multi-target identity determination method in this example embodiment will be described in more detail.

[0085] In step S1, a multi-layer data pool is constructed. Each layer of the data pool stores observation data from multiple engine platforms frame by frame. The multi-layer data pool is dynamically updated using a first-in-first-out strategy. Once the multi-layer data pool is filled with a set number of frames at a certain depth, the identity determination task is initiated.

[0086] Specifically, a data pool is set up to store data from multiple engine platforms. The data pool has multiple layers, with each layer containing a frame of data from multiple engine platforms (i.e., observation data of the engine platforms). The depth represents the number of frames of data stored in the data pool. A first-in, first-out strategy is adopted, and the data pool is updated after reading the next frame of data. If the data in the data pool does not meet the depth requirement, the identity determination preparation stage is entered. Once the data pool is full, the multi-target identity determination task is executed.

[0087] In step S2, based on all the observation data in the multi-layer data pool, spline interpolation fitting is performed on the pixel coordinates of each target in the imaging plane of each passive sensor for consecutive frames to generate a two-dimensional time-series track of each target in each passive sensor. The two-dimensional time-series tracks of each target are sorted based on the distance from the pixel coordinates of each target in the first frame of the multi-layer data pool to the center of the imaging plane of the passive sensor to obtain an ordered track set.

[0088] Specifically, based on all observation data within the multi-layer data pool, continuous pixel coordinate sequences of all targets are extracted; based on the continuous pixel coordinate sequences of all targets, cubic spline interpolation is used to fit and generate two-dimensional temporal tracks for each target; using the multi-target pixel coordinates of the first frame of the multi-layer data pool as a reference, the Euclidean distance from each target pixel coordinate to the center of the passive sensor imaging plane is calculated, and the two-dimensional temporal tracks of each target are sorted and numbered according to the Euclidean distance from smallest to largest to obtain an ordered track set.

[0089] In step S3, one passive sensor is designated as the main sensor and the remaining passive sensors as auxiliary sensors. The problem of determining the identity of multiple targets among multiple passive sensors is decomposed into multiple sub-problems of determination between main and auxiliary sensors.

[0090] Specifically, the problem of determining the identity of multiple targets among multiple sensors is decomposed into the problem of determining the identity of multiple targets among multiple pairs of main and auxiliary sensors.

[0091] In step S4, for each pair of primary and secondary sensors, a spatiotemporal geometric constraint identity determination matrix and a track feature constraint identity determination matrix are constructed respectively.

[0092] Specifically, the construction of the spatiotemporal geometric constraint identity determination matrix includes:

[0093] Based on the observations (i.e., two-dimensional pixel coordinate observation data) and attitude angles of the main sensor and auxiliary sensor in the passive sensor imaging plane, as well as the intrinsic and extrinsic parameters of the main sensor and auxiliary sensor, the unit direction finding vector pointing from the main sensor to its corresponding observation and the three-dimensional direction finding vector pointing from the auxiliary sensor to its corresponding observation in the world coordinate system are obtained. Based on the unit direction finding vector pointing from the main sensor to its corresponding observation and the three-dimensional direction finding vector pointing from the auxiliary sensor to its corresponding observation in the world coordinate system, the common perpendicular distance between the two direction finding lines pointing to the target by the main sensor and auxiliary sensor respectively is obtained. Based on the common perpendicular distance between the two direction finding lines pointing to the target by the main sensor and auxiliary sensor respectively within a continuous time interval, the mean constraint of the common perpendicular distance of the direction finding lines is obtained. The mean constraint of the common perpendicular distance of the direction finding lines is used as the spatiotemporal geometric constraint criterion to construct the spatiotemporal geometric constraint identity determination matrix.

[0094] The construction of the track feature constraint identity determination matrix specifically includes:

[0095] Extract the instantaneous motion vector of the target within a continuous time interval; decompose the motion direction feature and motion displacement feature from the instantaneous motion vector; fuse the target motion feature and motion direction feature within the continuous time interval to represent the temporal trajectory feature constraint criterion of the target; and construct the trajectory feature constraint identity judgment matrix based on the temporal trajectory feature constraint criterion.

[0096] In step S5, the geometric configuration relationship between the primary and secondary sensors is calculated based on the observation data of the first layer in the multi-layer data pool, and the first fusion weight of the spatiotemporal geometric constraint identity determination matrix and the second fusion weight of the track feature constraint identity determination matrix are adaptively determined.

[0097] Specifically, a method for calculating the fusion weights of the spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix is ​​designed. Combining the first layer of data in the data pool, the line of sight from the optical center of any primary and secondary sensor to the center of their respective fields of view is calculated, and the field of view overlap point of the primary and secondary sensors is calculated. The perpendicular line from the field of view overlap point to the baseline of the primary and secondary sensors is used to obtain the foot of the perpendicular line of the field of view overlap point on the baseline. By calculating the positional relationship between the foot of the perpendicular line of the field of view overlap point and the center point of the current primary and secondary sensor combined baseline, the relationship between the primary and secondary field of view overlap area and the geometric configuration of the primary and secondary sensors is evaluated, thereby adaptively determining the first fusion weight of the spatiotemporal geometric constraint identity determination matrix and the second fusion weight of the track feature constraint identity determination matrix.

[0098] In steps S6 and S7, the spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix are fused using the first fusion weight and the second fusion weight to obtain a dual-criteria identity determination matrix between the primary and secondary sensors. This matrix is ​​then solved using the Hungarian algorithm to obtain the multi-target identity determination result between a pair of primary and secondary sensors. The multi-target identity determination results of all primary and secondary sensors are statistically analyzed to obtain the global multi-target identity determination result for multiple passive sensors.

[0099] Specifically, the identity determination matrix for multi-target geometric constraints and the identity determination matrix for multi-target trajectory feature constraints are normalized and fused by combining the first fusion weight and the second fusion weight to obtain the identity determination matrix for multi-target dual criteria between the main and auxiliary sensors. The rows represent the two-dimensional temporal trajectories of all targets in the imaging plane of the main sensor, and the columns represent the two-dimensional temporal trajectories of all targets in the imaging plane of the auxiliary sensor. The Hungarian algorithm is used to solve the identity determination matrix for multi-target dual criteria between the main and auxiliary sensors, and the identity determination results for multi-target between the main and auxiliary sensors are output. The identity determination results for all main-auxiliary sensor combinations in the multi-sensor system are statistically analyzed, and the average accuracy index is used to calculate the identity determination results for multi-target between the sensors.

[0100] In one specific embodiment, this application proposes a method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors. This embodiment describes in detail the complete process steps of this application to complete the determination of the identity of multiple targets under spatiotemporal and trajectory dual constraints using four motorized platforms respectively equipped with passive sensors to collaboratively detect four moving targets. Figure 2 The typical scenario depicted, where multiple motorized platforms carrying multiple passive sensors collaboratively perceive multiple moving targets, involves four motorized platforms. Within a certain timeframe, all four platforms observe four moving targets within their shared field of view, forming four continuous two-dimensional temporal tracks in the 2D imaging plane of their respective passive sensors. The problem of determining the identity of multiple targets among the multiple passive sensors is essentially how to establish a one-to-one correspondence between these multiple two-dimensional tracks in the 2D imaging planes of different sensors.

[0101] This application assumes that multi-target two-dimensional tracking of multiple targets within a sensor imaging plane over a period of time has been achieved using a multi-target detection algorithm and a multi-target two-dimensional continuous tracking method, and that a batch number has been generated. The independent timestamps, positions, and attitudes of each onboard platform, the orientation of the onboard passive sensors, and the continuous tracking data of the pixel coordinates of all targets obtained by the onboard passive sensors are all stored in independent .csv format. For example... Figure 3 As shown, Figure 2The four .csv data files generated in the scenario shown are named zaiji+number.csv. Each .csv data file for a specific platform stores one frame of data in a single line, including: current frame time, X-axis coordinate position of the platform in the current frame, Y-axis coordinate position of the platform in the current frame, Z-axis coordinate position of the platform in the current frame, pitch attitude angle of the platform in the current frame, yaw attitude angle of the platform in the current frame, roll attitude angle of the platform in the current frame, pitch attitude angle of the passive sensor on the platform in the current frame, yaw attitude angle of the passive sensor on the platform in the current frame, horizontal pixel coordinate of target 1 detected in the current frame, vertical pixel coordinate of target 1 detected in the current frame, horizontal pixel coordinate of target 2 detected in the current frame, vertical pixel coordinate of target 2 detected in the current frame, horizontal pixel coordinate of target 3 detected in the current frame, vertical pixel coordinate of target 3 detected in the current frame, horizontal pixel coordinate of target 4 detected in the current frame, and vertical pixel coordinate of target 4 detected in the current frame. This application assumes that the spatial coordinates of the sensor's centroid or optical center coincide. Furthermore, the identity determination issue in this embodiment is not limited to... Figure 3 The multi-target identity determination problem in the scenario shown, where four passive sensors collaboratively perceive four targets, can also be extended to the multi-target identity determination problem among multiple passive sensors. This embodiment combines... Figure 2 The typical problem shown is assumed to have been obtained. Figure 3 The data storage format shown represents multiple engine platform data files whose identity needs to be determined, according to... Figure 1 The specific process steps shown in this embodiment are as follows:

[0102] Step S1: Set up a data pool to store data from multiple engine platforms. The data pool has multiple layers, with each layer containing data from multiple engine platforms in a single frame. The depth represents the number of frames of data stored in the data pool. A first-in, first-out strategy is adopted. The data pool is updated after reading the next frame of data. If the data in the data pool does not reach the depth requirement, the identity determination preparation stage is entered. Once the data pool is full, the multi-target identity determination task is executed.

[0103] In the system Each platform is equipped with one passive sensor; the data pool depth is defined as follows: Therefore, the data pool can be represented as:

[0104]

[0105] in For the first A multi-load platform data set of frames. It is the first The first carrier platform Frame data. The data pool uses a first-in, first-out (FIFO) strategy for updates. When the first frame of data is read... Frame data At that time, if the current number of frames accessed by the data pool Then Add to the data pool; if In the data pool Remove the earliest frame of data and will Add to data pool, data pool updated to The number of data frames stored in the data pool is always maintained at 1 Frame. For example... Figure 4 The diagram illustrates the data processing and storage methods for four airborne platforms. The top four images show the continuous tracking of multiple targets by the passive sensors of the four airborne platforms within their respective 2D imaging planes. The curves in each imaging plane represent the continuous two-dimensional motion trajectories of the multiple targets. The top right image visualizes the data pool storage structure, presenting the frame storage characteristics of the data pool in a multi-layered stacked format. Each layer corresponds to a frame containing a collection of observation data from all airborne platforms. The depth of the data pool is determined by the number of continuously stored frames. A first-in-first-out (FIFO) strategy is used to dynamically update the frame data. When the set depth is not reached, the pool is in a judgment preparation state. Once the pool is full, a similarity judgment task is initiated. The bottom image shows the raw data storage format of a particular airborne platform, presented as a .csv file. The file stores single-frame data line by line, including key parameters such as timestamps, platform 3D position, pitch / yaw / roll attitude angles, sensor orientation angles, and the horizontal / vertical pixel coordinates of each target.

[0106] Step S2: Combining all data in the data pool, and through interpolation and fitting of the pixel coordinates of each target across multiple consecutive frames, a two-dimensional temporal track of multiple targets observed in each sensor's imaging plane under continuous multi-frame conditions is formed. Then, the two-dimensional temporal tracks of multiple targets in each sensor's imaging plane are sorted based on the pixel coordinates of the multiple targets observed in the first layer of data in the data pool.

[0107] The trajectory in the two-dimensional imaging plane of each carrier platform's passive sensor is formed by the point traces of the same target in multiple consecutive frames, and the data pool contains the first... The sensor, which observed the first The first goal in The pixel coordinates of the frame are :

[0108]

[0109] in, These represent the data in the data pool. The first sensor observed the first The first goal in The x and y coordinates of the pixels in the frame. The discrete pixel coordinate sequence of the target obtained through cubic spline interpolation. By fitting, a target can be generated. In the A two-dimensional temporal trajectory in the 2D imaging plane of a sensor, the trajectory function expression is as follows:

[0110]

[0111] in, For continuous time variables, The cubic spline interpolation functions representing the pixel coordinates of the horizontal and vertical axes, respectively, satisfy:

[0112] Node continuity, i.e.:

[0113]

[0114] The continuity of the first and second derivatives, that is:

[0115]

[0116] in Indicates the first The first derivative. Indicates time The two moments before and after. Through the above interpolation process, the first... In the imaging plane of each sensor The set of two-dimensional time-series tracks of a target can be represented as:

[0117]

[0118] The first frame of the data pool Based on the multi-target pixel coordinates at time, for the first In the imaging plane of each sensor The two-dimensional time-series tracks of each target are sorted and numbered, and the specific sorting method is as follows.

[0119] First calculate the target Pixel coordinates in the first frame At the current moment, the center of the sensor's imaging plane Euclidean distance The expression is:

[0120]

[0121] in These represent the data in the data pool. The pixel x-coordinate and pixel y-coordinate of the center point of the sensor's imaging plane. Each goal From childhood to adulthood The tracks in the data are sorted and numbered to obtain an ordered set of tracks. The first element is assigned the number 1, the second element is assigned the number 2, and so on, with the last target assigned the number N.

[0122] Step S3 decomposes the problem of determining the identity of multiple targets among multiple sensors into the problem of determining the identity of multiple targets among multiple pairs of main and auxiliary sensors.

[0123] like Figure 5 As shown in the figure, the problem of determining the identity of multiple targets among four sensors is used as an example. This is a schematic diagram of the method to decompose the problem of determining the identity of multiple targets among multiple sensors into the problem of determining the identity of multiple targets among multiple pairs of main and auxiliary sensors. Figure 5 The left side of the image shows the two-dimensional tracks of multiple targets in the 2D plane of the sensors of four airborne platforms. The core problem is to establish a one-to-one correspondence between the tracks of different sensors. This method decomposes the multi-sensor determination problem into a "one master and multiple auxiliary" sensor pair determination mode. That is, airborne platform 1 is selected as the "master sensor", and airborne platforms 2, 3, and 4 are respectively regarded as "auxiliary sensors 1, 2, and 3". By constructing the track association relationship between the master sensor and each auxiliary sensor, the global determination problem between multiple sensors is transformed into the identity determination problem between multiple master-auxiliary sensor pairs. Figure 5 The dashed lines connecting the multiple target elements on the right side represent the association and combination of multiple targets among multiple sensors, with a total of 48 association and combination methods.

[0124] Step S4: Design a dual-criteria method for determining the identity of multiple targets between arbitrary main and auxiliary sensors. The first criterion is a spatiotemporal geometric constraint criterion, which is based on the epipolar geometric constraints of continuous multi-frames. It combines the imaging of multiple targets in different sensors to satisfy the epipolar constraints to achieve the identity determination of multiple targets and constructs a spatiotemporal geometric constraint identity determination matrix. The second criterion is a temporal trajectory feature constraint criterion, which is based on the cosine similarity of two-dimensional trajectory features. It combines the trajectory feature similarity of multiple targets in the imaging planes of different sensors to achieve the identity determination of multiple targets and constructs a trajectory feature constraint identity determination matrix.

[0125] The first criterion. The task of determining the identity of multiple targets essentially involves judging the correlation between multiple targets across different sensors, finding a one-to-one correspondence between the multiple targets observed by different sensors. If the correlation is correct, it indicates that they are the same target. Ideally, direction finding lines pointing to the same target from different carrier platforms should intersect at the target's three-dimensional spatial location point. However, due to measurement errors, the two direction finding lines often do not intersect and become skew lines, resulting in a shortest distance (i.e., the distance of the common perpendicular). The spatiotemporal geometric constraint criterion is a mathematical representation of this distance characteristic, providing a reliable basis for determining the correlation of multiple sensor targets from the perspective of physical geometric relationships.

[0126] At any moment Assuming the same target is on the main sensor and auxiliary sensors Corresponding observations in the imaging plane and ,sensor and The positions are respectively and By using the two-dimensional pixel coordinate observation data of the target in the sensor's two-dimensional imaging plane, the attitude of the carrier platform, and the sensor's intrinsic and extrinsic parameters, the value of the target in the world coordinate system, determined by the main sensor, is calculated. Pointing observation The unit direction finder vector is Similarly, the calculation of the auxiliary sensor in the world coordinate system is also obtained. Pointing observation The three-dimensional direction finding vector is Then the solution can be obtained from the sensor. and The distance between the common perpendiculars of the two direction finding lines pointing to the target is:

[0127]

[0128] in, The cross product of two direction finding vectors represents the common normal vector that is perpendicular to both direction finding lines. Let be the position difference vector between the two sensors. If the observation... and If they belong to the same goal, then It tends to be the minimum.

[0129] In continuous time intervals The mean constraint for calculating the distance between the common perpendiculars of the direction finding lines is expressed as:

[0130]

[0131] in, The length of the time interval. It represents a specific moment within a time interval. This indicates that all moments within the time interval are represented by the sensor. and The mean distance between the common perpendiculars of the two direction finding lines pointing to the target. This mean information integrates observations... The temporal and spatial constraints of associated targets can be expressed as observations. At any moment Spatiotemporal geometric constraint criterion ,Right now:

[0132]

[0133] Special criteria based on spatiotemporal geometric constraints Construct a spatiotemporal geometric constraint identity determination matrix, and the elements in the matrix... Multi-objective spatiotemporal geometric constraint feature similarity The definition is as follows:

[0134]

[0135] like Figure 6 The diagram illustrates a method for determining the spatiotemporal geometric constraint identity determination matrix based on spatiotemporal geometric constraint criteria. The diagram includes a motorized platform equipped with a main sensor and a motorized platform equipped with an auxiliary sensor. At a certain moment, the target observed by the main sensor can be calculated. The direction finding line and the main and auxiliary sensors point to the observed target. The common perpendicular between the direction finding lines By considering historical information, the mean of this common perpendicular over a certain period can be calculated. The main sensor is constructed based on this mean. and auxiliary sensors Spatiotemporal geometric constraint identity determination matrix of all targets in the imaging plane .

[0136] The second criterion. Since the motion of a target in spacetime is continuous and similar, its two-dimensional trajectory can also be considered as being composed of continuous two-dimensional point traces. That is, within a certain period of time, any target will form a continuous two-dimensional trajectory. The temporal trajectory feature constraint criterion refers to the motion trend of multiple targets in the sensor's two-dimensional imaging plane within a certain period of time, including motion direction features and motion displacement features.

[0137] In dynamic target tracking scenarios, temporal trajectory feature constraint criteria characterize the motion consistency of a target in a continuous time series. Let the sensor... At any moment The detected first The pixel coordinates of each target are In a continuous time interval Within it, its motion vector is defined as follows:

[0138]

[0139] in, Indicate target In the time interval The instantaneous motion vector, which contains two components, firstly the motion direction feature, can accurately represent the target's motion direction angle within this time interval, expressed as follows:

[0140]

[0141] Secondly, there is the motion displacement feature, which represents the Euclidean distance between adjacent frames, expressed as follows:

[0142]

[0143] Merging in time intervals The target's motion characteristics and direction of motion characteristics within the target area, for the target At any moment The temporal trajectory feature constraint criteria are represented as follows:

[0144]

[0145] Constraint Criteria Based on Temporal Track Features Construct a trajectory feature constraint identity determination matrix, and the elements in the matrix are... For multi-target track feature similarity The definition is as follows:

[0146]

[0147] in The two-dimensional track feature distance function is defined as follows:

[0148]

[0149] like Figure 7 The diagram illustrates a method for determining the identity determination matrix of track feature constraints based on temporal track feature criteria. The diagram includes four tracks in the 2D imaging plane of the main sensor and four tracks in the 2D imaging plane of the auxiliary sensor, at a certain moment... Extract the observed target from the main sensor Motion direction characteristics and motion displacement characteristics And the target observed in the auxiliary sensor Motion direction characteristics and motion displacement characteristics The main sensor is constructed based on two features. and auxiliary sensors The identity determination matrix of track features constrained by all targets in the imaging plane .

[0150] Step S5: Design a method for calculating the fusion weights of the spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix. Combining the first layer data in the data pool, calculate the line of sight from the optical center of any primary and secondary sensor to the center of their respective fields of view, and calculate the overlap point of the primary and secondary sensor fields of view. Draw a perpendicular line from the overlap point of the fields of view to the baseline of the primary and secondary sensors to obtain the foot of the perpendicular of the overlap point on the baseline. By calculating the positional relationship between the foot of the perpendicular of the overlap point and the center point of the current primary and secondary sensor combined baseline, evaluate the relationship between the overlap area of ​​the primary and secondary fields of view and the geometric configuration of the primary and secondary sensors, thereby adaptively determining the first fusion weight of the spatiotemporal geometric constraint identity determination matrix and the second fusion weight of the track feature constraint identity determination matrix.

[0151] At any moment The locations of the two sensors used to process the related tasks are respectively and Therefore, the baseline direction vector between the two sensors at that moment can be calculated as follows:

[0152]

[0153] The midpoint of the common perpendicular of the line of sight lines pointing from the optical centers of the primary and secondary sensors to their respective field-of-view centers at that moment is... , That is, the overlap point of the fields of view of the main and auxiliary sensors, from which the following can be calculated. At baseline The foot of the overlapping point of the field of view on and the midpoint of the baseline

[0154]

[0155] Geometric configuration parameters are set based on baseline length, perpendicular foot of field of view overlap point, and baseline midpoint. for:

[0156]

[0157] When geometry configuration parameters At this time, the initial pointing point of the target group is located inside the baseline region of the two correlated sensors, and the observed target has high field-of-view overlap and strong geometric projection stability. When the geometric configuration parameters... When the initial pointing point of the target group is located on the same side of the baselines of the two associated sensors, the geometric projection relationship gradually becomes blurred while the track similarity increases. However, when the initial pointing point of the target group is located on the same side of the baselines of the two associated sensors but deviates significantly, the geometric and track features are severely degraded. Therefore, a three-layer strategy is used to determine the first fusion weight of the spatiotemporal geometric constraint identity determination matrix and the second fusion weight of the track feature constraint identity determination matrix.

[0158] when At that time, the first fusion weight of the spatiotemporal geometric constraint identity determination matrix And the second fusion weight of the track feature constraint identity determination matrix The calculation method is as follows:

[0159]

[0160] when At that time, the first fusion weight of the spatiotemporal geometric constraint identity determination matrix And the second fusion weight of the track feature constraint identity determination matrix The calculation method is as follows:

[0161]

[0162] when At that time, the first fusion weight of the spatiotemporal geometric constraint identity determination matrix And the second fusion weight of the track feature constraint identity determination matrix The calculation method is as follows:

[0163]

[0164] Dynamically adjust the first fusion weight based on geometric configuration parameters And the second fusion weight of the track feature constraint identity determination matrix .like Figure 8 The diagram illustrates a method for adaptively fusion weights to determine the identity determination matrix by combining geometric configuration parameters. The diagram uses the data from frame 0 of each consecutive frame to calculate the direction finding lines from the main sensor to its own field of view center point and from the auxiliary sensor to its own field of view center point. From this, the overlap point of the main and auxiliary sensor fields of view at that moment is calculated. The perpendicular point of the overlap point to the baselines of the main and auxiliary sensors is then calculated, yielding the perpendicular point of the overlap point. The distance between the midpoint of the baseline and the perpendicular point of the overlap point is... The baseline length is The geometric configuration parameters are The first fusion weight is dynamically adjusted by determining the adaptive fusion weight design strategy through these geometric configuration parameters. and the second fusion weight .

[0165] In steps S6 and S7, the multi-target geometric constraint identity determination matrix and the multi-target trajectory feature constraint identity determination matrix are normalized and fused by combining the first fusion weight and the second fusion weight to obtain the multi-target dual-criteria identity determination matrix between the main and auxiliary sensors. The rows represent the two-dimensional temporal trajectories of all targets in the imaging plane of the main sensor, and the columns represent the two-dimensional temporal trajectories of all targets in the imaging plane of the auxiliary sensor. The Hungarian algorithm is used to solve the multi-target dual-criteria identity determination matrix between the main and auxiliary sensors, and the multi-target identity determination result between the main and auxiliary sensors is output. The multi-target identity determination results of all main-auxiliary sensor combinations in the multi-sensor system are statistically analyzed, and the average accuracy index is used to calculate the multi-target identity determination result between the multiple sensors.

[0166] Using the first fusion weight and the second fusion weight Before normalizing and fusing the identity determination matrix of multi-target geometric constraints and the identity determination matrix of multi-target track feature constraints, the identity determination matrix needs to be normalized. The normalization method is as follows:

[0167]

[0168] in This represents the normalized multi-objective geometric constraint identity determination matrix. This represents the normalized multi-target track feature constraint identity determination matrix. The above processing means that all elements in the identity determination matrix are divided by the element with the maximum value, thereby eliminating dimensional differences. After normalization and fusion, the multi-target dual-criteria identity determination matrix between the main and auxiliary sensors can be obtained. The specific solution is as follows:

[0169]

[0170] The Hungarian algorithm is used to solve the minimum weight maximum matching problem, that is, while ensuring that the number of matchings is equal to the minimum weight maximum matching problem. Minimize the total cost under the premise of [missing information]. Solving this matrix, the multi-objective identity determination problem can be formulated as:

[0171]

[0172] in The number of targets observed within the main sensor. Indicates the first observation within the main sensor One goal, To increase the number of targets observed within the auxiliary sensor, The first observation within the auxiliary sensor One goal, The weights for determining the identity of multiple targets between primary and secondary sensors are between 0 and 1. For identity determination decision variables, , This indicates that the target is not the same. This indicates that they are considered to be the same target.

[0173] like Figure 9 The diagram illustrates a method for obtaining multi-target identity determination results between multiple sensors based on a dual-criteria identity determination matrix between primary and secondary sensors. For scenarios where both primary and secondary sensors observe four targets, the normalized multi-target geometric constraint identity determination matrix and the normalized multi-target trajectory feature constraint identity determination matrix are fused using a first fusion weight and a second fusion weight to obtain a dual-criteria identity determination matrix. The Hungarian algorithm is then used to solve this matrix to obtain the identity determination results between any primary and secondary sensors for multiple targets. Figure 9 The results of the multi-target identity determination between a pair of primary and secondary sensors are shown as 1-1, 2-3, 3-2, 4-4. By analogy, the multi-target identity determination results between multiple sensors can be obtained.

[0174] The results of multi-target identity determination for all main-auxiliary sensor combinations in a multi-sensor system are statistically analyzed, and the average accuracy is used. The performance of the multi-sensor multi-target identity determination index is evaluated, and the average accuracy is defined as follows.

[0175]

[0176] in This represents the number of correct results in the multi-objective identity determination. This indicates the number of errors in the multi-objective identity determination result.

[0177] The aforementioned method for determining the identity of multiple targets using a multi-passive sensor approach with both spatiotemporal and trajectory constraints achieves several advantages. Firstly, by constructing spatiotemporal criteria based on continuous multi-frame epipolar geometric constraints and trajectory criteria based on cosine similarity of trajectory features, it can more accurately describe the identity determination characteristics of multiple targets across different sensors, thus significantly improving the accuracy and robustness of multi-target identity determination. Secondly, this method effectively solves the weight optimization problem under different sensor configurations and scene characteristics through an adaptive fusion weight mechanism, avoiding the performance degradation of fixed weight strategies in complex scenarios. This method possesses excellent scene adaptability and real-time response characteristics, fully meeting the rapid online processing requirements of resource-constrained airborne environments, while maintaining the flexibility and scalability of cross-platform deployment, providing reliable technical support for cutting-edge applications such as multi-aircraft collaborative sensing.

[0178] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0179] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0180] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors, characterized in that, include: Step S1: Construct a multi-layer data pool. Each layer of the data pool stores observation data from multiple engine platforms in frames. The multi-layer data pool is dynamically updated using a first-in-first-out strategy. Once the multi-layer data pool is filled with a set number of frames at a set depth, the identity determination task is started. Step S2: Based on all the observation data in the multi-layer data pool, spline interpolation fitting is performed on the pixel coordinates of each target in the imaging plane of each passive sensor for multiple consecutive frames to generate a two-dimensional time-series track of each target in each passive sensor. The two-dimensional time-series tracks of each target are sorted based on the distance from the pixel coordinates of each target in the first frame of the multi-layer data pool to the center of the imaging plane of the passive sensor to obtain an ordered track set. Step S3: Set one passive sensor as the main sensor and the rest as auxiliary sensors, and decompose the multi-target identity determination problem among multiple passive sensors into multiple sub-problems between main and auxiliary sensors. Step S4: For each pair of primary and secondary sensors, construct the spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix respectively. Step S5: Calculate the geometric configuration relationship between the primary and secondary sensors based on the observation data of the first layer in the multi-layer data pool, and adaptively determine the first fusion weight of the spatiotemporal geometric constraint identity determination matrix and the second fusion weight of the track feature constraint identity determination matrix. Step S6: Using the first fusion weight and the second fusion weight, the spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix are fused to obtain the dual-criteria identity determination matrix between the main and auxiliary sensors. The Hungarian algorithm is then used to solve the dual-criteria identity determination matrix between the main and auxiliary sensors to obtain the multi-target identity determination result between a pair of main and auxiliary sensors. Step S7: Statistically analyze the multi-target identity determination results of all primary and secondary sensors to obtain the global multi-target identity determination results of multiple passive sensors.

2. The method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors according to claim 1, characterized in that, Step S1 specifically includes: A multi-layer data pool is constructed, with each layer storing observation data from multiple onboard platforms frame by frame. The observation data includes the timestamps, 3D positions, attitude angles, orientation angles of the passive sensors mounted on the onboard platforms, and the pixel coordinates of all targets obtained by them. The depth of each data pool represents the number of frames of stored data. A first-in-first-out strategy is adopted, and the update of multiple data pools is completed after reading the next frame of observation data. If the observation data in the data pool has not reached the preset depth, then the identity determination preparation stage will begin. If the observation data in the data pool is full at the set depth, the multi-target identity determination task will be executed.

3. The method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors according to claim 2, characterized in that, Step S2 specifically includes: Based on all observation data within the multi-layer data pool, extract continuous pixel coordinate sequences for all targets; Based on the continuous pixel coordinate sequence of all targets, cubic spline interpolation is used to fit and generate two-dimensional time-series tracks for each target. Using the multi-target pixel coordinates of the first frame of the multi-layer data pool as a reference, the Euclidean distance from the pixel coordinates of each target to the center of the imaging plane of the passive sensor is calculated. The two-dimensional time-series tracks of each target are sorted and numbered according to the Euclidean distance from smallest to largest to obtain an ordered track set.

4. The method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors according to claim 3, characterized in that, The construction of the spatiotemporal geometric constraint identity determination matrix specifically includes: Based on the observation and attitude angles of the main sensor and the auxiliary sensor in the passive sensor imaging plane, as well as the internal and external parameters of the main sensor and the auxiliary sensor, the unit direction finding vector pointing from the main sensor to its corresponding observation and the three-dimensional direction finding vector pointing from the auxiliary sensor to its corresponding observation in the world coordinate system are obtained. Based on the unit direction finding vector of the main sensor pointing to its corresponding observation in the world coordinate system and the three-dimensional direction finding vector of the auxiliary sensor pointing to its corresponding observation, the common perpendicular distance between the two direction finding lines of the main sensor and the auxiliary sensor pointing to the target is obtained. The mean constraint of the common perpendicular distance between the two direction finding lines pointed to the target by the main sensor and the auxiliary sensor within a continuous time interval is obtained. The mean value constraint of the distance between the common perpendiculars of the direction finding lines is used as the criterion for spatiotemporal geometric constraints to construct a spatiotemporal geometric constraint identity determination matrix.

5. The method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors according to claim 4, characterized in that, The construction of the track feature constraint identity determination matrix specifically includes: Extract the instantaneous motion vector of the target within a continuous time interval; Decompose the motion direction characteristics and motion displacement characteristics from the instantaneous motion vector; By integrating the target motion characteristics and motion direction characteristics within a continuous time interval, the temporal trajectory characteristic constraint criteria of the target are represented. Based on the temporal trajectory feature constraint criterion, a trajectory feature constraint identity determination matrix is ​​constructed.

6. The method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors according to claim 5, characterized in that, Step S5 specifically includes: Based on the observation data of the first layer in the multi-layer data pool, the baseline direction vectors of the main and auxiliary sensors are calculated; The midpoint of the common perpendicular of the line of sight between the optical centers of the primary and secondary sensors pointing to the center of their respective fields of view is taken as the overlap point of the fields of view of the primary and secondary sensors. Combined with the baseline direction vectors of the primary and secondary sensors, the foot of the perpendicular and the midpoint of the baseline at this point are calculated. Calculate the geometric configuration parameters based on the baseline length and the distance between the perpendicular foot and the midpoint of the baseline. ; When geometry configuration parameters At that time, the first fusion weight of the spatiotemporal geometric constraint identity determination matrix The second fusion weight of the identity determination matrix of track feature constraints The calculation method is as follows: When geometry configuration parameters At that time, the first fusion weight of the spatiotemporal geometric constraint identity determination matrix The second fusion weight of the identity determination matrix of track feature constraints The calculation method is as follows: When geometry configuration parameters At that time, the first fusion weight of the spatiotemporal geometric constraint identity determination matrix The second fusion weight of the identity determination matrix of track feature constraints The calculation method is as follows: 。 7. The method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors according to claim 6, characterized in that, Step S6 specifically includes: The spatiotemporal geometric constraint identity determination matrix and the track feature constraint identity determination matrix are normalized. The normalized spatiotemporal geometric constraint identity determination matrix and the normalized track feature constraint identity determination matrix are fused using the first fusion weight and the second fusion weight to obtain the dual-criteria identity determination matrix between the main and auxiliary sensors. Based on the dual-criteria identity determination matrix between the main and auxiliary sensors, the Hungarian algorithm is used to solve the minimum weight maximum matching problem to obtain the multi-target identity determination result between the main and auxiliary sensors.

8. The method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors according to claim 7, characterized in that, The expression for the dual-criteria identity determination matrix between the primary and secondary sensors is: in, This is the normalized spatiotemporal geometric constraint identity determination matrix. This is the normalized track feature constraint identity determination matrix; Solving for the identity determination matrix between the primary and secondary sensors based on dual criteria, the multi-target identity determination problem can be formulated as follows: in, The number of targets observed within the main sensor. Indicates the first observation within the main sensor One goal, To increase the number of targets observed within the auxiliary sensor, The first observation within the auxiliary sensor One goal, The weights for determining the identity of multiple targets between primary and secondary sensors are between 0 and 1. For identity determination decision variables, , This indicates that the target is not the same. This indicates that they are considered to be the same target.

9. The method for determining the identity of multiple targets under spatiotemporal and trajectory dual constraints using multiple passive sensors according to claim 8, characterized in that, Step S7 specifically includes: Statistically analyze the multi-target identity determination results of all primary and secondary sensors, and use the average accuracy. The evaluation metrics assess the effectiveness of multi-target identity determination among multiple sensors to obtain a global multi-target identity determination result; the average accuracy rate is: In the formula, This represents the number of correct results in the multi-objective identity determination. This represents the number of errors in the multi-objective identity determination result.

Images