A multi-source target fusion method based on multi-modal detection information

Abstract

The application provides a multi-source target fusion method based on multi-modal detection information, which comprises the following steps: firstly, acquiring multi-source heterogeneous sensing data and performing space-time unified pretreatment; judging the dynamic and static state according to the target position change; performing single-source double-time cross calibration on the static target, using the azimuth angle, the pitch angle and the sensor displacement of the adjacent two time points, obtaining the calibrated position through triangulation and confidence weighting, and performing cross-source association with type constraint; for the dynamic target, the correlation matrix is generated by using recursive trajectory similarity calculation and density hierarchical clustering; the association cluster is identified by depth-first search, and the robust fusion is performed by using the random sample consensus method, and high-precision target information is output. The application is suitable for multi-source heterogeneous detection information fusion of air / ground manned / unmanned detection platforms in land confrontation environment, and can generate high-confidence and anti-deception target intelligence to support command decision.

Description

Technical Field

[0001] This invention belongs to the field of multi-modal detection information and multi-source heterogeneous data fusion technology, specifically involving a multi-source target fusion method based on multi-modal detection information, which is applicable to collaborative detection by various detection platforms in complex land environments. Background Technology

[0002] In modern land-based confrontation environments, the environmental situation exhibits nonlinear and rapidly evolving characteristics. With the increase in dynamic countermeasures and jamming methods, the dynamic coupling and spatiotemporal alignment of multi-source heterogeneous detection data (such as electro-optical, radar, and acoustic data) face severe challenges. Multi-domain collaborative missions require effective integration of detection information from various manned and unmanned detection platforms, including airborne and ground-based systems, to promptly and accurately grasp the opponent's dynamics, precisely predict their intentions, and gain a deep understanding of the overall situation. Currently, the multimodal heterogeneous information acquired by various detection platforms contains a large amount of redundant, contradictory, and false information due to factors such as inconsistent sampling frequencies, inconsistent data formats, signal attenuation and distortion, and active interference from the opponent. Furthermore, existing single-sensor / platform-based data accumulation processing models lack data cleaning mechanisms and cross-modal target correlation capabilities under adversarial conditions, resulting in significant time delays and judgment biases in intention prediction and threat assessment.

[0003] Existing technologies include several multi-source target fusion methods. For example, CN118247693A discloses a data processing method based on the separation of dynamic and static targets. However, this method only manages different attention queues for dynamic and static targets, without performing precise position calibration for static targets or deep association of dynamic targets. Similarly, CN103617163B discloses a target association method based on clustering analysis, but this method is mainly designed for general multi-sensor scenarios and does not consider the high-precision positioning requirements of static targets and the differentiated processing of dynamic and static targets in land-based combat environments.

[0004] Therefore, there is a need to develop a highly robust intelligent processing method that can adapt to complex land-based combat scenarios, especially a method that can perform online self-calibration of static targets. Summary of the Invention

[0005] The purpose of this invention is to solve the challenge of fusing multi-source heterogeneous target information in land exploration. Addressing issues such as redundancy and inconsistency in multimodal exploration data under complex environments, this invention provides a multi-source target fusion method based on multimodal exploration information. This method establishes a standardized access mechanism for multi-source heterogeneous exploration data, seamlessly integrating real-time task scenario information collected by various sensors such as photoelectric, radar, and acoustic sensors, effectively eliminating redundancy, contradictions, and false information. It constructs association models for both dynamic and static targets, achieving accurate association and matching of heterogeneous exploration data. Specifically, for static targets, it proposes a self-calibration method based on single-sensor time-series observations, significantly improving the positioning accuracy of static targets without relying on external reference stations. A robust fusion method is employed to reduce the impact of outliers, generating high-confidence, high-completeness target exploration intelligence, providing a reliable basis for decision-making. This invention focuses on improving the accuracy and timeliness of target fusion information, strengthening environmental situational awareness capabilities, and meeting the urgent needs of intelligent processing of multi-source exploration data in complex environmental tasks.

[0006] The technical solution of this invention is as follows:

[0007] A multi-source target fusion method based on multimodal detection information is proposed. This method is applied to a detection platform equipped with multiple heterogeneous sensing devices / payloads. By integrating the platform to obtain multi-source heterogeneous information and unifying the data format, it achieves autonomous association and efficient fusion of multiple targets. The method includes the following steps:

[0008] Step 1: Acquire multi-source heterogeneous sensing data collected by at least two detection devices, wherein the detection devices integrate one or more detection payloads;

[0009] Step 2: Perform spatiotemporal unified preprocessing on the multi-source heterogeneous sensing data, and determine whether the target is a dynamic target or a static target based on the change in the target position over time;

[0010] Step 3: For targets determined to be static, perform single-source dual-time cross-calibration to obtain the calibrated static target position, and perform cross-source correlation based on the calibrated position to generate a static target correlation matrix;

[0011] Step 4: For targets identified as dynamic, perform association matching based on trajectory similarity to generate a dynamic target association matrix;

[0012] Step 5: Based on the static target correlation matrix and the dynamic target correlation matrix, obtain the associated target clusters, and perform robust fusion of the multi-source observations in each cluster to output the fused target information.

[0013] In a further preferred embodiment, the method for determining the target's dynamic or static state in step 2 includes:

[0014] Calculate the change in difference before and after

[0015]

[0016] in The time window length, To detect equipment The detection payload mounted on the top exist The target position vector obtained from the sensing data at any given moment, after undergoing spatiotemporal unification; if Then the target is static; if The target is dynamic; These are the dynamic and static thresholds.

[0017] In a further preferred embodiment, the single-source dual-time cross-calibration in step 3 specifically includes:

[0018] Acquiring the same probe payload at two adjacent time points and Observational data for the same static target, including azimuth angle. , Pitch angle: , And the geocentric coordinates of the probe itself at the corresponding moment. and ;

[0019] Based on the principle of triangulation, the azimuth difference, elevation difference, and displacement of the probe payload at two different times are used to calculate the target's position relative to the x-axis. The coordinates of the equipment at all times are in the northeast-central coordinate system with the origin as the coordinate. and

[0020] Based on the confidence level of the probe payload output , Weighted fusion is performed to obtain the calibrated target position. .

[0021] A further preferred embodiment is that the calculation formula based on the triangulation principle is:

[0022]

[0023]

[0024] in To detect the projected distance of the equipment's position on the horizontal plane at two different times, The angle between the projection line and the north coordinate axis.

[0025] A further preferred embodiment is that the weighted fusion calculation formula is as follows:

[0026]

[0027] in This is the rotation matrix from the Earth-centered Earth-fixed coordinate system to the Northeast-Sky coordinate system.

[0028] In a further preferred embodiment, the cross-source association in step 3 includes:

[0029] Calculate the static target association confidence:

[0030]

[0031] in and The targets are respectively and target Category It is a spatial distance scaling factor; and The target and target The final calibration location; and Each is for the target and target Perceived data; This is an indicator function that takes the value 1 when the condition is true and 0 otherwise.

[0032] Generate the association matrix of static targets based on the association confidence of static targets. :

[0033]

[0034] in, It is a preset static correlation threshold.

[0035] In a further preferred embodiment, the dynamic target association matching in step 4 includes:

[0036] Calculate trajectory similarity

[0037]

[0038] in It is the Euclidean distance:

[0039]

[0040] A density-based hierarchical clustering algorithm is used to cluster the similarity matrix to generate a dynamic target association matrix.

[0041] In a further preferred embodiment, the robust fusion in step 5 employs a random sampling consensus method, specifically including:

[0042] For each associated target cluster Randomly select from them Calculate the initial fusion position from each observation:

[0043]

[0044] Calculate the residuals of all observations within the cluster. The residual modulus is less than the threshold. The observed values ​​are marked as interior points;

[0045] Repeat the above process multiple times, select the sample set with the largest number of interior points, and recalculate the fusion position using all interior points in the sample set as the final output.

[0046] Beneficial effects

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. Differentiated processing of static and dynamic targets: By judging the static and dynamic states of the target, static targets and dynamic targets are separated, and different association and calibration strategies are adopted for each, which avoids mutual interference between static and dynamic targets and improves the association accuracy.

[0049] 2. Static target single-source dual-time cross-calibration: By observing the difference in azimuth and elevation angles of the same sensor at two adjacent times, and combining the sensor's own position changes, the online self-calibration of the static target position is achieved through the principle of triangulation. This does not require external reference stations or cooperative targets, and significantly improves the positioning accuracy of static targets.

[0050] 3. Confidence-based weighted fusion: The confidence level of the sensor output is introduced as a weighting coefficient during the calibration process, so that high-confidence observations have a greater weight in the calibration results, thereby improving the reliability of the calibration.

[0051] 4. Static target association with type constraints: In static target cross-source association, target type consistency constraints are introduced, and combined with the exponential decay model of spatial distance, which effectively distinguishes different types of targets and reduces the false alarm rate.

[0052] 5. Robust fusion mechanism: The random sampling consensus method is used to remove outliers in the associated clusters, which can maintain the stability of the fusion results even under strong interference environment.

[0053] In summary, this invention provides a high-precision and robust solution for the fusion of multi-source heterogeneous detection information in complex land-based combat environments.

[0054] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0055] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0056] Figure 1 This is a flowchart of a multi-source target fusion strategy method based on multimodal detection information.

[0057] Figure 2 It is a simulated trajectory diagram of a target perceived by a sensor.

[0058] Figure 3 It is a trajectory map after fusing multi-source heterogeneous sensing information of the target. Detailed Implementation

[0059] The embodiments of the present invention are described in detail below. These embodiments are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0060] This invention provides a multi-source target fusion method based on multimodal detection information, applicable to land-based confrontation scenarios. Specifically, when integrating heterogeneous information such as photoelectric images, radar traces, electronic reconnaissance data, and acoustic signals acquired from various manned and unmanned aerial and ground-based platforms, including high-altitude reconnaissance aircraft, UAVs, and ground-based reconnaissance vehicles, this method effectively addresses data corruption caused by differences in sampling frequencies, data formats, signal attenuation, and strong electromagnetic interference from the opposing side. By performing unified spatiotemporal registration, feature extraction, and cross-validation on this multimodal information, accurate association and fusion of multiple real targets within the same area are achieved, effectively eliminating redundancy and false information.

[0061] The technical process of the method of the present invention is as follows: Figure 1 As shown, the process includes: determining multi-source heterogeneous detection information, preprocessing sensing data, spatiotemporal unification of target information, dynamic and static state judgment, single-source calibration and cross-source association of static targets, similarity calculation and cluster association of dynamic target trajectories, identification of associated target clusters, and random sampling consistency fusion. Each step is described in detail below with reference to specific embodiments.

[0062] Step 1: Determine the accessed multi-source heterogeneous sensing data:

[0063] Depending on the needs of the detection scenario, the distributed detection implementation method includes two approaches: "area selection" or "payload selection".

[0064] "Area Selection" mode: In response to mission requirements, key detection areas are delineated on the map visualization interface, and at least two detection devices are automatically dispatched to conduct collaborative detection of the area, and multi-source sensing data is transmitted back in real time.

[0065] "Payload Selection" method: Based on the characteristics of the detection mission requirements, select multiple detection equipment that need to be given priority from the detection equipment resource pool to form a distributed detection network.

[0066] The detection equipment integrates one or more detection payloads, including but not limited to photoelectric, radar, and acoustic payloads. During detection missions, at least two of the detection payloads in the equipment operate simultaneously. The target perception information detected by the detection payloads is recorded as follows: ,in, It is a detection device. It refers to the type of payload mounted on the detection equipment. The included sensing information includes: target identification, time, type, source equipment, source location, target location, azimuth, elevation, friend or foe attributes, and confidence level.

[0067] Step 2: Preprocess multi-source heterogeneous sensing data, unify representation, and determine target state:

[0068] The system performs spatiotemporal unified preprocessing on the accessed multi-source heterogeneous sensing information, distinguishes between dynamic and static targets, and provides a unified data format.

[0069] S21, Time Alignment:

[0070] Each detection device uses the BeiDou timing signal as a reference to synchronize detection data with different sampling times and frequencies. Registration Time satisfy ,in, The latest moment in the current returned data packet. This is the latest time in the current data packet being sent back.

[0071] Select a certain time range before registration time Each internal detection payload senses data According to the registration time A data fitting method was used to unify the high-precision observation data of each probe payload to a unified time. The fitting process is as follows:

[0072]

[0073] in, For detection equipment The detection payload mounted on the top exist Data at any given moment. The summation index variable represents the time window. A discrete point in time within the time frame. from Traversing to Used to accumulate each time step The corresponding interpolation basis function multiplied by the observed data at that moment . For the product index variable, in relation to each fixed... In product operations, Traversal All time points within all other time windows.

[0074] S22, Spatial Alignment:

[0075] The target location information acquired by each detection payload and the equipment's own positioning information are uniformly converted to the Earth-centered Earth-fixed coordinate system (ECEF) to achieve spatial domain standardization of multi-source heterogeneous data, establishing a consistent spatiotemporal reference for subsequent fusion processing. Latitude and longitude Earth-centered Earth-fixed Coordinates The conversion formula is as follows:

[0076]

[0077] in, Longitude Latitude It refers to altitude. Let be the radius of curvature. Earth's eccentricity:

[0078]

[0079] in, The semi-major axis of the Earth's ellipsoid. It is the minor semi-axis of the Earth's ellipsoid.

[0080] S23. Determining whether an attribute is dynamic or static:

[0081] According to time range The data within the target area is used to determine the target's dynamic and static attributes using the difference method. The calculation method is as follows:

[0082]

[0083] in, This is a static / dynamic attribute, where 0 indicates static and 1 indicates dynamic. For dynamic and static thresholds, The change in the difference before and after:

[0084]

[0085] in To detect equipment The detection payload mounted on the top exist The target position vector obtained from the sensing data at any given moment after undergoing spatiotemporal unification.

[0086] S24. Data format is standardized:

[0087] After completing the temporal and spatial registration of multi-source heterogeneous sensing data, the target information format of different detection payloads is unified, denoted as... ,in The target number is used. The data format is shown in Table 1.

[0088] Table 1. Unified Data Format

[0089]

[0090] Step 3: Perform single-source calibration and correlation on static multi-source heterogeneous target information.

[0091] For targets determined to be static, single-source dual-time cross-calibration is performed to obtain the calibrated static target positions. Based on these calibrated positions, cross-source correlation is then performed to generate a static target correlation matrix. The size of the matrix is , It refers to the number of static targets.

[0092] S31. Single-Source Dual-Time Cross-Calibration: During the detection of static targets by the same sensor, due to inherent system errors, environmental interference, and platform motion, deviations may occur between the detected static target positions and the actual positions at different times and locations. To improve the accuracy of multi-source fusion, position calibration of the static target observation data acquired by a single sensor is required. This step utilizes the observation differences between two adjacent times using the same sensor for online calibration. Specifically, a single-source calibration model is established by combining real-time attitude compensation and georegistration algorithms to achieve accurate correction of the spatial position of the static target and ensure the consistency of cross-sensor data correlation.

[0093] Let the current time be The previous moment was Acquire the same sensor (detection payload) at and Observational data of the same static target at any given time:

[0094] The azimuth angle of the target observed by the sensor: , ;

[0095] The elevation angle of the target observed by the sensor: , ;

[0096] The sensor's own geocentric coordinates: , ;

[0097] Sensor elevation: , ;

[0098] Confidence level: , ;

[0099] First, transform the sensor coordinates to... The time sensor is positioned in a northeast-central coordinate system with the origin at the top. Define the rotation matrix. Convert geocentric coordinates to northeast-sky coordinates:

[0100]

[0101] in yes The longitude and latitude of the time sensor.

[0102] but The coordinates of the time sensor in the northeast-northeast coordinate system with the sensor position at time T as the origin are:

[0103]

[0104] calculate Time sensor position Projection point on the horizontal plane arrive Time sensor position distance :

[0105]

[0106] Calculate projection points and Time sensor position The angle between the line forming the north coordinate axis :

[0107]

[0108] Based on the principle of triangulation, select Time and The sensing data at different times, combined with the azimuth, elevation, and laser detection range of the same sensor observing a static target at different times and locations, are used to calculate the target's position in the distance. Time and The coordinates of the static target in the northeast-northeast coordinate system at a given time. First, calculate the coordinates of the static target in the northeast-northeast coordinate system. The time sensor position is in the northeast-northeast coordinate system with the origin. Coordinates of time:

[0109]

[0110] Recalculate the static target in the following way The time sensor position is in the northeast-northeast coordinate system with the origin. Coordinates of time:

[0111]

[0112] The calculation results at the two time points are weighted and fused using confidence levels to obtain the final calibrated position of the static target. :

[0113]

[0114] In actual calculations, if the absolute value of the azimuth difference... Less than If the value is in radians, then skip the calibration and use the observation from the previous moment as the calibration value for the current moment.

[0115] S32. Cross-Source Association: Due to differences in the volume, shape, and radar scattering characteristics of different static targets (such as vehicles and gas stations, airports, buildings, etc.), the echo center of the sensor will exhibit varying degrees of distance deviation from the actual geometric center when observed from different azimuth angles. To accurately characterize this phenomenon, the impact of target type characteristics on positioning accuracy must be comprehensively considered when constructing the static target association confidence score. The static target association confidence score is calculated as follows:

[0116]

[0117] in, and The targets are respectively and target Category It is a spatial distance scaling factor; and The target and target The final calibration location; and Each is for the target and target Perceived data; This is an indicator function; it takes the value 1 if the condition is true, and 0 otherwise.

[0118] Generate the association matrix of static targets based on the association confidence of static targets. :

[0119]

[0120] in, It is a preset static correlation threshold.

[0121] Step 4: Perform spatiotemporal correlation processing on dynamic multi-source heterogeneous target information:

[0122] For targets identified as dynamic, trajectory similarity calculation and cluster analysis are used for multi-dimensional feature matching to establish a dynamic target association matrix. The size of the matrix is , It refers to the number of dynamic targets.

[0123] S41: Trajectory similarity calculation. This is performed over a certain time range. Data on dynamic targets within the target area are used to calculate the similarity of target trajectories for preliminary association. Assuming a time range... Inside, numbered and The perceived target data are respectively denoted as and Similarity is calculated recursively:

[0124]

[0125] in, The similarity of the trajectory associations. It is the Euclidean distance:

[0126]

[0127] The boundary condition for recursive computation is: when hour, The similarity at the start of the time window is directly taken as the Euclidean distance at the current time.

[0128] S42. Clustering Association: A density-based spatial clustering algorithm is used to match target attributes and target states in multi-source sensing data. The local density of each target observation is calculated. and minimum distance :

[0129]

[0130]

[0131] in, For the set similarity threshold, This is an indicator function.

[0132] Based on density threshold and distance threshold Select density peak point (satisfy and As the initial cluster center, clusters with similarity higher than [a certain value] are gradually merged. Clusters are formed to create clustering results. .

[0133] Generate an association matrix of dynamic targets based on the clustering results. :

[0134]

[0135] Step 5: Fusion of multi-source related target information:

[0136] Based on the correlation matrices of dynamic and static targets, clusters of associated targets are obtained, and random sampling consistency is further utilized to reduce the impact of outliers on the fusion results.

[0137] S51: Associated target clusters. Based on the dynamic target association matrix. and static target association matrix The final correlation matrix of the multi-source heterogeneous sensing data was calculated. The size of the matrix is , Indicates the target quantity.

[0138]

[0139] During the fusion process, potential clusters of related targets are identified through the correlation matrix. Each cluster represents a group of interconnected target observations, corresponding to different sensor observations or multiple time-stamp trajectories of the same real target. The essence of a cluster is the connected components of the correlation matrix. All connected subgraphs of the correlation matrix are found using depth-first search, calculated as follows:

[0140]

[0141] in, It is a depth-first search recursive function used in the association matrix. Finding nodes All nodes belonging to the same connected component, i.e., the associated target cluster, whose input is the node number. The output contains nodes. and the set of all its connected nodes; For nodes The access flag, 0 for not accessed, 1 for accessed; Represents a node With nodes There are correlations in the correlation matrix, which are different observations or trajectory segments belonging to the same real target; This indicates that for each node... Directly related nodes recursive call Find its connected nodes and take the union of all results.

[0142] After completing the depth-first search, a set of clusters of related targets can be obtained. ,in This represents the set of associated target clusters, obtained by depth-first search (DFS) from the association matrix. The set of all connected components delineated from the data. Each connected component corresponds to a real physical target (composed of observations from multiple sensors or trajectory segments from multiple moments). This is the cluster index (serial number), ranging from 1 to... ; The total number of clusters is the number of connected components found through depth-first search, which is equal to the number of targets in the final fusion output (each cluster is fused into one target).

[0143] S52: Random Sample Consistency Fusion. In target association clusters, some observations may be outliers due to sensor noise, association errors, or target maneuverability. A robust fusion method based on random sample consistency is used to reduce the impact of these outliers on the final fusion result.

[0144] From clusters Random selection One associated target value ( ), calculate the initial fusion position :

[0145]

[0146] in, It is a cluster The Middle Location information of each observed target.

[0147] Computational clusters The residuals of all observations :

[0148]

[0149] like , If the preset interior point residual threshold is used, then the first... Each observed target is denoted as an inlier. The above random sampling process is repeated multiple times, and the sample set with the largest number of inliers is selected. Using all the inliers in this sample set, the fusion position is recalculated according to formula (22) to obtain the optimal fusion position. .

[0150] final, The related clusters merged into The target information is 1, and the format of the fused target information is shown in Table 2.

[0151] Table 2 Format of Target Data After Fusion

[0152]

[0153] To illustrate the superiority of the method proposed in this invention, this embodiment constructs a high-fidelity, quantifiable, and evaluable simulated dynamic land environment. The simulation system is pre-defined with a mission scenario, including simulated targets, and is assigned diverse and variable motion modes. The simulation system integrates multiple heterogeneous typical detection payloads, configured on different air / ground and manned / unmanned platforms, to conduct cooperative cross-detection of targets. Non-ideal observation characteristics are injected into each payload: significantly different asynchronous sampling frequencies are set, and errors consistent with their physical mechanisms are introduced. Specifically, we deployed a high-altitude unmanned aerial vehicle (UAV) (equipped with synthetic aperture radar and an electro-optical pod), a ground detection vehicle (equipped with millimeter-wave radar and acoustic sensors), and a handheld detection device (equipped with an infrared imager). These platforms conduct cooperative detection of multiple targets in the same area, including: a static enemy radar station (static non-motorized target) and a moving enemy armored vehicle convoy (dynamic target). The sampling frequencies of each platform are different (10Hz for UAV radar, 2Hz for electro-optical, 5Hz for ground radar, and 1Hz for handheld device), the data formats are different, and there is strong electromagnetic interference causing observation noise and false points.

[0154] First, the multi-source heterogeneous sensing data to be accessed is determined. Using a "region selection" method, a rectangular area encompassing the activity range of the radar station and armored vehicle convoy is delineated on the command map. The system automatically schedules the three detection platforms to conduct coordinated detection of this area, transmitting sensing data back in real time. Data packets transmitted by each platform... It includes target number, timestamp, type (e.g., "radar station"), source equipment ID, source location (latitude, longitude and elevation), target location (polar coordinates or geocentric coordinates relative to the sensor), azimuth, elevation, target identification attributes (identified as "other party"), and confidence level.

[0155] Then, data preprocessing and dynamic / static analysis are performed, including:

[0156] Time alignment: Based on BeiDou time service, select the current registration time. Set time window Seconds (including data within the first 2 seconds). The raw data for each payload are Lagrange interpolated according to formula (1) to unify all observation data to... time.

[0157] Spatial alignment: Convert the target positions (polar coordinates or latitude and longitude) reported by each load into coordinates in the geocentric coordinate system according to formulas (2) and (3). .

[0158] Dynamic / Static Judgment: For each target, calculate its dynamic / static status. Change in position within the time window For example, for radar station targets, its meters per second (caused by sensor noise), set a threshold. meters per second, then The condition is determined to be static; for tanks in an armored vehicle convoy, their... Meters per second, is considered dynamic.

[0159] For static targets, a single-source, dual-time cross-calibration is first performed, taking the observation of a radar station by an UAV's electro-optical payload as an example. The current time is taken. and the previous moment Observational data:

[0160] Time, azimuth Pitch angle UAV coordinates Confidence level .

[0161] Time, azimuth Pitch angle UAV coordinates Confidence level .

[0162] Calculate the rotation matrix and relative displacement using formulas (6)-(9). and included angle The target coordinates in the northeast-northeast coordinate system are calculated according to formulas (10) and (11). The confidence-weighted fusion is performed according to formula (12) to obtain the calibrated ground-centered and ground-fixed coordinates of the radar station. After calibration, the error between the radar station position and the pre-mapped known position is reduced from about 15 meters in the original observation to about 3 meters.

[0163] Finally, cross-source correlation is performed: assuming there is another radar station target detected by ground radar in the same area, also of type "radar station". The correlation confidence between the two observations is calculated using formula (13). If it is greater than the threshold In the static correlation matrix The two observations are marked as related.

[0164] For dynamic targets, the trajectory similarity is calculated for each armored vehicle's dynamic observation data using formulas (15) and (16). For example, the recursive similarity between the trajectory of armored vehicle A observed by the UAV radar and the trajectory of armored vehicle A observed by the ground radar is relatively high, while the similarity with the trajectory of armored vehicle B is relatively low. The local density and minimum distance are calculated using formulas (17) and (18), and the density peak point is selected as the cluster center. After clustering, three clusters are obtained, each corresponding to one of the three armored vehicles. The dynamic target association matrix is ​​generated using formula (19). .

[0165] Then, the fusion is performed: First, the overall correlation matrix is ​​constructed according to formula (20). Depth-first search yielded three dynamic target clusters (corresponding to three armored vehicles) and one static target cluster (corresponding to a radar station).

[0166] Then, RANSAC fusion was performed on each cluster. Taking the armored vehicle cluster A as an example, this cluster contains four observations: UAV radar, UAV electro-optical, ground radar, and handheld infrared. Random selection... Calculate the initial fusion position for each observation, calculate the residuals for each observation, and set the interior point threshold. Meters. Iteration After that, the sample set with the most interior points was selected, and a obviously deviated false point was removed. The fused position was recalculated using the remaining 3 interior points to obtain the final high-precision armored vehicle position.

[0167] The final output is the fused target information in Table 2 format: 3 dynamic targets and 1 static target, whose positional accuracy is significantly higher than that of any single sensor.

[0168] Figure 2 The simulation results show the trajectory diagrams (including noise and outliers) of four heterogeneous sensors detecting dynamic targets. Figure 3 The target trajectory diagram after fusion using the method of this invention is given. Figure 2 and Figure 3 The results show that the fused trajectory is basically consistent with the target's true trajectory, effectively eliminating noise and false points.

[0169] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the present invention.

Claims

1. A multi-source target fusion method based on multimodal detection information, characterized in that: Includes the following steps: Step 1: Acquire multi-source heterogeneous sensing data collected by at least two detection devices, wherein the detection devices integrate one or more detection payloads; Step 2: Perform spatiotemporal unified preprocessing on the multi-source heterogeneous sensing data, and determine whether the target is a dynamic target or a static target based on the change in the target position over time; Step 3: For targets determined to be static, perform single-source dual-time cross-calibration to obtain the calibrated static target position, and perform cross-source correlation based on the calibrated position to generate a static target correlation matrix; Step 4: For targets identified as dynamic, perform association matching based on trajectory similarity to generate a dynamic target association matrix; Step 5: Based on the static target correlation matrix and the dynamic target correlation matrix, obtain the associated target clusters, and perform robust fusion of the multi-source observations in each cluster to output the fused target information.

2. The multi-source target fusion method based on multimodal detection information according to claim 1, characterized in that: The method for determining the target's dynamic or static state in step 2 includes: Calculate the change in difference before and after in The time window length, To detect equipment Upper-mounted detection payload exist The target position vector obtained from the sensing data at any given moment, after undergoing spatiotemporal unification; if Then the target is static; if The target is dynamic; These are the dynamic and static thresholds.

3. The multi-source target fusion method based on multimodal detection information according to claim 1, characterized in that: The single-source dual-time cross-calibration in step 3 specifically includes: Acquiring the same probe payload at two adjacent time points and Observational data for the same static target, including azimuth angle. , Pitch angle: , And the geocentric coordinates of the probe itself at the corresponding moment. and ; Based on the principle of triangulation, the azimuth difference, elevation difference, and displacement of the probe payload at two different times are used to calculate the target's position relative to the x-axis. The coordinates of the equipment at all times are in the northeast-central coordinate system with the origin as the coordinate. and Based on the confidence level of the probe payload output , Weighted fusion is performed to obtain the calibrated target position. .

4. The multi-source target fusion method based on multimodal detection information according to claim 3, characterized in that: The calculation formula based on the principle of triangulation is as follows: in To detect the projected distance of the equipment's position on the horizontal plane at two different times, The angle between the projection line and the north coordinate axis.

5. The multi-source target fusion method based on multimodal detection information according to claim 4, characterized in that: The formula for calculating the weighted fusion is as follows: in It is the rotation matrix from the Earth-centered Earth-fixed coordinate system to the Northeast-Sky coordinate system.

6. The multi-source target fusion method based on multimodal detection information according to claim 1, characterized in that: The cross-source association in step 3 includes: Calculate the static target association confidence: in and The targets are respectively and target Category It is a spatial distance scaling factor; and The target and target The final calibration location; and Each is for the target and target Perceived data; This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. Generate the association matrix of static targets based on the association confidence of static targets. : in, It is a preset static correlation threshold.

7. The multi-source target fusion method based on multimodal detection information according to claim 1, characterized in that: The dynamic target association matching in step 4 includes: Calculate trajectory similarity in It is the Euclidean distance: A density-based hierarchical clustering algorithm is used to cluster the similarity matrix to generate a dynamic target association matrix.

8. The multi-source target fusion method based on multimodal detection information according to claim 1, characterized in that: The robust fusion in step 5 employs a random sample consensus method, specifically including: For each associated target cluster Randomly select from them Calculate the initial fusion position from each observation: Calculate the residuals of all observations within the cluster. The residual modulus is less than the threshold. The observed values ​​are marked as interior points; Repeat the above process multiple times, select the sample set with the largest number of interior points, and recalculate the fusion position using all interior points in the sample set as the final output.

Images