A method, system, terminal and medium for low-altitude heterogeneous sensor signal fusion
By using a sequential extended Kalman filter framework and Jacobian matrix calculation, the problem of inconsistent observation dimensions in low-altitude heterogeneous sensor data fusion was solved, and the collaborative processing of three-dimensional distance constraints and two-dimensional angle information was realized, thereby improving the stability and accuracy of state estimation.
Patent Information
- Application Number
- CN202611585007.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-14
- Publication Date
- 2026-11-10
AI Technical Summary
In existing technologies, the data fusion schemes of low-altitude heterogeneous sensors cannot directly construct extended-dimensional observation matrices due to inconsistent observation dimensions, mismatched covariance matrix dimensions, and the inability of pure angle observations from photoelectric sensors to uniquely determine the three-dimensional spatial position of the target, resulting in the failure of the fusion structure.
A sequential extended Kalman filter framework is adopted. By acquiring the perception results of radar and photoelectric sensors, timestamp alignment and coordinate system transformation are performed. The equivalent measurement covariance matrix is calculated using the Jacobian matrix, and correlation matching and two-step filtering update are performed to achieve dimensionless filtering fusion of heterogeneous information.
It achieves collaborative processing of three-dimensional distance constraints and two-dimensional angle information, solves the technical obstacle of inconsistent observation dimensions, ensures the stability and accuracy of system state estimation, and supports global optimal allocation and continuous track tracking in multi-target scenarios.
Smart Images

Figure CN123069299A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of signal filtering and fusion technology, specifically relating to a low-altitude heterogeneous sensor signal fusion method, system, terminal, and medium. Background Technology
[0002] With the development of the urban low-altitude economy, low-altitude airspace surveillance and multi-sensor data fusion technologies play a crucial role in low-altitude flight service platforms. To effectively monitor low-altitude targets, the system typically deploys both low-altitude surveillance radar and optoelectronic equipment for collaborative detection. The radar primarily provides three-dimensional target position information, including radial distance, while optoelectronic sensors are often used to provide high-resolution two-dimensional angular information.
[0003] In multi-sensor collaborative detection scenarios, existing technologies such as centralized fusion or distributed federated filtering are typically used for data processing to obtain a unified target trajectory. Centralized fusion involves inputting all sensor measurements into a Kalman filter simultaneously, constructing an expanded-dimensional observation vector and an expanded-dimensional observation noise covariance matrix through measurement dimension expansion, and then performing filter updates. Another common distributed federated filtering approach involves setting up multiple local sub-filters, allowing each sensor's corresponding local filter to independently complete and generate local state estimates, and then using a main filter to aggregate and fuse global information.
[0004] However, in practical applications, radar sensors output three-dimensional ranging perception results, while photoelectric sensors only output two-dimensional non-ranging perception results when ranging is limited, resulting in inconsistent observation dimensions. When a centralized fusion scheme is adopted, the requirement to directly expand the dimensions of observation data from different dimensions within the same measurement space leads to the inability to directly construct the expanded observation matrix and a mismatch in the dimensions of the covariance matrix, hindering the data fusion process. When a federated filtering scheme is adopted, this scheme forces each local sub-filter to be able to independently complete a complete state estimation, but the pure angle observations of photoelectric sensors cannot uniquely determine the three-dimensional spatial position of the target, causing the local sub-filter states to diverge, ultimately leading to the failure of the overall fusion structure. Summary of the Invention
[0005] This invention addresses the problems in the prior art by providing a low-altitude heterogeneous sensor signal fusion method, system, terminal, and medium. It solves the problem in the background art where, when using a centralized fusion scheme, the requirement to directly expand the dimensions of observation data from different dimensions within the same measurement space leads to the inability to directly construct the expanded observation matrix and a mismatch in the dimensions of the covariance matrix, hindering the data fusion process. Simultaneously, it solves the problem when using a federated filtering scheme, where the scheme forces each local sub-filter to independently complete a full state estimate, but pure angle observations from photoelectric sensors cannot uniquely determine the three-dimensional spatial position of the target, causing local sub-filter state divergence and ultimately leading to the failure of the overall fusion structure.
[0006] The technical solution adopted in this invention is as follows: In a first aspect, this application provides a method for fusing signals from low-altitude heterogeneous sensors, comprising the following steps: Step S1: Obtain the ranging perception results output by the radar sensor and the non-ranging perception results output by the photoelectric sensor; The ranging perception result includes the target's three-dimensional position information, while the non-ranging perception result only includes the target's two-dimensional angle information, lacking the distance dimension. Step S2: Perform timestamp alignment on the ranging sensing results and the non-ranging sensing results, and uniformly convert the sensing results under different coordinate systems to the global geographic coordinate system; Step S3: Associate and match the radar points in the converted ranging and sensing results with existing tracks to obtain successfully matched pairs. Step S4: Based on the sequential extended Kalman filter framework, the first step of filtering update is performed on the existing track using the ranging sensing results in the matching pair to obtain a temporary state estimate containing distance constraints. Step S5: Transform the target position in the temporary state estimate to the azimuth coordinate system of the photoelectric sensor to calculate the Jacobian matrix of the nonlinear angle observation equation with respect to the state variables; Based on the Jacobian matrix, the second step of filtering and updating is performed on the temporary state estimate using the angular residuals of the non-range sensing results in the matching pair, thereby realizing the dimensionless filtering fusion of heterogeneous information and obtaining the final state estimate of the target.
[0007] Furthermore, in step S2, a unified transformation of the ranging sensing results under different coordinate systems is performed on the ranging sensing results, and the equivalent measurement covariance matrix is calculated, specifically including: Convert the polar coordinate observations in the ranging sensing results to Cartesian space coordinates:
[0008]
[0009]
[0010] in, Radial distance, It is the azimuth angle. The pitch angle, The installation height of the radar sensor; Based on the first-order Taylor expansion, the equivalent measurement covariance matrix in Cartesian space after transformation is calculated. :
[0011] in, The original polar coordinate measurement noise covariance matrix for radar. Let be the Jacobian matrix of the polar coordinate to Cartesian coordinate transformation equation with respect to the polar coordinate variables.
[0012] Furthermore, in step S3, the radar points in the converted ranging perception result are associated and matched with existing tracks, specifically including: Regarding the first Observation vector of each radar point With the Predicted status of existing flight paths Calculate the Mahalanobis distance between the two. :
[0013] in, For radar observation mapping function, The new information covariance matrix, For the trajectory prediction covariance matrix, For the observation matrix, The equivalent measurement covariance matrix ; Construct an association cost matrix using all calculated Mahalanobis distances, and then apply the KM bipartite graph optimal matching algorithm to find the global optimal matching pair in the association cost matrix. A gating decision is applied to the matching pair: when the Mahalanobis distance is greater than a preset threshold, the corresponding match is determined to be invalid.
[0014] Furthermore, the two-step filter update in steps S4 and S5 specifically includes: Cartesian observations corresponding to the ranging sensing results Perform the first step of filtering and updating to obtain the temporary state estimate. and temporary covariance :
[0015] in, and These represent the prior state prediction and prior error covariance before the first step of filtering and updating, respectively. For radar observation matrix, For radar Kalman gain, The observation vector is composed of the Cartesian space coordinates. The equivalent measurement covariance matrix ; Using the temporary state estimation The spatial coordinates and the relative displacement of the photoelectric sensor Construct a nonlinear angle observation equation:
[0016] Calculate the first-order partial derivatives of the nonlinear angle observation equation with respect to the state variables to obtain the Jacobian matrix. ; Calculate the angular Kalman gain Then, the second step of filtering and updating is performed to obtain the final state estimate. With final covariance :
[0017] in, This is the two-dimensional angle observation vector of the non-range sensing result. This is the noise covariance matrix for photoelectric angle measurement.
[0018] Furthermore, the method also includes a step of performing pure angle state correction for targets not associated with radar tracks but captured by photoelectric sensors, specifically including: Extract the target's trajectory state from the previous moment and predict it to the current moment to obtain the predicted state and prediction covariance; By performing a pure angle Kalman filter update using the two-dimensional angle observation vector output by the photoelectric sensor at the current moment, the updated covariance matrix is obtained. ; For the covariance matrix Apply an expansion factor to the mid-range component. The corrected covariance matrix is obtained. :
[0019] in, , The covariance matrix Chinese correspondence The diagonal element in the axial distance direction.
[0020] Furthermore, the method also includes trajectory optimization and output steps, specifically including: Extract the state vector, error covariance matrix, and fusion source identifier bitmask that marks the sensor participation state of the active track within the filtering period; Perform fixed-interval smoothing on the trajectory state sequence of a preset number of frames, and output the smoothed position sequence and velocity sequence; The track confidence score is calculated based on the trace and residual whitening test of the covariance matrix, and the output is a structured fused track information containing the position sequence, the velocity sequence and the track confidence score.
[0021] Furthermore, both the radar sensor and the photoelectric sensor include multiple units; In step S4, performing the first step of filter update specifically includes: The ranging perception results of multiple radar sensors are input into the filter in chronological order according to their timestamps, and the prediction and update of the ranging dimension are performed independently for each sensor. In step S5, the second step of filtering update specifically includes: The non-range sensing results of multiple photoelectric sensors are sequentially input into the filter according to the timestamp order. Before each update, the current state estimate is independently converted to the azimuth coordinate system of the corresponding photoelectric sensor and the corresponding Jacobian matrix is recalculated. The state correction of the angle dimension is performed independently for each sensor.
[0022] Secondly, this application provides a low-altitude heterogeneous sensor signal fusion system for performing the low-altitude heterogeneous sensor signal fusion method described in the first aspect, comprising: The data acquisition module is used to acquire the ranging perception results output by the radar sensor and the non-ranging perception results output by the photoelectric sensor; wherein, the ranging perception results contain the three-dimensional position information of the target, and the non-ranging perception results only contain the two-dimensional angle information of the target, lacking the distance dimension; The spatiotemporal unification module is used to perform timestamp alignment on the ranging sensing results and the non-ranging sensing results, and to uniformly convert the sensing results under different coordinate systems to the global geographic coordinate system. The association matching module is used to associate and match the radar spot in the converted ranging perception result with the existing track to obtain successfully matched pairs. The first filtering module is used to perform the first step of filtering update on the existing track based on the sequential extended Kalman filter framework and the ranging perception results in the matching pair to obtain a temporary state estimate containing distance constraints. The second filtering module is used to transform the target position in the temporary state estimate to the azimuth coordinate system of the photoelectric sensor, so as to calculate the Jacobian matrix of the nonlinear angle observation equation with respect to the state variables; based on the Jacobian matrix, the second filtering update is performed on the temporary state estimate using the angle residual of the non-range sensing results in the matching pair, so as to realize the dimensionless filtering fusion of heterogeneous information and obtain the final state estimate of the target.
[0023] Thirdly, this application provides a terminal, including: Memory, used to store the signal fusion program for low-altitude heterogeneous sensors; A processor is configured to implement the steps of the low-altitude heterogeneous sensor signal fusion method as described in the first aspect when executing the low-altitude heterogeneous sensor signal fusion program.
[0024] Fourthly, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the low-altitude heterogeneous sensor signal fusion method as described in the first aspect.
[0025] As can be seen from the above technical solutions, the advantages of the present invention are: By acquiring the perception results of heterogeneous sensors with different dimensions, and performing the first step of filtering update for ranging perception results and the second step of filtering update for non-ranging perception results in stages based on the sequential extended Kalman filter framework, the collaborative processing of three-dimensional distance constraints and two-dimensional angle information is realized. This solves the technical obstacle that makes it difficult to directly construct extended-dimensional observation matrices for centralized fusion due to the inconsistent observation dimensions of heterogeneous sensors.
[0026] By converting radar polar coordinate observations to Cartesian space coordinates and calculating the equivalent measurement covariance matrix using the Jacobian matrix based on first-order Taylor expansion, the objective propagation of measurement noise during the transformation between different coordinate systems is realized, providing a reliable error covariance benchmark that conforms to statistical characteristics for subsequent filtering updates.
[0027] By calculating the Mahalanobis distance that integrates the track prediction covariance and measurement noise, and applying the KM bipartite graph optimal matching algorithm to solve for the globally optimal matching pair in the association cost matrix, the globally optimal allocation of radar points and existing tracks in multi-target scenarios is achieved, reducing the erroneous association rate when targets are dense or intersecting.
[0028] By setting a specific Kalman gain calculation formula for the first step of filtering update, and a second step update formula for the Jacobian matrix of the nonlinear angle observation equation derived based on spatial relative displacement, a logical closed loop for the sequential fusion of heterogeneous dimensionless data at the underlying mathematical calculation level is realized, ensuring the feasibility of system state estimation and the stability of numerical calculation.
[0029] By performing pure angular Kalman filtering updates when only two-dimensional angular observation vectors exist, and applying an expansion factor to the diagonal components of the corresponding range direction in the updated covariance matrix, continuous track tracking is achieved when radar ranging information for the target is missing. This also reasonably reflects the natural accumulation of range dimension uncertainty under single-station pure angular tracking and prevents the numerical divergence of the filter.
[0030] By performing fixed-interval smoothing on the active track state sequence of a preset number of frames and calculating the track confidence score based on the trace and residual whitening test results of the covariance matrix, the smoothness of the fused track output is further improved, enabling the output of structured track data with clear quality quantification results to the upper-level system.
[0031] By inputting the sensing results of multiple radars and multiple photoelectric sensors into a filter in chronological order according to their timestamps, and independently completing coordinate transformation and corresponding dimensional state correction for each sensor, distributed pipeline fusion in a multi-sensor network environment is realized, which can support the dynamic and flexible addition and removal of sensor nodes in the monitoring network. Attached Figure Description
[0032] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0033] Figure 1 This is a flowchart of a low-altitude heterogeneous sensor signal fusion method according to an embodiment of the present invention; Figure 2 This is a structural framework diagram of a low-altitude heterogeneous sensor signal fusion system according to an embodiment of the present invention. Detailed Implementation
[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0035] Please see Figure 1 As shown, this application provides a method for fusing signals from heterogeneous low-altitude sensors, comprising the following steps: Step S1: Obtain the ranging perception results output by the radar sensor and the non-ranging perception results output by the photoelectric sensor; The ranging perception result includes the target's three-dimensional position information, while the non-ranging perception result only includes the target's two-dimensional angle information, lacking the distance dimension. Step S2: Perform timestamp alignment on the ranging sensing results and the non-ranging sensing results, and uniformly convert the sensing results under different coordinate systems to the global geographic coordinate system; Step S3: Associate and match the radar points in the converted ranging and sensing results with existing tracks to obtain successfully matched pairs. Step S4: Based on the sequential extended Kalman filter framework, the first step of filtering update is performed on the existing track using the ranging sensing results in the matching pair to obtain a temporary state estimate containing distance constraints. Step S5: Transform the target position in the temporary state estimate to the azimuth coordinate system of the photoelectric sensor to calculate the Jacobian matrix of the nonlinear angle observation equation with respect to the state variables; Based on the Jacobian matrix, the second step of filtering and updating is performed on the temporary state estimate using the angular residuals of the non-range sensing results in the matching pair, thereby realizing the dimensionless filtering fusion of heterogeneous information and obtaining the final state estimate of the target.
[0036] In practical engineering deployments, the execution entity of the aforementioned low-altitude heterogeneous sensor signal fusion method can be a data fusion processing server deployed on an edge computing node or in a cloud computing center. This processing server is internally configured with a dedicated sensor data bus interface for receiving data streams from the underlying hardware.
[0037] Before executing step S1, the system prerequisites usually require the completion of time synchronization calibration of each sensor and initial configuration of the spatial mounting matrix.
[0038] For example, the data output period of the radar sensor can be set to the order of ten milliseconds to fifty milliseconds, and the data output period of the photoelectric sensor can be set to the order of ten milliseconds. The ranging perception result specifically includes the target's geodetic longitude, geodetic latitude, altitude and the corresponding measurement covariance, while the non-ranging perception result only includes the azimuth angle, pitch angle and the corresponding angle measurement variance output by the photoelectric encoder.
[0039] By acquiring the aforementioned data and unifying the transformation with the global geographic coordinate system (such as the WGS84 coordinate system), a unified spatiotemporal reference is established, eliminating the system deviation caused by differences in the physical deployment of heterogeneous sensors, thereby improving the alignment accuracy and spatial consistency of multi-source sensing data before entering the filtering framework.
[0040] In some embodiments, step S2 involves performing a unified transformation of the ranging sensing results in different coordinate systems and calculating the equivalent measurement covariance matrix, specifically including: Convert the polar coordinate observations in the ranging sensing results to Cartesian space coordinates:
[0041]
[0042]
[0043] in, Radial distance, It is the azimuth angle. The pitch angle, The installation height of the radar sensor; Based on the first-order Taylor expansion, the equivalent measurement covariance matrix in Cartesian space after transformation is calculated. :
[0044] in, The original polar coordinate measurement noise covariance matrix for radar. Let be the Jacobian matrix of the polar coordinate to Cartesian coordinate transformation equation with respect to the polar coordinate variables.
[0045] In the specific execution logic, the coordinate transformation and covariance propagation processes are performed by the preprocessing algorithm engine within the fusion processing server. The radar perception results must undergo a preset sliding window filter before being input into the algorithm engine to remove isolated clutter interference points.
[0046] In the derivation of the equivalent measurement covariance matrix, the first-order Taylor expansion mechanism is used to linearly map the nonlinear measurement noise in polar coordinates to Cartesian space.
[0047] This nonlinear covariance propagation mechanism based on the Jacobian matrix ensures the statistical rigor of the observation noise during the mapping process in different coordinate spaces, avoids the truncation error caused by directly assuming that the Cartesian space noise is isotropic Gaussian white noise, and enhances the objective accuracy of subsequent filter gain calculation.
[0048] In some embodiments, step S3 involves associating and matching the radar spots in the converted ranging sensing results with existing tracks, specifically including: Regarding the first Observation vector of each radar point With the Predicted status of existing flight paths Calculate the Mahalanobis distance between the two. :
[0049] in, For radar observation mapping function, The new information covariance matrix, For the trajectory prediction covariance matrix, For the observation matrix, The equivalent measurement covariance matrix ; Construct an association cost matrix using all calculated Mahalanobis distances, and then apply the KM bipartite graph optimal matching algorithm to find the global optimal matching pair in the association cost matrix. A gating decision is applied to the matching pair: when the Mahalanobis distance is greater than a preset threshold, the corresponding match is determined to be invalid.
[0050] The aforementioned point-track and track association matching logic is typically triggered by a data association coprocessor configured within the system. Before performing the specific matching calculation, the system needs to extrapolate and predict historical states to the current radar observation time based on the dynamic state transition matrix of the existing track, thereby generating a track prediction covariance matrix.
[0051] After the cost matrix is constructed, the KM matching algorithm engine achieves global assignment by solving for the set of matching edges with the minimum total weight. Through the above-mentioned rigorous chi-square test gating and global optimal matching algorithm, the system significantly reduces the statistical probability of misassociation between point tracks and tracks when dealing with dense clutter environments or multi-target cross-movement scenarios, maintaining the purity of track data and the robustness of multi-target tracking.
[0052] In some embodiments, the two-step filter update in steps S4 and S5 specifically includes: Cartesian observations corresponding to the ranging sensing results Perform the first step of filtering and updating to obtain the temporary state estimate. and temporary covariance :
[0053] in, and These represent the prior state prediction and prior error covariance before the first step of filtering and updating, respectively. For radar observation matrix, For radar Kalman gain, The observation vector is composed of the Cartesian space coordinates. The equivalent measurement covariance matrix ; Using the temporary state estimation The spatial coordinates and the relative displacement of the photoelectric sensor Construct a nonlinear angle observation equation:
[0054] Calculate the first-order partial derivatives of the nonlinear angle observation equation with respect to the state variables to obtain the Jacobian matrix. ; Calculate the angular Kalman gain Then, the second step of filtering and updating is performed to obtain the final state estimate. With final covariance :
[0055] in, This is the two-dimensional angle observation vector of the non-range sensing result. This is the noise covariance matrix for photoelectric angle measurement.
[0056] The underlying matrix operation core of the sequential extended Kalman filter framework serves as the acceleration core for digital signal processing within the system.
[0057] During the first step of filtering and updating, the target's state variables are typically instantiated as a six-dimensional state vector containing three-dimensional position parameters and three-dimensional velocity parameters. The output of the temporary state estimation essentially establishes the target's distance dimension baseline constraint in space.
[0058] For example, when constructing the nonlinear angle observation equation, the calculation of the relative displacement of the photoelectric sensor needs to read the real-time attitude bias parameters fed back by the servo gimbal in advance for system error compensation; in the matrix inversion operation in the process of calculating the Kalman gain, the Cholesky decomposition method can be used to ensure the numerical stability of floating-point operations.
[0059] By leveraging the aforementioned distributed two-step sequential update mechanism, the system can effectively integrate the angle residual information with missing distance dimension into the filtering loop without constructing a high-dimensional nonlinear joint observation matrix. This solves the direct fusion obstacle caused by the difference in the dimensionality of heterogeneous sensor data, and improves the convergence speed of the final state estimation and the accuracy of spatial orientation positioning.
[0060] In some embodiments, the method further includes a step of performing a pure angle state correction for targets not associated with radar tracks but captured by photoelectric sensors, specifically including: Extract the target's trajectory state from the previous moment and predict it to the current moment to obtain the predicted state and prediction covariance; By performing a pure angle Kalman filter update using the two-dimensional angle observation vector output by the photoelectric sensor at the current moment, the updated covariance matrix is obtained. ; For the covariance matrix Apply an expansion factor to the mid-range component. The corrected covariance matrix is obtained. :
[0061] in, , The covariance matrix Chinese correspondence The diagonal element in the axial distance direction.
[0062] For the specific process of pure angle state correction, its execution logic is taken over by the degraded tracking and maintenance engine within the system. The pre-triggered conditions for this state correction mechanism are usually that the target maneuvers into the radar detection blind zone, or that the radar receiving channel is subjected to active deception interference, resulting in the continuous loss of effective ranging points.
[0063] When the fusion processing server identifies that the target has only received photoelectric two-dimensional angle observation vectors for several consecutive processing cycles, the maintenance engine applies an inflation factor of the corresponding magnitude to the distance component of the covariance matrix according to the above formula, so as to increase the state uncertainty of this dimension at the mathematical level.
[0064] This adaptive processing mechanism, which introduces a covariance expansion factor, objectively conforms to the physical law that the distance error of a single-angle sensor diverges over time when the distance observability is lacking. It avoids the filter from generating false convergence states in unobservable directions, and ensures the positive definiteness of the covariance matrix and the system's anti-lockout capability in pure angle tracking mode.
[0065] In some embodiments, the method further includes a trajectory optimization and output step, specifically including: Extract the state vector, error covariance matrix, and fusion source identifier bitmask that marks the sensor participation state of the active track within the filtering period; Perform fixed-interval smoothing on the trajectory state sequence of a preset number of frames, and output the smoothed position sequence and velocity sequence; The track confidence score is calculated based on the trace and residual whitening test of the covariance matrix, and the output is a structured fused track information containing the position sequence, the velocity sequence and the track confidence score.
[0066] The track optimization and output steps are executed by the track lifecycle management and external interface service modules in the system backend. Before outputting data, the interface service module needs to implement queue management for the current fused tracks in the caching layer.
[0067] For example, the above fixed interval smoothing operation can be instantiated as an RTS smoothing algorithm. The preset number of frames can be set to 15 consecutive historical data frames including the current frame, so as to use backward recursion to smooth the historical trajectory. The calculation of the track confidence score is to map it to a standardized score range of 0-100 by comparing the normalized ratio of the root mean square of the residual of the innovation sequence to the trace of the theoretical covariance matrix.
[0068] Through post-processing level interval smoothing and whitening test evaluation, the algorithm engine effectively suppresses the high-frequency random jitter of the forward filter output state, making the smooth position sequence of the output more consistent with the physical kinematic laws of the real target, and providing the downstream command and control system with a standardized high-precision data source with clear quality identification.
[0069] In some embodiments, both the radar sensor and the photoelectric sensor include multiple units; In step S4, performing the first step of filter update specifically includes: The ranging perception results of multiple radar sensors are input into the filter in chronological order according to their timestamps, and the prediction and update of the ranging dimension are performed independently for each sensor. In step S5, the second step of filtering update specifically includes: The non-range sensing results of multiple photoelectric sensors are sequentially input into the filter according to the timestamp order. Before each update, the current state estimate is independently converted to the azimuth coordinate system of the corresponding photoelectric sensor and the corresponding Jacobian matrix is recalculated. The state correction of the angle dimension is performed independently for each sensor.
[0070] The aforementioned distributed networking and fusion process involving multiple heterogeneous sensors relies on a distributed sensor network fusion server with high concurrency data throughput capabilities. When each heterogeneous sensor node accesses the processing network, it needs to achieve microsecond-level alignment of the entire network's clock frequency through a high-precision network time protocol (such as the PTP protocol).
[0071] For example, in a wide-area surveillance zone containing two solid-state surveillance radars and four electro-optical search and trackers, the network fusion server constructs a monotonically increasing ordered processing queue based on the absolute timestamp tag carried by each sensing data packet.
[0072] When performing angle dimension state correction, the system dynamically loads the local deployment latitude, longitude and elevation parameters of the corresponding labeled photoelectric sensor in each sequential loop to update the Jacobian matrix under the current observation link.
[0073] This pipeline-style sequential processing architecture eliminates the need for re-deriving and reconstructing the overall state transition matrix of the core filter when the underlying system faces dynamic deployment of sensor nodes or network outages due to faults. This enhances the elasticity and scalability of the multi-source sensor network architecture and improves the fault tolerance and data utilization rate of the system under complex topology networks.
[0074] Please see Figure 2 As shown, this application provides a low-altitude heterogeneous sensor signal fusion system, comprising: The data acquisition module is used to acquire the ranging perception results output by the radar sensor and the non-ranging perception results output by the photoelectric sensor; wherein, the ranging perception results contain the three-dimensional position information of the target, and the non-ranging perception results only contain the two-dimensional angle information of the target, lacking the distance dimension; The spatiotemporal unification module is used to perform timestamp alignment on the ranging sensing results and the non-ranging sensing results, and to uniformly convert the sensing results under different coordinate systems to the global geographic coordinate system. The association matching module is used to associate and match the radar spot in the converted ranging perception result with the existing track to obtain successfully matched pairs. The first filtering module is used to perform the first step of filtering update on the existing track based on the sequential extended Kalman filter framework and the ranging perception results in the matching pair to obtain a temporary state estimate containing distance constraints. The second filtering module is used to transform the target position in the temporary state estimate to the azimuth coordinate system of the photoelectric sensor, so as to calculate the Jacobian matrix of the nonlinear angle observation equation with respect to the state variables; based on the Jacobian matrix, the second filtering update is performed on the temporary state estimate using the angle residual of the non-range sensing results in the matching pair, so as to realize the dimensionless filtering fusion of heterogeneous information and obtain the final state estimate of the target.
[0075] In some embodiments, this application provides a terminal, including: Memory, used to store the signal fusion program for low-altitude heterogeneous sensors; A processor is configured to execute the steps of the low-altitude heterogeneous sensor signal fusion method when implementing the low-altitude heterogeneous sensor signal fusion system.
[0076] In some embodiments, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the low-altitude heterogeneous sensor signal fusion method.
[0077] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of one or more embodiments of this specification.
Claims
1. A method for fusing signals from heterogeneous low-altitude sensors, characterized in that, Includes the following steps: Step S1: Obtain the ranging perception results output by the radar sensor and the non-ranging perception results output by the photoelectric sensor; The ranging perception result includes the target's three-dimensional position information, while the non-ranging perception result only includes the target's two-dimensional angle information, lacking the distance dimension. Step S2: Perform timestamp alignment on the ranging sensing results and the non-ranging sensing results, and uniformly convert the sensing results under different coordinate systems to the global geographic coordinate system; Step S3: Associate and match the radar points in the converted ranging and sensing results with existing tracks to obtain successfully matched pairs. Step S4: Based on the sequential extended Kalman filter framework, the first step of filtering update is performed on the existing track using the ranging sensing results in the matching pair to obtain a temporary state estimate containing distance constraints. Step S5: Transform the target position in the temporary state estimate to the azimuth coordinate system of the photoelectric sensor to calculate the Jacobian matrix of the nonlinear angle observation equation with respect to the state variables; Based on the Jacobian matrix, the second step of filtering and updating is performed on the temporary state estimate using the angular residuals of the non-range sensing results in the matching pair, thereby realizing the dimensionless filtering fusion of heterogeneous information and obtaining the final state estimate of the target.
2. The low-altitude heterogeneous sensor signal fusion method according to claim 1, characterized in that, In step S2, a unified transformation of the ranging sensing results under different coordinate systems is performed on the ranging sensing results, and the equivalent measurement covariance matrix is calculated, specifically including: Convert the polar coordinate observations in the ranging sensing results to Cartesian space coordinates: in, Radial distance, It is the azimuth angle. The pitch angle, The installation height of the radar sensor; Based on the first-order Taylor expansion, the equivalent measurement covariance matrix in Cartesian space after transformation is calculated. : in, The original polar coordinate measurement noise covariance matrix for radar. Let be the Jacobian matrix of the polar coordinate to Cartesian coordinate transformation equation with respect to the polar coordinate variables.
3. The low-altitude heterogeneous sensor signal fusion method according to claim 2, characterized in that, In step S3, the radar points in the converted ranging sensing results are associated and matched with existing tracks, specifically including: Regarding the first Observation vectors of radar points With the Predicted status of existing flight paths Calculate the Mahalanobis distance between the two. : in, For radar observation mapping function, The new information covariance matrix, For the trajectory prediction covariance matrix, For the observation matrix, The equivalent measurement covariance matrix ; Construct an association cost matrix using all calculated Mahalanobis distances, and then apply the KM bipartite graph optimal matching algorithm to find the global optimal matching pair in the association cost matrix. A gating decision is applied to the matching pair: when the Mahalanobis distance is greater than a preset threshold, the corresponding match is determined to be invalid.
4. The low-altitude heterogeneous sensor signal fusion method according to claim 3, characterized in that, The two-step filter update in steps S4 and S5 specifically includes: Cartesian observations corresponding to the ranging sensing results Perform the first step of filtering and updating to obtain the temporary state estimate. and temporary covariance : in, and These represent the prior state prediction and prior error covariance before the first step of filtering and updating, respectively. For radar observation matrix, For radar Kalman gain, The observation vector is composed of the Cartesian space coordinates. The equivalent measurement covariance matrix ; Using the temporary state estimation The spatial coordinates and the relative displacement of the photoelectric sensor Construct a nonlinear angle observation equation: Calculate the first-order partial derivatives of the nonlinear angle observation equation with respect to the state variables to obtain the Jacobian matrix. ; Calculate the angular Kalman gain Then, the second step of filtering and updating is performed to obtain the final state estimate. With final covariance : in, This is the two-dimensional angle observation vector of the non-range sensing result. This is the noise covariance matrix for photoelectric angle measurement.
5. The low-altitude heterogeneous sensor signal fusion method according to claim 1, characterized in that, The method also includes a step of performing pure angle state correction for targets not associated with radar tracks but captured by photoelectric sensors, specifically including: Extract the target's trajectory state from the previous moment and predict it to the current moment to obtain the predicted state and prediction covariance; By performing a pure angle Kalman filter update using the two-dimensional angle observation vector output by the photoelectric sensor at the current moment, the updated covariance matrix is obtained. ; For the covariance matrix Apply an expansion factor to the mid-range component. The corrected covariance matrix is obtained. : in, , The covariance matrix Chinese correspondence The diagonal element in the axial distance direction.
6. The low-altitude heterogeneous sensor signal fusion method according to claim 1, characterized in that, The method also includes trajectory optimization and output steps, specifically including: Extract the state vector, error covariance matrix, and fusion source identifier bitmask that marks the sensor participation state of the active track within the filtering period; Perform fixed-interval smoothing on the trajectory state sequence of a preset number of frames, and output the smoothed position sequence and velocity sequence; The track confidence score is calculated based on the trace and residual whitening test of the covariance matrix, and the output is a structured fused track information containing the position sequence, the velocity sequence and the track confidence score.
7. The low-altitude heterogeneous sensor signal fusion method according to claim 1, characterized in that, Both the radar sensor and the photoelectric sensor include multiple units; In step S4, performing the first step of filter update specifically includes: The ranging perception results of multiple radar sensors are input into the filter in chronological order according to their timestamps, and the prediction and update of the ranging dimension are performed independently for each sensor. In step S5, the second step of filtering update specifically includes: The non-range sensing results of multiple photoelectric sensors are sequentially input into the filter according to the timestamp order. Before each update, the current state estimate is independently converted to the azimuth coordinate system of the corresponding photoelectric sensor and the corresponding Jacobian matrix is recalculated. The state correction of the angle dimension is performed independently for each sensor.
8. A low-altitude heterogeneous sensor signal fusion system, used to execute the low-altitude heterogeneous sensor signal fusion method as described in any one of claims 1 to 7, characterized in that, include: The data acquisition module is used to acquire the ranging perception results output by the radar sensor and the non-ranging perception results output by the photoelectric sensor; wherein, the ranging perception results contain the three-dimensional position information of the target, and the non-ranging perception results only contain the two-dimensional angle information of the target, lacking the distance dimension; The spatiotemporal unification module is used to perform timestamp alignment on the ranging sensing results and the non-ranging sensing results, and to uniformly convert the sensing results under different coordinate systems to the global geographic coordinate system. The association matching module is used to associate and match the radar spot in the converted ranging perception result with the existing track to obtain successfully matched pairs. The first filtering module is used to perform the first step of filtering update on the existing track based on the sequential extended Kalman filter framework and the ranging perception results in the matching pair to obtain a temporary state estimate containing distance constraints. The second filtering module is used to transform the target position in the temporary state estimate to the azimuth coordinate system of the photoelectric sensor, so as to calculate the Jacobian matrix of the nonlinear angle observation equation with respect to the state variables; based on the Jacobian matrix, the second filtering update is performed on the temporary state estimate using the angle residual of the non-range sensing results in the matching pair, so as to realize the dimensionless filtering fusion of heterogeneous information and obtain the final state estimate of the target.
9. A terminal, characterized in that, include: Memory, used to store the signal fusion program for low-altitude heterogeneous sensors; A processor is configured to implement the steps of the low-altitude heterogeneous sensor signal fusion method as described in any one of claims 1-7 when executing the low-altitude heterogeneous sensor signal fusion program.
10. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions. When the computer reads the computer instructions in the storage medium, the computer executes the low-altitude heterogeneous sensor signal fusion method as described in any one of claims 1-7.
