A sonar and photoelectric asynchronous spot fusion localization method based on structural anchor point constraints

By using a sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints, the problems of misassociation and offset of target point tracking data in maritime collaborative sensing scenarios are solved, and the accurate fusion and stable output of target point tracking are achieved.

CN123112918APending Publication Date: 2026-11-17HARBIN INST OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611255774.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-19
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In scenarios involving maritime, port, waterway, shore-based monitoring, and collaborative perception of unmanned platforms, the target point data output by sonar and optoelectronic modules are prone to mis-associations, omissions, target center shifts, and abrupt changes in fusion results when directly matched due to differences in sampling frequency, reporting delay, and coordinate reference. This is especially true when the target has a certain spatial arrangement structure, making it difficult to stably recover the overall spatial distribution of the target when there is a local loss of point data in a single frame or coordinate drift.

Method used

A sonar and photoelectric asynchronous track fusion localization method based on structural anchor point constraints is adopted. Valid tracks are screened through a time window, PCA spatial attitude correction and hierarchical structure extraction are performed to generate candidate anchor points, which are then used as constraints for multi-source track fusion. Weights are calculated by combining source reliability and confidence, abnormal tracks are eliminated, and the fusion center is dynamically updated.

Benefits of technology

It reduces the impact of missed detections and noise points in a single frame on the fusion results, improves the accuracy of target points and the stability of the fused trajectory, reduces false associations, and ensures continuous output of target positions under asynchronous data conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123112918A_ABST
    Figure CN123112918A_ABST
Patent Text Reader

Abstract

A sonar and photoelectric asynchronous track fusion localization method based on structural anchor point constraints belongs to the field of target detection technology. To solve the problem of accurate target track detection, this invention includes: acquiring multi-source track data packets output by a sonar module and a photoelectric module; performing latitude and longitude decoding and coordinate transformation; using a time window approach to filter effective image frames to obtain an effective track set; performing PCA spatial attitude correction, calculating the track mean and covariance matrix, extracting the eigenvector corresponding to the largest eigenvalue as the principal axis, and using its orthogonal direction as the secondary axis to establish a structural adaptive coordinate system; extracting the hierarchical spatial structure of the effective track set in the structural adaptive coordinate system to obtain multi-source track fusion with candidate anchor points as constraints, including candidate anchor point neighborhood filtering, cross-source association, calculating track weights, and extracting the fusion center; inversely transforming the fusion center from the PCA coordinate system back to the local EN coordinate system and restoring it to latitude and longitude coordinates.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints. Background Technology

[0002] In scenarios such as maritime, port, waterway, shore-based monitoring, and collaborative perception by unmanned platforms, the system receives target observation results from sonar and optoelectronic modules. The sonar module forms target traces based on acoustic detection results, while the optoelectronic module forms target traces based on visible light or infrared target detection and tracking results. Both types of modules report discrete traces in time-sharing data packets. Each trace data packet typically includes fields such as source identifier, timestamp, target category or tag, latitude and longitude or local coordinates, confidence level, and trace quality. Because trace data from different sources differ in sampling frequency, reporting delay, coordinate reference, equipment installation attitude, observation noise, and missed detection probability, directly matching traces from different sources based on a single moment or nearest neighbor distance can easily lead to problems such as false associations, missed associations, target center shifts, and jumps in fusion results. Especially when the target has a certain spatial arrangement structure, or when single-frame traces have local missing parts or coordinate drift, simply relying on single-frame point-to-point matching is insufficient to stably recover the overall spatial distribution of the target. Summary of the Invention

[0003] The problem this invention aims to solve is the accurate detection of target points. It proposes a sonar and photoelectric asynchronous point fusion positioning method based on structural anchor point constraints.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A sonar and photoelectric asynchronous point tracking fusion localization method based on structural anchor point constraints includes the following steps:

[0006] S1. In a collaborative sensing scenario, initial data is collected based on the maximum acquisition duration or the maximum number of image frames acquired, and multi-source point trace data packets output by the sonar module and the photoelectric module are acquired.

[0007] S2. For the target point data packet in the multi-source point data packet obtained in step S1, perform latitude and longitude decoding and coordinate transformation, and convert the point coordinates to local EN coordinates with the preset reference point as the origin to obtain the target point data packet after coordinate transformation.

[0008] S3. For the target point data packet obtained after coordinate transformation in step S2, the effective image frames are filtered using a time window method, including setting point number thresholds and spatial span thresholds for point frames entering the time window, to obtain a set of effective point traces.

[0009] S4. Perform PCA spatial attitude correction on the set of valid points obtained in step S3, calculate the mean and covariance matrix of the points, extract the eigenvector corresponding to the largest eigenvalue as the principal axis, and use its orthogonal direction as the secondary axis to establish a structural adaptive coordinate system.

[0010] S5. Extract the hierarchical spatial structure of the effective point set in the structural adaptive coordinate system. First, establish a one-dimensional histogram along the secondary axis and smooth it to extract column peaks. Then, establish a one-dimensional histogram along the principal axis in the point set corresponding to each column peak to extract row peaks. Generate candidate anchor points based on the intersection of column peaks and row peaks.

[0011] S6. Perform multi-source point fusion with candidate anchor points as constraints, including candidate anchor point neighborhood selection, cross-source association, calculation of point weights, and extraction of fusion centers;

[0012] S7. Inversely transform the fusion center from the PCA coordinate system back to the local EN coordinate system and restore it to latitude and longitude coordinates;

[0013] S8. Based on the method of steps S1-S7, dynamically update the candidate anchor point and fusion center by combining the current anchor point with the newly arrived data packet, and output the unified target location, fusion point track or fusion track.

[0014] Furthermore, the target point data packet in the multi-source point data packet output by the sonar module and the photoelectric module in step S1 includes the source of the target point, the target point tag, the target point data packet generation time, the longitude and latitude fixed-point coding values ​​of the target point, the source's confidence in the target point, the quality evaluation value of the target point, and extended fields. The extended fields are used to store the target speed, heading, equipment number, internal target number of the source, and other additional information.

[0015] Furthermore, the specific implementation method of step S2 includes the following steps:

[0016] S2.1. The longitude fixed-point encoding value in the data packet of the i-th target point obtained in step S1 and latitude fixed point code value The fields are converted to 32-bit signed integers using the same number of binary bits, and then converted to actual longitude and latitude in degrees according to the I31 fixed-point angle scaling factor. The expression is:

[0017]

[0018]

[0019] in, Let be the 32-bit signed integer corresponding to the fixed-point longitude code value in the data packet of the i-th target point. is a 32-bit signed integer corresponding to the latitude fixed-point encoding value in the i-th target point trace data packet; This means converting an unsigned integer to a 32-bit signed integer based on its original binary bits; Let i be the actual longitude of the data packet for the i-th target point. The actual latitude of the i-th target point trace data packet;

[0020] S2.2. Based on the preset reference origin The coordinates of the i-th target point data packet are converted to local EN coordinates, expressed as:

[0021]

[0022] in, Let be the local coordinates of the i-th target point data packet. Represents the coordinate transformation function. and Let be the coordinates of the i-th target point data packet relative to the reference origin in the east and north directions, respectively. The actual longitude of the preset reference origin, The actual latitude of the preset reference origin, This is the matrix transpose.

[0023] Furthermore, the expressions for filtering by point count threshold and spatial span threshold in step S3 are as follows:

[0024]

[0025] in, For the first The number of targets detected in the frame image The minimum number of targets per frame threshold; and They represent the first The spatial span of all target points in the frame image in the east and north directions; and These are the maximum allowable spans in the east and north directions, respectively;

[0026] Image frames that meet the above conditions are retained, and the local coordinate system coordinates of all target point data packets contained therein are summarized to form a valid point set. It is used for subsequent spatial structure restoration.

[0027] Furthermore, the specific implementation method of step S4 includes the following steps:

[0028] S4.1. For the set of valid points Calculate the mean of the dots :

[0029]

[0030] Where n is the total number of valid points in the set of valid points;

[0031] Then calculate the covariance matrix. :

[0032] ;

[0033] S4.2. For the covariance matrix Perform eigenvalue decomposition and extract the eigenvector corresponding to the largest eigenvalue. As the principal axis direction, take the eigenvector orthogonal to it. As the secondary axis direction, project the EN coordinates onto the PCA coordinate system:

[0034]

[0035] in, Indicates the direction coordinates of the secondary axis. Indicates the coordinates of the principal axis direction; for Coordinates after PCA projection.

[0036] Furthermore, the specific implementation method of step S5 includes the following steps:

[0037] S5.1. In the PCA coordinate system, the coordinates of each valid point trace along the secondary axis direction are... A one-dimensional frequency histogram is constructed using statistical data. First, the minimum and maximum values ​​among all secondary axis coordinates are taken as the statistical range of the histogram. Then, the histogram is constructed according to the preset bin width. The statistical range of the histogram is divided into several continuous intervals, and the number of points falling into each interval is counted, thus obtaining a one-dimensional histogram along the secondary axis. ;

[0038] S5.2. Suppress local noise using moving average, Gaussian smoothing, or other smoothing methods. Search for the peak positions of the smoothed one-dimensional histogram along the secondary axis to obtain the peak set. :

[0039]

[0040] in, Let m be the j-th column peak, and m be the total number of column peaks.

[0041] S5.3. For each peak Select the one that satisfies The dot pattern as the first Build a set of points and then construct a one-dimensional histogram along the principal axis. Extracting the peak set :

[0042]

[0043] in, For the first The first peak was extracted from the set of corresponding points. Individual peak positions;

[0044] S5.4. Generate candidate anchor points from the intersection of column peaks and row peaks:

[0045]

[0046] in, For the first The first peak and its first Candidate anchor points formed by the intersection of individual peaks;

[0047] Then, all candidate anchor points are numbered sequentially according to the index order of column peaks and row peaks, and uniformly denoted as candidate anchor points. ,in , This represents the total number of candidate anchor points.

[0048] Furthermore, the specific implementation method of step S6 includes the following steps:

[0049] S6.1. For the candidate anchor points obtained in step S5, search for points within their neighborhood in the PCA coordinate system to construct a candidate fusion set, expressed as:

[0050]

[0051] in, For the candidate fusion set, For the current moment of integration, The radius of the anchor point's neighborhood. As a time threshold, For the first The timestamp of the generation of each target point trace data packet. Let i be the target point trace and candidate anchor point in the PCA coordinate system. The Euclidean distance;

[0052] S6.2. Cross-source association: The spatial region defined by the candidate anchor point is used as the common association unit for sonar traces and photoelectric traces. For any candidate anchor point, the sonar traces and photoelectric traces falling within the neighborhood of that anchor point are read respectively, and the association is performed based on the data packet. Distinguish the source of the traces; when traces from different sources fall into the neighborhood of the same candidate anchor point, they are regarded as observations of the same potential target under different detection modules, and are jointly added to the candidate fusion set corresponding to the candidate anchor point;

[0053] S6.3. Calculate the point weights for the points in the candidate fusion set from the cross-source association in step S6.2. The calculation formula is:

[0054]

[0055] in, For the confidence level of the dot, For dot quality, The time difference between the current point and the current fusion time. The distance between the point and the candidate anchor point. and These are the weighting coefficients for confidence and quality, respectively. The time decay coefficient, This refers to the anchor point distance scale parameter;

[0056] S6.4. On the candidate fusion set The points in the data are weighted to calculate the fusion center:

[0057]

[0058] in, Anchor point in PCA coordinate system The integration center For the i-th target point trace data packet;

[0059] Then, the secondary axis coordinates and principal axis coordinates of the fusion center are calculated, as shown in the following expression:

[0060]

[0061] in, The coordinates of the fusion center along the secondary axis of the PCA coordinate system. The coordinates of the fusion center along the principal axis of the PCA coordinate system.

[0062] Furthermore, in step S6, a consistency check is performed on the traces in the candidate fusion set. If a trace meets any of the following conditions, it is marked as an abnormal trace and removed from the list: the distance between the trace and the candidate anchor point exceeds the threshold, the trace confidence is lower than the threshold, or the trace timestamp is significantly delayed or advanced. After the abnormal trace is removed, the fusion center is recalculated.

[0063] Furthermore, the expression for inversely transforming the fusion center from the PCA coordinate system back to the local EN coordinate system in step S7 is as follows:

[0064]

[0065] in, This represents the local EN fusion coordinates of the candidate anchor point output. and These represent the coordinates of the updated anchor point along the PCA secondary axis and primary axis, respectively.

[0066] Furthermore, the expression for smoothly updating the fusion center and the anchor point position at the current time step in step S8 is as follows:

[0067]

[0068] in, Let this be the updated anchor point position at time t. This represents the anchor point position at the previous time t-1. To update the coefficients.

[0069] The beneficial effects of this invention are:

[0070] This invention discloses a sonar and photoelectric asynchronous track fusion localization method based on structural anchor point constraints. It accumulates effective tracks within a time window, reducing the impact of single-frame missed detections, noise points, and local biases on the fusion results. Through PCA principal / secondary axis correction, the tilted track structure is transformed into a more easily extracted row-column structure, improving the stability of column and row peak extraction. Hierarchical structure extraction, prioritizing column peaks over row peaks, avoids interference between longitudinal displacement deviations of different columns, improving the accuracy of candidate anchor point recovery. Multi-source track fusion under candidate anchor point constraints reduces false associations caused by global nearest neighbor matching. Joint weighting of source reliability, confidence, quality, time decay, and distance consistency reduces the impact of low-quality sources or anomalous tracks on the fusion center. Anomaly removal and multi-source missing point compensation ensure continuous output of the fusion localization results even when some sources are interrupted or arrive asynchronously. An update mechanism suppresses jumps in the fusion target position between consecutive time points, improving the stability of the fusion track. Attached Figure Description

[0071] Figure 1 This is a flowchart of a sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints, as described in this invention.

[0072] Figure 2 This is a statistical result of the effective frame count and span in the East direction in this invention;

[0073] Figure 3This represents the distribution of all points and true points in the EN coordinate system of this invention.

[0074] Figure 4 The point density, column peaks, and candidate point distribution in the PCA projection coordinate system of this invention;

[0075] Figure 5 This is a histogram of peaks along the secondary axis of the present invention;

[0076] Figure 6 This is a histogram of row peaks along the main axis of each target column in this invention. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0078] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0079] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 -Appendix Figure 6 Detailed explanation is as follows:

[0080] Example 1:

[0081] A sonar and photoelectric asynchronous point tracking fusion localization method based on structural anchor point constraints includes the following steps:

[0082] S1. In a collaborative sensing scenario, initial data is collected based on the maximum acquisition duration or the maximum number of image frames acquired, and multi-source point trace data packets output by the sonar module and the photoelectric module are acquired.

[0083] Furthermore, the target point data packet in the multi-source point data packet output by the sonar module and the photoelectric module in step S1 includes the source of the target point, the target point tag, the target point data packet generation time, the longitude and latitude fixed-point coding values ​​of the target point, the source's confidence in the target point, the quality evaluation value of the target point, and extended fields. The extended fields are used to store the target speed, heading, equipment number, internal target number of the source, and other additional information.

[0084] Furthermore, with This indicates the image frame number during the initial data collection phase. One or more targets can be detected in a single image frame, with each target corresponding to a target point data packet. ,in This indicates the target point data packet number. Additionally, it sets... Indicates the first The number of targets detected in the frame image. The target point data packets output by the sonar module and the photoelectric module can be uniformly represented as:

[0085]

[0086] in, The value indicates the source of the data point; 0 indicates that the data point is output by the sonar module, and 1 indicates that the data point is output by the photoelectric module. Indicates the target label; Indicates the time when the trace data packet was generated; and This represents the longitude and latitude fixed-point code values; Indicates the source's confidence level in the trace; This indicates the quality evaluation value of the dot; This is an extended field used to store target speed, heading, equipment number, source internal target number, and other additional information.

[0087] S2. For the target point data packet in the multi-source point data packet obtained in step S1, perform latitude and longitude decoding and coordinate transformation, and convert the point coordinates to local EN coordinates with the preset reference point as the origin to obtain the target point data packet after coordinate transformation.

[0088] Furthermore, the specific implementation method of step S2 includes the following steps:

[0089] S2.1. The longitude fixed-point encoding value in the data packet of the i-th target point obtained in step S1 and latitude fixed point code value The fields are converted to 32-bit signed integers using the same number of binary bits, and then converted to actual longitude and latitude in degrees according to the I31 fixed-point angle scaling factor. The expression is:

[0090]

[0091]

[0092] in, Let be the 32-bit signed integer corresponding to the fixed-point longitude code value in the data packet of the i-th target point. is a 32-bit signed integer corresponding to the latitude fixed-point encoding value in the i-th target point trace data packet; This means converting an unsigned integer to a 32-bit signed integer based on its original binary bits; Let i be the actual longitude of the data packet for the i-th target point. The actual latitude of the i-th target point trace data packet;

[0093] S2.2. Based on the preset reference origin The coordinates of the i-th target point data packet are converted to local EN coordinates, expressed as:

[0094]

[0095] in, Let be the local coordinates of the i-th target point data packet. Represents the coordinate transformation function. and Let be the coordinates of the i-th target point data packet relative to the reference origin in the east and north directions, respectively. The actual longitude of the preset reference origin, The actual latitude of the preset reference origin, This is the matrix transpose.

[0096] S3. For the target point data packet obtained after coordinate transformation in step S2, the effective image frames are filtered using a time window method, including setting point number thresholds and spatial span thresholds for point frames entering the time window, to obtain a set of effective point traces.

[0097] Furthermore, the expressions for filtering by point count threshold and spatial span threshold in step S3 are as follows:

[0098]

[0099] in, For the first The number of targets detected in the frame image The minimum number of targets per frame threshold; and They represent the first The spatial span of all target points in the frame image in the east and north directions; and These are the maximum allowable spans in the east and north directions, respectively;

[0100] Image frames that meet the above conditions are retained, and the local coordinate system coordinates of all target point data packets contained therein are summarized to form a valid point set. It is used for subsequent spatial structure restoration.

[0101] Furthermore, The maximum number of image frames during the initial data collection phase is set to 120, with a setting of 50. Maximum allowable span in direction Set to 30.0m. Maximum allowable span in direction The parameters can be set according to the target arrangement range and the size of the detection area. These parameters can be adjusted based on the detection scenario, target scale, and device sampling frequency.

[0102] S4. Perform PCA spatial attitude correction on the set of valid points obtained in step S3, calculate the mean and covariance matrix of the points, extract the eigenvector corresponding to the largest eigenvalue as the principal axis, and use its orthogonal direction as the secondary axis to establish a structural adaptive coordinate system.

[0103] Furthermore, the specific implementation method of step S4 includes the following steps:

[0104] S4.1. For the set of valid points Calculate the mean of the dots :

[0105]

[0106] Where n is the total number of valid points in the set of valid points;

[0107] Then calculate the covariance matrix. :

[0108] ;

[0109] S4.2. For the covariance matrix Perform eigenvalue decomposition and extract the eigenvector corresponding to the largest eigenvalue. As the principal axis direction, take the eigenvector orthogonal to it. As the secondary axis direction, project the EN coordinates onto the PCA coordinate system:

[0110]

[0111] in, Indicates the direction coordinates of the secondary axis. Indicates the coordinates of the principal axis direction; for Coordinates after PCA projection.

[0112] S5. Extract the hierarchical spatial structure of the effective point set in the structural adaptive coordinate system. First, establish a one-dimensional histogram along the secondary axis and smooth it to extract column peaks. Then, establish a one-dimensional histogram along the principal axis in the point set corresponding to each column peak to extract row peaks. Generate candidate anchor points based on the intersection of column peaks and row peaks.

[0113] Furthermore, the specific implementation method of step S5 includes the following steps:

[0114] S5.1. In the PCA coordinate system, the coordinates of each valid point trace along the secondary axis direction are... A one-dimensional frequency histogram is constructed using statistical data. First, the minimum and maximum values ​​among all secondary axis coordinates are taken as the statistical range of the histogram. Then, the histogram is constructed according to the preset bin width. The statistical range of the histogram is divided into several continuous intervals, and the number of points falling into each interval is counted, thus obtaining a one-dimensional histogram along the secondary axis. ;

[0115] S5.2. Suppress local noise using moving average, Gaussian smoothing, or other smoothing methods. Search for the peak positions of the smoothed one-dimensional histogram along the secondary axis to obtain the peak set. :

[0116]

[0117] in, Let m be the j-th column peak, and m be the total number of column peaks.

[0118] S5.3. For each peak Select the one that satisfies The dot pattern as the first Build a set of points and then construct a one-dimensional histogram along the principal axis. Extracting the peak set :

[0119]

[0120] in, For the first The first peak was extracted from the set of corresponding points. Individual peak positions;

[0121] S5.4. Generate candidate anchor points from the intersection of column peaks and row peaks:

[0122]

[0123] in, For the first The first peak and its first Candidate anchor points formed by the intersection of individual peaks;

[0124] Then, all candidate anchor points are numbered sequentially according to the index order of column peaks and row peaks, and uniformly denoted as candidate anchor points. ,in , This represents the total number of candidate anchor points.

[0125] S6. Perform multi-source point fusion with candidate anchor points as constraints, including candidate anchor point neighborhood selection, cross-source association, calculation of point weights, and extraction of fusion centers;

[0126] Furthermore, the specific implementation method of step S6 includes the following steps:

[0127] S6.1. For the candidate anchor points obtained in step S5, search for points within their neighborhood in the PCA coordinate system to construct a candidate fusion set, expressed as:

[0128]

[0129] in, For the candidate fusion set, For the current moment of integration, The radius of the anchor point's neighborhood. As a time threshold, For the first The timestamp of the generation of each target point trace data packet. Let i be the target point trace and candidate anchor point in the PCA coordinate system. The Euclidean distance;

[0130] Furthermore, spatial neighborhood constraints can prevent adjacent target points from being incorrectly associated; time threshold constraints can eliminate historically lagging points, reducing the impact of asynchronous arrival of sonar and photoelectric data on the fusion results. When the same point simultaneously meets the screening criteria of multiple candidate anchor points, it is assigned to the nearest candidate anchor point. Target category and source information are further determined in subsequent cross-source association processes.

[0131] S6.2. Cross-source association: The spatial region defined by the candidate anchor point is used as the common association unit for sonar traces and photoelectric traces. For any candidate anchor point, the sonar traces and photoelectric traces falling within the neighborhood of that anchor point are read respectively, and the association is performed based on the data packet. Distinguish the source of the traces; when traces from different sources fall into the neighborhood of the same candidate anchor point, they are regarded as observations of the same potential target under different detection modules, and are jointly added to the candidate fusion set corresponding to the candidate anchor point;

[0132] S6.3. Calculate the point weights for the points in the candidate fusion set from the cross-source association in step S6.2. The calculation formula is:

[0133]

[0134] in, For the confidence level of the dot, For dot quality, The time difference between the current point and the current fusion time. The distance between the point and the candidate anchor point. and These are the weighting coefficients for confidence and quality, respectively. The time decay coefficient, This refers to the anchor point distance scale parameter;

[0135] The fusion weights can be determined by source reliability, trace confidence, trace quality, time freshness, and distance consistency.

[0136] S6.4. On the candidate fusion set The points in the data are weighted to calculate the fusion center:

[0137]

[0138] in, Anchor point in PCA coordinate system The integration center For the i-th target point trace data packet;

[0139] Then, the secondary axis coordinates and principal axis coordinates of the fusion center are calculated, as shown in the following expression:

[0140]

[0141] in, The coordinates of the fusion center along the secondary axis of the PCA coordinate system. The coordinates of the fusion center along the principal axis of the PCA coordinate system.

[0142] When the sum of the weights of all participating points is lower than a preset threshold, the system may temporarily not output a new fusion center, but instead use the anchor point position from the previous moment or mark the candidate anchor point as a low-confidence anchor point.

[0143] Furthermore, in step S6, a consistency check is performed on the traces in the candidate fusion set. If a trace meets any of the following conditions, it is marked as an abnormal trace and removed from the list: the distance between the trace and the candidate anchor point exceeds the threshold, the trace confidence is lower than the threshold, or the trace timestamp is significantly delayed or advanced. After the abnormal trace is removed, the fusion center is recalculated.

[0144] Furthermore, even when only some sources report traces at a given moment, the fusion result can still be output based on the arrived source traces, historical anchor point locations, and source reliability. Sources that did not report at that moment do not participate in the weighting, but this will not cause the fusion link to be interrupted. For continuously missing sources, their expected contribution in subsequent moments is gradually reduced to avoid the missing sources causing a lag in the fusion result.

[0145] S7. Inversely transform the fusion center from the PCA coordinate system back to the local EN coordinate system and restore it to latitude and longitude coordinates;

[0146] Furthermore, the expression for inversely transforming the fusion center from the PCA coordinate system back to the local EN coordinate system in step S7 is as follows:

[0147]

[0148] in, This represents the local EN fusion coordinates of the candidate anchor point output. and These represent the coordinates of the updated anchor point along the PCA secondary axis and primary axis, respectively.

[0149] S8. Based on the method of steps S1-S7, dynamically update the candidate anchor point and fusion center by combining the current anchor point with the newly arrived data packet, and output the unified target location, fusion point track or fusion track.

[0150] Furthermore, the expression for smoothly updating the fusion center and the anchor point position at the current time step in step S8 is as follows:

[0151]

[0152] in, Let this be the updated anchor point position at time t. This represents the anchor point position at the previous time t-1. To update the coefficients.

[0153] The following is an application example of this embodiment:

[0154] Based on existing marine experimental data, the sonar module and the optoelectronic module respectively output target point data packets. The fusion time window was set to 20 seconds, the total number of experimental data frames was 271, the maximum number of frames for screening was 120, the number of usable frames after screening was 80, and the total number of original points was 5313. Key parameters included: minimum number of points per frame threshold of 50; maximum allowable span in the East direction of 30.0m; effective matching distance threshold between candidate points and ground truth points of 10.0m; bin width of column peak histogram in the secondary axis direction of 0.8m; and bin width of row peak histogram in the primary axis direction of 1.0m. Figure 2The number of points in each frame and the spatial span in the East direction are given to illustrate the criteria for selecting valid frames. Figure 3 The spatial distribution of all filtered points in the EN coordinate system is given, and the positions of the ground truth points are superimposed. Figure 4 The density distribution, peak positions, and candidate points of the points are given after they are projected onto the PCA principal and secondary axis coordinate systems. Figure 5 The results of the secondary axis direction histogram, smoothing curve, and peak extraction are presented. Figure 6 The results of row peak extraction along the main axis of the three target columns are presented. Experimental results show that the system automatically identifies the three target columns, generating a total of 59 candidate points. The total number of ground truth points is 54. According to the preset one-to-one spatial matching rule, there are 53 true positives, 6 false positives, and 1 false negative, corresponding to a Precision of 0.8983, a Recall of 0.9815, and an F1 score of 0.9381. The minimum matching distance is 0.0998m, and the maximum matching distance is 8.8692m. These results demonstrate that the proposed method can stably recover the spatial structure of the target and fuse the target position under conditions of multi-source asynchronous point traces, local noise disturbances, and coordinate skew.

[0155] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0156] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints, characterized in that, Includes the following steps: S1. In a collaborative sensing scenario, initial data is collected based on the maximum acquisition duration or the maximum number of image frames acquired, and multi-source point trace data packets output by the sonar module and the photoelectric module are acquired. S2. For the target point data packet in the multi-source point data packet obtained in step S1, perform latitude and longitude decoding and coordinate transformation, and convert the point coordinates to local EN coordinates with the preset reference point as the origin to obtain the target point data packet after coordinate transformation. S3. For the target point data packet obtained after coordinate transformation in step S2, the effective image frames are filtered using a time window method, including setting point number thresholds and spatial span thresholds for point frames entering the time window, to obtain a set of effective point traces. S4. Perform PCA spatial attitude correction on the set of valid points obtained in step S3, calculate the mean and covariance matrix of the points, extract the eigenvector corresponding to the largest eigenvalue as the principal axis, and use its orthogonal direction as the secondary axis to establish a structural adaptive coordinate system. S5. Extract the hierarchical spatial structure of the effective point set in the structural adaptive coordinate system. First, establish a one-dimensional histogram along the secondary axis and smooth it to extract column peaks. Then, establish a one-dimensional histogram along the principal axis in the point set corresponding to each column peak to extract row peaks. Generate candidate anchor points based on the intersection of column peaks and row peaks. S6. Perform multi-source trace fusion with candidate anchor points as constraints, including candidate anchor point neighborhood selection, cross-source association, calculation of trace weights, and extraction of fusion centers; S7. Inversely transform the fusion center from the PCA coordinate system back to the local EN coordinate system and restore it to latitude and longitude coordinates; S8. Based on the method of steps S1-S7, dynamically update the candidate anchor point and fusion center by combining the current anchor point and the newly arrived data packet, and output the unified target location, fusion point track or fusion track.

2. The sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints according to claim 1, characterized in that, The target point data packet in the multi-source point data packet output by the sonar module and the photoelectric module in step S1 includes the source of the target point, the target point tag, the target point data packet generation time, the longitude and latitude fixed-point coding values ​​of the target point, the source's confidence in the target point, the quality evaluation value of the target point, and extended fields. The extended fields are used to store the target speed, heading, equipment number, internal target number of the source, and other additional information.

3. A sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints according to claim 1 or 2, characterized in that, The specific implementation method of step S2 includes the following steps: S2.

1. The longitude fixed-point encoding value in the data packet of the i-th target point obtained in step S1 and latitude fixed point code value The fields are converted to 32-bit signed integers using the same number of binary bits, and then converted to actual longitude and latitude in degrees according to the I31 fixed-point angle scaling factor. The expression is: in, Let be the 32-bit signed integer corresponding to the fixed-point longitude encoding value in the data packet of the i-th target point. For the latitude fixed-point encoding value in the data packet of the i-th target point; This means converting an unsigned integer to a 32-bit signed integer based on its original binary bits; Let i be the actual longitude of the data packet for the i-th target point. The actual latitude of the i-th target point data packet; S2.

2. Based on the preset reference origin The coordinates of the i-th target point data packet are converted to local EN coordinates, expressed as: in, Let be the local coordinates of the i-th target point data packet. Represents the coordinate transformation function. and Let be the coordinates of the i-th target point data packet relative to the reference origin in the east and north directions, respectively. The actual longitude of the preset reference origin, The actual latitude of the preset reference origin, This is the matrix transpose.

4. The sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints according to claim 3, characterized in that, The expressions for filtering by point count threshold and spatial span threshold in step S3 are as follows: in, For the first The number of targets detected in the frame image The minimum number of targets per frame threshold; and They represent the first The spatial span of all target points in the frame image in the east and north directions; and These are the maximum allowable spans in the east and north directions, respectively; Image frames that meet the above conditions are retained, and the local coordinate system coordinates of all target point data packets contained therein are summarized to form a valid point set. It is used for subsequent spatial structure restoration.

5. The sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints according to claim 4, characterized in that, The specific implementation method of step S4 includes the following steps: S4.

1. For the set of valid points Calculate the mean of the dots : Where n is the total number of valid points in the set of valid points; Then calculate the covariance matrix. : ; S4.

2. For the covariance matrix Perform eigenvalue decomposition and extract the eigenvector corresponding to the largest eigenvalue. As the principal axis direction, take the eigenvector orthogonal to it. As the secondary axis direction, project the EN coordinates onto the PCA coordinate system: in, Indicates the direction coordinates of the secondary axis. Indicates the coordinates of the principal axis direction; for Coordinates after PCA projection.

6. The sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints according to claim 5, characterized in that, The specific implementation method of step S5 includes the following steps: S5.

1. In the PCA coordinate system, the coordinates of each valid point trace along the secondary axis direction are... A one-dimensional frequency histogram was constructed using statistical data. First, the minimum and maximum values ​​of all secondary axis coordinates are taken as the statistical range of the histogram; then, according to the preset bin width... The statistical range of the histogram is divided into several continuous intervals, and the number of points falling into each interval is counted, thus obtaining a one-dimensional histogram along the secondary axis. ; S5.

2. Suppress local noise using moving average, Gaussian smoothing, or other smoothing methods. Search for the peak positions of the smoothed one-dimensional histogram along the secondary axis to obtain the peak set. : in, Let m be the j-th column peak, and m be the total number of column peaks. S5.

3. For each peak Select the one that satisfies The dot pattern as the first Build a set of points and then construct a one-dimensional histogram along the principal axis. Extracting the peak set : in, For the first The first peak was extracted from the set of corresponding points. Individual peak positions; S5.

4. Generate candidate anchor points from the intersection of column peaks and row peaks: in, For the first The first peak and its first Candidate anchor points formed by the intersection of individual peaks; Then, all candidate anchor points are numbered sequentially according to the index order of column peaks and row peaks, and uniformly denoted as candidate anchor points. ,in , This represents the total number of candidate anchor points.

7. The sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints according to claim 6, characterized in that, The specific implementation method of step S6 includes the following steps: S6.

1. For the candidate anchor points obtained in step S5, search for points within their neighborhood in the PCA coordinate system to construct a candidate fusion set, expressed as: in, For the candidate fusion set, For the current moment of integration, The radius of the anchor point's neighborhood. As a time threshold, For the first The timestamp of the generation of each target point trace data packet. Let i be the target point trace and candidate anchor point in the PCA coordinate system. The Euclidean distance; S6.

2. Cross-source association: The spatial region defined by the candidate anchor point is used as the common association unit for sonar traces and photoelectric traces. For any candidate anchor point, the sonar traces and photoelectric traces falling within the neighborhood of that anchor point are read respectively, and the association is performed based on the data packet. Distinguish the source of the traces; when traces from different sources fall into the neighborhood of the same candidate anchor point, they are regarded as observations of the same potential target under different detection modules, and are jointly added to the candidate fusion set corresponding to the candidate anchor point; S6.

3. Calculate the point weights for the points in the candidate fusion set from the cross-source association in step S6.

2. The calculation formula is: in, For the confidence level of the dot, For dot quality, The time difference between the current point and the current fusion time. The distance between the point and the candidate anchor point. and These are the weighting coefficients for confidence and quality, respectively. The time decay coefficient, This refers to the anchor point distance scale parameter; S6.

4. On the candidate fusion set The points in the data are weighted to calculate the fusion center: in, Anchor point in PCA coordinate system The integration center For the i-th target point trace data packet; Then, the secondary axis coordinates and principal axis coordinates of the fusion center are calculated, as shown in the following expression: in, The coordinates of the fusion center along the secondary axis of the PCA coordinate system. The coordinates of the fusion center along the principal axis of the PCA coordinate system.

8. The sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints according to claim 7, characterized in that, In step S6, a consistency check is performed on the traces in the candidate fusion set. If a trace meets any of the following conditions, it is marked as an abnormal trace and removed: the distance between the trace and the candidate anchor point exceeds the threshold, the trace confidence is lower than the threshold, or the trace timestamp is significantly lagging or ahead. After the abnormal trace is removed, the fusion center is recalculated.

9. The sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints according to claim 8, characterized in that, The expression for inversely transforming the fusion center from the PCA coordinate system back to the local EN coordinate system in step S7 is as follows: in, This represents the local EN fusion coordinates of the candidate anchor point output. and These represent the coordinates of the updated anchor point along the PCA secondary axis and primary axis, respectively.

10. The sonar and photoelectric asynchronous point tracking fusion positioning method based on structural anchor point constraints according to claim 9, characterized in that, The expression for smoothly updating the fusion center and the anchor point position at the current time step in step S8 is as follows: in, Let this be the updated anchor point position at time t. This represents the anchor point position at the previous time t-1. To update the coefficients.