Dual-mode trajectory anti-spoofing defense method, system and electronic device

Abstract

This disclosure provides a dual-modal trajectory anti-spoofing defense method, system, and electronic device, belonging to the field of trajectory processing technology. The dual-modal trajectory anti-spoofing defense method includes: acquiring an original fused trajectory generated based on observations from a first sensor and a second sensor, and the corresponding historical trajectory state; determining multi-source reliable information and trajectory risk information of the original fused trajectory based on the first sensor observations, the second sensor observations, the original fused trajectory, and the historical trajectory state; generating multiple candidate trajectories based on multiple different information sources, and determining the weight corresponding to each candidate trajectory based on the multi-source reliable information to obtain a recovered trajectory; and selecting either the original fused trajectory or the recovered trajectory as the output trajectory based on the trajectory risk information and the normal maintenance information of the original fused trajectory. The solution provided in this disclosure can improve the reliability, stability, and anti-spoofing capability of dual-modal trajectory output.

Description

Technical Field

[0001] This disclosure belongs to the field of trajectory processing technology, specifically relating to a dual-modal trajectory anti-spoofing defense method, system, and electronic device. Background Technology

[0002] In applications such as target detection, UAV monitoring, and intelligent sensing, different types of sensors, such as lidar and millimeter-wave radar, are typically used to acquire target observation information. Multimodal fusion models then integrate the observation results from different sensors to generate the target's fused position or continuous trajectory. Compared to single-sensor applications, multi-sensor fusion leverages the complementary information between different sensing modalities to improve the stability and continuity of trajectory estimation.

[0003] Existing dual-modal trajectory fusion methods typically assume that the observation information provided by different sensors is accurate and reliable, and directly input the observations from each sensor into the fusion model to generate the fused trajectory. However, when sensor observations are affected by data contamination, false target induction, anomaly injection, or missing observations, these abnormal observations may propagate to the trajectory output during the fusion process, causing the generated fused trajectory to deviate from the actual motion state of the target. When spatial conflicts occur between the observation results of the two sensors, the fusion model may also struggle to determine the reliability of different observation information, thus continuously outputting a trajectory influenced by abnormal information.

[0004] Some trajectory anomalies also exhibit continuous accumulation characteristics. For example, anomaly observations can gradually change the target position over multiple consecutive moments, causing the trajectory to gradually shift in a relatively smooth manner. The positional change caused by such anomalies at a single moment may be small and difficult to identify through single-frame abrupt change detection, but after continuous accumulation, it can cause significant trajectory deviations. Existing technologies, if primarily based on sensor observations at the current moment to generate fused trajectories, lack a comprehensive evaluation of historical trajectory states, the consistency between different sensor observations, and the reliability of the fused trajectory, making it easy for anomaly observations to be continuously introduced into subsequent trajectories. Summary of the Invention

[0005] The purpose of this disclosure is to provide a dual-modal trajectory anti-spoofing defense method, system, and electronic device, which can improve the reliability, stability, and anti-spoofing capability of dual-modal trajectory output.

[0006] To achieve the above objectives, the technical solution provided in this disclosure is as follows:

[0007] In a first aspect, this disclosure provides a dual-modal trajectory anti-spoofing defense method, which includes: acquiring an original fused trajectory generated based on observations from a first sensor and observations from a second sensor, and the corresponding historical trajectory state; determining multi-source credible information and trajectory risk information of the original fused trajectory based on the first sensor observations, the second sensor observations, the original fused trajectory, and the historical trajectory state; generating multiple candidate trajectories based on multiple different information sources, and determining the weight corresponding to each candidate trajectory based on the multi-source credible information to obtain a recovered trajectory; and selecting either the original fused trajectory or the recovered trajectory as the output trajectory based on the trajectory risk information and the normal maintenance information of the original fused trajectory.

[0008] In one or more embodiments, the first sensor is a lidar, and the second sensor is a millimeter-wave radar; the multi-source trusted information includes first support trusted information characterizing the degree of support of the first sensor observations for the original fused trajectory, second support trusted information characterizing the degree of support of the second sensor observations for the original fused trajectory, cross-modal trusted information characterizing the degree of consistency between the first sensor observations and the second sensor observations, and historical motion trusted information characterizing the degree of consistency between the original fused trajectory and historical trajectory states; the trajectory risk information is determined at least based on the cross-modal trusted information and the historical motion trusted information.

[0009] In one or more embodiments, determining the multi-source reliable information of the original fused trajectory includes: determining a first observation center corresponding to the first sensor observation and a second observation center corresponding to the second sensor observation; determining a first residual between the first observation center and the original fused trajectory, a second residual between the second observation center and the original fused trajectory, a cross-modal residual between the first observation center and the second observation center, and a motion residual between the original fused trajectory and a historical motion reference position obtained based on the historical trajectory state; and determining the first support reliable information, the second support reliable information, the cross-modal reliable information, and the historical motion reliable information based on the first residual, the second residual, the cross-modal reliable information, and the motion reliable information.

[0010] In one or more embodiments, for different types of residuals, a first reference boundary and a second reference boundary higher than the first reference boundary are determined according to the statistical distribution of the corresponding type of residuals in the normal training sequence; and the current residual is mapped to the corresponding confidence score according to the relationship between the current residual and the first reference boundary and the second reference boundary.

[0011] In one or more embodiments, the historical trajectory state includes trajectory states and evidence information corresponding to multiple consecutive historical frames; the historical motion reference position corresponding to the current frame is determined according to the historical trajectory state; the historical motion credibility information is determined according to the motion deviation between the original fused trajectory and the historical motion reference position; and historical motion candidates are generated from the multiple candidate trajectories based on the historical motion reference position.

[0012] In one or more embodiments, the plurality of candidate trajectories includes: original fused trajectory candidates generated based on the original fused trajectory; first single-mode observation candidates generated based on the observations of the first sensor; second single-mode observation candidates generated based on the observations of the second sensor; historical motion candidates generated based on the historical trajectory states; and residual recovery candidates generated by applying a local correction to the original fused trajectory.

[0013] In one or more embodiments, the weights corresponding to each candidate trajectory are determined based on the multi-source reliable information, the historical trajectory state, and the validity of each candidate trajectory; when the first sensor observation or the second sensor observation is missing or the corresponding reliability is reduced, the weight of the corresponding single-modal observation candidate is reduced; when the consistency between the first sensor observation and the second sensor observation is reduced and / or the consistency between the original fused trajectory and the historical trajectory state is reduced, the weight of at least one of the historical motion candidate and the residual recovery candidate is increased.

[0014] In one or more embodiments, the normal maintenance information includes a normal maintenance probability determined based on the historical trajectory status, the multi-source trusted information, and the status of the multiple candidate trajectories; when the normal maintenance probability meets a preset normal maintenance condition and the trajectory risk represented by the trajectory risk information meets a preset low-risk condition, the original fused trajectory is selected as the output trajectory; otherwise, the recovered trajectory is selected as the output trajectory.

[0015] Secondly, this disclosure provides a dual-modal trajectory anti-spoofing defense system, comprising: an acquisition module for acquiring an original fused trajectory generated based on observations from a first sensor and a second sensor, and the corresponding historical trajectory state; an evaluation module for determining multi-source credible information and trajectory risk information of the original fused trajectory based on the first sensor observations, the second sensor observations, the original fused trajectory, and the historical trajectory state; a recovery module for generating multiple candidate trajectories based on multiple different information sources, and determining the weight corresponding to each candidate trajectory based on the multi-source credible information to obtain a recovered trajectory; and a selection module for selecting either the original fused trajectory or the recovered trajectory as the output trajectory based on the trajectory risk information and the normal maintenance information of the original fused trajectory.

[0016] Thirdly, this disclosure provides an electronic device, including a memory, a processor, and program instructions stored in the memory and executable on the processor, wherein the processor, when executing the program instructions, implements the dual-modal trajectory anti-spoofing defense method as described above.

[0017] This disclosure evaluates the multi-source reliable information and trajectory risk information of the original fused trajectory by using observations from the first sensor, observations from the second sensor, the original fused trajectory, and the historical trajectory status. This enables the system to identify whether the trajectory is affected by abnormal observations, cross-modal conflicts, or continuous offsets from two dimensions: current multi-source observations and temporal continuity. Based on this, multiple candidate trajectories are generated based on different information sources, and the weights of each candidate trajectory are dynamically adjusted according to the multi-source reliable information. This allows the recovered trajectory to adaptively select more reliable information sources according to different anomaly types, reducing the risk of recovery failure caused by fixed reliance on a single sensor. Finally, the original fused trajectory or the recovered trajectory is selected as the output trajectory by combining trajectory risk information and normal maintenance information. This allows the system to promptly correct trajectories affected by anomalies under high-risk conditions and retain the original fusion results under low-risk conditions, reducing the additional errors introduced by over-correction, thereby improving the reliability, stability, and anti-spoofing ability of the dual-modal trajectory output. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a dual-modal trajectory anti-spoofing defense method in one embodiment of this disclosure;

[0020] Figure 2 This is a schematic diagram illustrating the trajectory recovery effect in a continuous trajectory hijacking scenario according to one embodiment of this disclosure;

[0021] Figure 3 This is a schematic diagram illustrating the trajectory recovery effect in a cross-modal conflict scenario according to one embodiment of this disclosure;

[0022] Figure 4 This is a schematic diagram illustrating the trajectory recovery effect in a continuous pseudo-target inducement scenario according to one embodiment of the present disclosure;

[0023] Figure 5 This is a schematic diagram of a dual-modal trajectory anti-spoofing defense system in one embodiment of the present disclosure;

[0024] Figure 6This is a schematic diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions in this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0026] While dual-modal trajectory fusion models can integrate observation information from different sensors to reduce trajectory estimation errors caused by sparse point clouds, missing observations, or environmental noise from individual sensors, dual-modal fusion itself cannot guarantee the inherent reliability of the fusion result. Existing fusion models typically assume that the input observations from both sensors are true and valid, and assume reasonable temporal and spatial consistency between the two sensors. When one sensor is affected by data contamination, false target injection, missing observations, or other anomalies, this abnormal information may enter the original fused trajectory during the fusion process. When there is continuous conflict between the two sensors, the fusion model may also continuously output a trajectory that deviates from the true target position without effective and reliable judgment.

[0027] Trajectory spoofing can also be highly covert. For example, trajectory anomalies caused by an attack may not manifest as sudden changes in position within a single frame, but rather as gradual alterations in trajectory direction or position through small positional shifts over multiple consecutive frames. When observing a single frame, the current trajectory may still conform to the variation range of ordinary measurement noise, but after accumulating across multiple consecutive frames, the original fused trajectory has clearly deviated from the true motion state. Relying solely on single-frame detection or the anomaly filtering of a single sensor is insufficient to promptly identify such smooth drifts or continuous trajectory hijacking.

[0028] Based on the above understanding, this disclosure utilizes observations from a first sensor, observations from a second sensor, the original fused trajectory, and historical trajectory states to establish multi-source reliable information. It analyzes whether the original fused trajectory is supported by different information sources and determines trajectory risk information accordingly. Multiple different information sources further form candidate trajectories, and the multi-source reliable information controls the weight of each candidate trajectory to form a recovered trajectory. The system can also determine whether the current frame truly needs recovery based on the normal preservation information of the original fused trajectory, avoiding frequent modifications to the original fused trajectory in normal scenarios.

[0029] The original fused trajectory, as referred to in this disclosure, can mean the target trajectory obtained after the observations from the first and second sensors are processed by a dual-modal fusion model and a trajectory tracking module. Taking the current frame as frame t as an example, the original fused trajectory can be represented as follows: . The original fused trajectory can represent the position of the current target in three-dimensional space, and may also include trajectory state information such as position, velocity, and movement direction. The present disclosure mainly takes the example that the original fused trajectory includes the three-dimensional position of the target for description.

[0030] The historical trajectory state referred to in the present disclosure may include trajectory states in a plurality of consecutive historical frames before the current frame, and may further include evidence information such as sensor observation states, residuals, credibility information, the number of observation points, and observation space range corresponding to each historical frame. The historical trajectory state is used to provide the temporal context of the recent movement of the target, so that the current frame trajectory can be rechecked in combination with the recent movement law.

[0031] In an exemplary embodiment, the first sensor is a LiDAR, and the second sensor is a millimeter-wave radar. The observation of the first sensor is the LiDAR point cloud collected by the LiDAR, and the observation of the second sensor is the Radar point trace collected by the millimeter-wave radar. The LiDAR point cloud can provide the three-dimensional spatial structure information of the target, and the Radar point trace can provide relatively independent target position observation. After time synchronization and spatial registration, the two sensors are jointly used to generate the original fused trajectory.

[0032] Refer Figure 1 , the anti-spoofing defense method for bimodal trajectories provided by the present disclosure specifically includes steps S101 to S104.

[0033] S101: Obtain the original fused trajectory generated based on the observation of the first sensor and the observation of the second sensor, and the corresponding historical trajectory state.

[0034] When the current frame is the t-th frame, the observation of the first sensor, the observation of the second sensor and the original fused trajectory of the current frame can be obtained , the original fused trajectory is obtained based on the observation of the first sensor and the observation of the second sensor corresponding to the same target. The observation of the first sensor and the observation of the second sensor can be sent to a bimodal fusion network after time synchronization, and can also be matched according to time stamps under the condition that a certain time deviation is allowed.

[0035] Taking the first sensor being LiDAR and the second sensor being millimeter-wave radar as an example, the first sensor observation of the current frame may include the target point cloud obtained by the LiDAR, and the second sensor observation may include one or more point traces formed by the millimeter-wave radar for the same target. The original bimodal fusion model obtains the three-dimensional position observation of the target according to the observation features of the two sensors, and the trajectory tracking module generates the original fused trajectory of the current frame according to the current three-dimensional position observation and the previous trajectory state .

[0036] The historical trajectory state can be composed of the trajectory states and evidence information of the most recent W consecutive frames. Let the comprehensive feature corresponding to the t-th frame be... Then the historical state sequence corresponding to the current frame It can be represented as:

[0037] ;

[0038] Where W represents the historical window length, and each comprehensive feature It may include at least one of the following: the original fused trajectory of the corresponding frame, trajectory velocity, first sensor observation center, second sensor observation center, different types of residuals, multi-source reliable information, number of sensor observation points, and observation spatial range.

[0039] Trajectory velocity can be calculated from two or more adjacent trajectory points. The number of sensor observation points can reflect whether there are obvious observation gaps or anomalies in the current sensor, and the observation spatial range can reflect the distribution scale of the point cloud or point trace in space. When false targets are injected, anomalies increase, or the target point cloud structure changes significantly, the number of observation points and the observation spatial range may change simultaneously, which can serve as supplementary basis for subsequent credibility judgment and candidate weight determination.

[0040] The historical trajectory status is not only used to save historical trajectory points, but the system can also use the historical trajectory status to obtain recent changes in the target's position, velocity, sensor consistency, and observation quality, thereby determining whether the current original fused trajectory is a continuation of the normal motion state or has deviated from the recent motion trend.

[0041] In one exemplary embodiment, the historical window length W can be set to 12, meaning that the historical trajectory state corresponding to the current frame is combined with the trajectory states and evidence information of the most recent 12 frames. The historical window length can also be determined based on the target's motion speed, sensor sampling frequency, and defense response speed. When the window is shorter, the system responds faster to changes in motion; when the window is longer, richer temporal context can be obtained to identify small cumulative offsets over multiple frames.

[0042] When the first and second sensor observations have different sampling frequencies, cross-modal matching can be performed based on the timestamps of the two sensors. In one specific embodiment, the maximum allowed time difference between the two sensor observations is 0.3 s. When the time difference between the two observations does not exceed 0.3 s, they can be considered to correspond to the target state near the same moment and used for fusion or consistency analysis. When the time difference exceeds the maximum time difference, the effectiveness of the corresponding observations can be reduced, or the set of observations can be omitted from calculating cross-modal reliability information.

[0043] S102: Based on the observations of the first sensor, the observations of the second sensor, the original fused trajectory, and the historical trajectory status, determine the multi-source reliable information and trajectory risk information of the original fused trajectory.

[0044] Step S102 is used to check whether the original fused trajectory is supported by the current sensor observations and recent motion patterns. The multi-source reliable information here does not only determine whether a certain sensor itself is abnormal, but evaluates the original fused trajectory from the support relationship of single-mode observations, the cross-modal relationship between the two sensors, and the time relationship between the current trajectory and recent motion patterns.

[0045] In one exemplary embodiment, the multi-source trusted information includes first supporting trusted information, second supporting trusted information, cross-modal trusted information, and historical motion trusted information.

[0046] The first support credibility information characterizes the degree of support of the first sensor observation for the original fused trajectory. When the target position reflected by the first sensor observation is close to the original fused trajectory, the first support credibility information indicates that the first sensor observation has strong support for the original fused trajectory; when the deviation between the two increases, the first support credibility information decreases accordingly. The second support credibility information characterizes the degree of support of the second sensor observation for the original fused trajectory, and its judgment method can be similar to that of the first support credibility information.

[0047] Cross-modal reliability information characterizes the degree of consistency between observations from the first and second sensors. Even if both sensors produce valid observations, they may point to different spatial locations. Cross-modal reliability information can directly reflect whether a significant conflict has occurred between the two sensors.

[0048] Historical motion reliability information characterizes the degree of consistency between the original fused trajectory and the historical trajectory state. If the current original fused trajectory deviates significantly from the recent motion direction or recent positional change trend, the historical motion reliability information decreases. This information can help identify trajectory drift that is not obvious in a single frame but gradually accumulates over multiple consecutive frames.

[0049] Specifically, a first observation center corresponding to the first sensor observation and a second observation center corresponding to the second sensor observation can be determined respectively. The first observation center can be represented as... The second observation center can be represented as .

[0050] Taking a lidar as the first sensor as an example, a first observation center can be determined based on multiple three-dimensional points belonging to the target in the lidar point cloud. The first observation center can employ a robust observation center to reduce the impact of a small number of outliers or anomalies on the observation center. For example, outliers that significantly deviate from the target's main point cluster can be removed first, and then the centroid can be calculated based on the retained points. Alternatively, the first observation center can be determined based on the median coordinates, the truncated mean, the center of the main cluster, or the confidence weights of different points. This disclosure does not limit the robust observation center to a fixed calculation method; any reference position reflecting the target's main spatial location obtained from sensor observations is acceptable.

[0051] When the second sensor is a millimeter-wave radar, the second observation center can be determined based on one or more Radar points belonging to the current target. When multiple Radar points exist, the second observation center can be determined based on the average, median, confidence-weighted result, or the center of the main point cluster.

[0052] After obtaining the first observation center Second Observation Center Then, the first residual between the first observation center and the original fused trajectory can be calculated. The second residual between the second observation center and the original fused trajectory Cross-modal residuals between the first and second observation centers and the motion residuals between the original fused trajectory and the historical motion reference position. .

[0053] When using the L2 distance, the above residuals can be expressed as:

[0054]

[0055]

[0056]

[0057] ;

[0058] in, This indicates the historical motion reference position of the current frame, obtained based on the historical trajectory state.

[0059] First residual The second residual reflects the spatial deviation between the observation center of the first sensor and the original fused trajectory. The smaller this residual, the more the first sensor observations support the current original fused trajectory. This reflects the spatial deviation between the observation center of the second sensor and the original fused trajectory. Cross-modal residual. By comparing the positional differences between the first and second observation centers, we can identify situations where the two sensors produce observation results, but the two observation results clearly point to different spatial locations.

[0060] Motion residuals This reflects the deviation between the current original fused trajectory and the historical motion reference position obtained based on recent historical motion patterns. When continuous trajectory hijacking or smooth drift occurs, the single-frame sensor residual may not show obvious abrupt changes, but the deviation between the original fused trajectory and recent motion patterns may gradually increase. The motion residual can provide evidence of temporal consistency for such anomalies.

[0061] The four types of residuals mentioned above have different sources, and their numerical distributions and normal fluctuation ranges may differ significantly. For example, lidar can provide denser spatial point clouds, and the fluctuation range of its observation center may differ from that of the point trace center of millimeter-wave radar. Cross-modal residuals and historical motion residuals also have their own different normal statistical distributions. If the same fixed distance threshold is used directly to judge all residuals, it may cause one type of residual to be overly sensitive or overly insensitive.

[0062] In one exemplary embodiment, a first reference boundary and a second reference boundary higher than the first reference boundary can be determined based on the statistical distribution of the corresponding type of residual in the normal training sequence for different types of residuals. The normal training sequence can refer to a data sequence without the addition of attacks, spurious targets, or abnormal contamination.

[0063] The first reference boundary can represent the main fluctuation range of normal residuals, while the second reference boundary can represent the boundary of larger fluctuations that are less likely to occur under normal circumstances but may still exist. In a specific embodiment, the first reference boundary is the 95th quantile P of the corresponding type of residuals in the normal training sequence. 95 The second reference boundary adopts the 99th quantile P of the corresponding type of residual. 99 .

[0064] For any type of current residual d, the corresponding confidence score q(d) can be determined according to the following confidence scoring function:

[0065] ;

[0066] Here, ε is a very small positive number used to avoid the denominator being zero.

[0067] When the current residual d does not exceed P 95 When max(0,dP) 95 A residual of 0 indicates that the confidence score q(d) is close to or equal to 1, meaning the residual is still within the main fluctuation range of the normal training data. When the current residual d exceeds P... 95 Subsequently, the confidence score gradually decreases as the current residual increases. P99 -P 95 Used to adjust the descent scale of the confidence score.

[0068] Based on the above method, we can obtain:

[0069]

[0070]

[0071]

[0072] ;

[0073] in, It can serve as the first supporting reliable information, used to indicate the degree to which the first sensor observations support the original fused trajectory; It can serve as a second, reliable source of information; It can be used as reliable information across modalities; It can serve as reliable information about historical movements.

[0074] By employing a confidence mapping based on the statistical distribution of normal training sequences, residuals from different sources and with different normal fluctuation scales can be transformed to a relatively uniform confidence scale. Subsequent trajectory risk assessment and candidate trajectory weight determination do not require direct comparison of original residuals of different magnitudes, but can instead utilize the unified multi-source confidence information.

[0075] In other implementations, the first and second reference boundaries are not limited to the 95th and 99th quantiles, but can also be determined based on the mean, variance, standard deviation range, other quantiles, or a preset statistical confidence range of the normal training data. The mapping relationship for the confidence scores is also not limited to the aforementioned exponential function; a monotonically decreasing function, a piecewise function, or a mapping relationship obtained from the learning model can be used, as long as the confidence level decreases accordingly when the current residual increases to deviate from the normal fluctuation range.

[0076] Historical Movement Reference Location The short-term history window can be determined based on historical trajectory states. It contains the target's position and velocity changes across multiple recent frames, and can also include the observation centers, residuals, reliability information, number of points, and spatial extent from different sensors. The system obtains the target's recent motion context based on the short-term history window and infers the reference position corresponding to the current frame under normal motion continuation.

[0077] In one exemplary embodiment, a GRU (Gated Recurrent Unit) can be used to encode the historical trajectory state. The GRU sequentially receives the comprehensive features corresponding to each historical frame in the historical window to form historical encoded features that characterize the recent motion state and the trend of evidence changes. Based on the historical encoded features, the GRU outputs the historical motion reference position of the current frame. Since the historical motion reference position is mainly obtained from multiple recent historical frames and does not directly depend on the observation center of a single sensor in the current frame, this historical motion reference position can provide a relatively independent motion reference when the current sensor observation is contaminated.

[0078] This disclosure utilizes the historical movement reference location in a dual way: firstly, based on the original fusion trajectory... Reference position of historical movement Motion residuals between Determining credible information about historical movements First, it is used to determine whether the current original fused trajectory conforms to recent motion patterns; second, it directly uses historical motion reference positions. As a historical motion candidate among multiple subsequent candidate trajectories, the same historical motion reference not only participates in risk assessment, but also directly participates in trajectory recovery under high-risk conditions.

[0079] Trajectory risk information is used to indicate the degree of risk that the original fused trajectory is affected by cross-modal conflicts, continuous offsets, or other anomalies. Trajectory risk information can be determined at least based on cross-modal reliability information and historical motion reliability information.

[0080] In an exemplary embodiment, the trajectory risk score It can be represented as:

[0081] ;

[0082] Cross-modal trusted information A decrease indicates an increased degree of conflict between the first and second sensor observations. Historical motion reliability information. A decrease indicates that the current original fused trajectory deviates from the recent motion trend. Constructing a risk score by taking the smaller of the two values ​​allows the trajectory risk to increase accordingly whenever either consistency significantly decreases.

[0083] First support trusted information Second support trusted information It is mainly used for subsequent candidate trajectory weight determination. For example, when the confidence information of the first support is high and the confidence information of the second support is low, it can increase the relative influence of the candidate trajectory corresponding to the first sensor and reduce the influence of the candidate trajectory corresponding to the second sensor.

[0084] In other implementations, trajectory risk information may also be determined by combining at least one of cross-modal reliable information, historical motion reliable information, first support reliable information, second support reliable information, changes in the number of sensor observation points, changes in the observation spatial range, and changes in trajectory velocity. Trajectory risk information may be represented by a risk score, risk level, or risk category.

[0085] S103: Generate multiple candidate trajectories based on multiple different information sources, and determine the weights corresponding to each candidate trajectory according to the multi-source credible information to obtain the recovered trajectory.

[0086] Step S103 can provide an alternative trajectory source when there is an abnormal risk in the original fused trajectory. Fixed fallback to a certain sensor cannot adapt to different types of attacks, because the attack may occur on the first sensor side, or on the second sensor side, or it may manifest as obvious conflict between the two sensors or simultaneous reduction in observation quality of the two sensors.

[0087] This disclosure preserves candidate trajectories from multiple different information sources, ensuring that trajectory recovery does not depend on a single fixed source. Each candidate trajectory corresponds to a different type of observation or motion data, and the weight of each candidate trajectory is determined through multi-source reliable information.

[0088] In one exemplary embodiment, the multiple candidate trajectories include original fused trajectory candidates, first single-mode observation candidates, second single-mode observation candidates, historical motion candidates, and residual recovery candidates.

[0089] The five candidate trajectories of frame t can be represented as follows:

[0090]

[0091]

[0092]

[0093]

[0094] ;

[0095] in, Candidates for the original fusion trajectory; It is the first single-mode observation candidate; It is a candidate for the second single-mode observation; Candidate for historical movement; Candidates for residual recovery; This represents the local correction amount learned based on current multi-source reliable information, historical trajectory status, or other relevant features.

[0096] The original fused trajectory candidates directly retain the original fused trajectories generated by the existing dual-modal fusion model and trajectory tracking module. In normal scenarios or low-risk conditions, the original fused trajectory usually combines complementary information from the two sensors and maintains good trajectory continuity. Therefore, the original fused trajectory candidate is still an effective source of information in the recovery process.

[0097] The first single-mode observation candidate is generated based on the observation of the first sensor. When the first sensor is a lidar, the first observation center can be directly used. As the first single-mode observation candidate, the corresponding single-mode trajectory candidate can also be generated based on the observation results of multiple consecutive frames from the first sensor.

[0098] The second single-mode observation candidate is generated based on observations from the second sensor. When the second sensor is a millimeter-wave radar, a second observation center can be used. As a second single-mode observation candidate.

[0099] Historical motion candidates are generated based on historical trajectory states, specifically by directly using the historical motion reference positions obtained in step S102. When there are obvious conflicts, point cloud contamination, or missing observations in the current frame sensor observations, historical motion candidates can provide continuity constraints independent of the current observations based on the recent motion state before the attack or before the accumulation of anomalies.

[0100] Residual recovery candidates are obtained by analyzing the original fusion trajectory. Apply local correction amount The local correction amount is obtained. This can be learned by a recovery network. The input to the recovery network can include the encoded results of historical trajectory states, first support confidence information, second support confidence information, cross-modal confidence information, historical motion confidence information, different types of residuals, and the relative positional relationships of each candidate trajectory. The recovery network outputs a three-dimensional correction vector. It is used to correct the original fusion trajectory.

[0101] The residual recovery candidate retains the effective information in the original fused trajectory and makes local corrections based on multi-source evidence. It is suitable for situations where it is difficult to accurately recover the trajectory by simply using a certain observation center or historical motion reference position.

[0102] After obtaining multiple candidate trajectories, the candidate weights corresponding to each candidate trajectory are determined based on multi-source reliable information, historical trajectory status, and the validity of each candidate trajectory.

[0103] Let the candidate trajectory of class k be Its candidate weight is Then the trajectory is restored. It can be represented as:

[0104] ;

[0105] And it satisfies:

[0106] ;

[0107] The larger the candidate weight, the more the recovery trajectory in the current state tends to refer to the corresponding candidate trajectory.

[0108] Candidate weights can be determined by a candidate trajectory selection network. The candidate trajectory selection network can receive multi-source reliable information, the encoded features of short-term historical windows, the spatial relationships between candidate trajectories, and the validity of candidates, and output the candidate score corresponding to each candidate trajectory. Then, the candidate scores are normalized to obtain the candidate weights.

[0109] In one optional implementation, a validity flag can be set for the k-th type of candidate trajectory. When the candidate trajectory is valid, Set to 1. When corresponding sensor observations are missing, the number of observation points is insufficient, the corresponding observations exceed the allowable time synchronization error, or an effective observation center cannot be formed, It can be set to 0. Let the candidate trajectory selection network output the corresponding candidate score. Candidate weights can be determined in the following way:

[0110] ;

[0111] This method excludes invalid candidates from the weighting of the recovered trajectory, while dynamically allocating weights among valid candidates based on their corresponding candidate scores. This formula is only one optional implementation, and this disclosure does not limit the determination of candidate weights to this specific normalization method.

[0112] A correspondence can be established between multi-source reliable information and candidate weights. When the first sensor observation is missing or the first supporting reliable information decreases, the weight of the first single-mode observation candidate can be reduced. When the second sensor observation is missing or the second supporting reliable information decreases, the weight of the second single-mode observation candidate can be reduced. When the consistency between the first and second sensor observations decreases, it indicates a significant conflict between the two sensors. In this case, the system can reduce its dependence on the current single-mode observation or the original fused trajectory and increase the weight of at least one of the historical motion candidate and residual recovery candidate.

[0113] A decrease in the consistency between the original fused trajectory and the historical trajectory indicates that the original fused trajectory has deviated from the recent motion trend. The system can increase the relative role of historical motion candidates, making the recovered trajectory more constrained by the recent normal motion trend. When both sensor observations maintain high confidence, high cross-modal consistency, and high historical motion consistency, the weight of the original fused trajectory candidates can be increased, so that the recovered branch itself is as close as possible to the original fused trajectory.

[0114] Therefore, multi-source reliable information is not only used to detect anomalies, but also directly controls the contribution of different candidate information sources to the recovered trajectory. When the first sensor malfunctions, the first single-mode observation candidate can be weakened. When the second sensor malfunctions, the second single-mode observation candidate can be weakened. When there are significant cross-modal conflicts or continuous trajectory deviations, the role of historical motion candidates or residual recovery candidates can be enhanced. The system does not need to pre-fix any one sensor to always be reliable.

[0115] Step S104: Based on the trajectory risk information and the normal maintenance information of the original fused trajectory, select the original fused trajectory or the recovered trajectory as the output trajectory.

[0116] After the restored trajectory is generated, it does not necessarily replace the original fused trajectory directly. In normal scenes, under random measurement noise, or with slight single-modal degradation, the original fused trajectory may still have high accuracy and continuity. If the defense module forces the use of the restored trajectory for every frame, it may disrupt the stable output already obtained by the original fusion model and generate new trajectory errors. Therefore, this disclosure performs the final trajectory selection based on trajectory risk information and normal preservation information, whereby the normal preservation information indicates whether the original fused trajectory is currently suitable for continued retention.

[0117] In one exemplary embodiment, the normal hold information includes the normal hold probability. The normal retention probability can be output by the normal retention gating branch. This gating branch takes the historical trajectory state, multi-source reliable information, and the states of multiple candidate trajectories as input, and determines whether the current original fused trajectory should be retained through learning.

[0118] The state of a candidate trajectory can include at least one of the following: the position of each candidate trajectory, the relative distance between different candidate trajectories, candidate validity, and candidate weight. When multiple highly credible candidates are close to the original fused trajectory, the basis for normal preservation of the original fused trajectory can be enhanced. When multiple credible candidates deviate significantly from the original fused trajectory, the probability of normal preservation can be reduced.

[0119] Let the normal maintenance threshold be The trajectory risk score is The risk threshold is Then the output trajectory It can be represented as:

[0120] ;

[0121] When the probability of normal trajectory preservation reaches or exceeds the normal trajectory preservation threshold, and the trajectory risk is lower than the risk threshold, the original fused trajectory can be determined to meet the normal trajectory preservation condition, and the original fused trajectory can be directly selected. As the output trajectory. When the probability of normal maintenance is lower than the normal maintenance threshold, or when the trajectory risk reaches or exceeds the risk threshold, the trajectory recovery option can be selected. As the output trajectory.

[0122] Simultaneously employing both the probability of normal maintenance and trajectory risk information can reduce erroneous corrections caused by a single judgment condition. Trajectory risk information primarily reflects whether cross-modal consistency and historical motion consistency are abnormal. The probability of normal maintenance integrates multi-source reliable information, historical trajectory states, and the states of multiple candidate trajectories to further evaluate whether the original fused trajectory is worth retaining. The system only retains the original output when the original fused trajectory has a high probability of normal maintenance and is in a low-risk state.

[0123] In one specific embodiment, the risk threshold can be set to 0.45, and the normal maintenance threshold can be set to 0.72. This parameter is only used to illustrate one specific implementation; in actual applications, it can be adjusted according to the target type, sensor error, environmental conditions, and tolerance for false alarms.

[0124] Through steps S101 to S104 described above, the defense method forms a complete online inference chain. The system first establishes multi-source reliable information using observations from the first and second sensors, the original fused trajectory, and historical trajectory states, and determines trajectory risks. Then, multiple candidate trajectories are generated based on various information sources, and the candidate weights are dynamically determined by the multi-source reliable information. After the recovered trajectory is generated, the system decides whether to actually replace the original fused trajectory based on trajectory risk information and normal maintenance information. This reduces the intervention of the defense module in the original system under normal conditions, and allows for trajectory recovery using more reliable information sources under high-risk conditions.

[0125] The defense model disclosed herein may not use the ground truth trajectory during the online inference phase, but can use the ground truth trajectory as a supervision signal during the training phase, so that candidate trajectory selection, residual recovery and normal maintenance gating learn reasonable output methods under different abnormal states.

[0126] Let the final output trajectory of frame t be... The corresponding truth locus is The three-dimensional position error of the current frame It can be represented as:

[0127] ;

[0128] To constrain the overall trajectory recovery accuracy, a trajectory recovery loss can be set. :

[0129] ;

[0130] Where T represents the length of the training sequence. This represents the trajectory position regression loss. The trajectory position regression loss can be L1 loss, L2 loss, or other loss functions that can constrain the output trajectory to be close to the true trajectory.

[0131] Simply constraining the output trajectory after defense to closely approximate the ground truth trajectory may cause the defense model to frequently modify the original fused trajectory. In normal scenarios, the original fused trajectory itself may already possess high accuracy. To reduce over-correction, a normal preservation loss can be further set.

[0132] remember This indicates whether the training labels of the original fused trajectory should be maintained in the current frame. When the original fused trajectory is within the normal error range, it can be set... =1. When the original fused trajectory exceeds the normal error range, it can be set to 1. =0.

[0133] Normal maintenance loss It can be represented as:

[0134] ;

[0135] The loss primarily constrains the output trajectory to be close to the original fused trajectory in the training frames where the original trajectory should be preserved, thus reducing unnecessary modifications to the normal trajectory by the defense model.

[0136] Errors caused by trajectory spoofing are usually not uniformly distributed. A large number of normal frames may have only small errors, while a small number of high-risk frames may produce severe positional shifts. If the training process only optimizes the average trajectory error, the impact of these severely shifted frames may be diluted by the large number of normal frames.

[0137] To suppress severely misaligned frames, a larger error loss can be set. Let the large error threshold be... Large error loss It can be represented as:

[0138] ;

[0139] Current frame error No more than When the current frame error exceeds a certain threshold, no additional large error penalty will be incurred. At that time, only for those exceeding Additional penalties are added to the error component, and the larger the error, the faster the additional penalties increase. This approach allows the model to pay more attention to severe trajectory deviations that occur under strong anomalies such as persistent false target induction, cross-modal conflicts, and continuous trajectory hijacking.

[0140] Normality maintenance gating can also be trained using a supervised approach. Let the normality maintenance probability output by the model be... Training labels are Then the gate loss It can be represented as:

[0141] ;

[0142] This gated loss increases the probability of normal maintenance when the original fused trajectory is reliable, and decreases the probability of normal maintenance when the original fused trajectory is affected by anomalies.

[0143] Based on the above losses, the overall training objective of the model is... It can be represented as:

[0144] ;

[0145] in, , and These represent the weighting coefficients for normal hold loss, large error loss, and gating loss, respectively.

[0146] The training objective is to enable the model to learn different tasks simultaneously, and the trajectory recovery loss makes the output trajectory closer to the true trajectory. The normal hold loss reduces overcorrection in normal states, the large error loss focuses on constraining severely offset frames, and the gating loss improves the ability to distinguish between low-risk hold and high-risk recovery.

[0147] In one specific embodiment, a large error threshold The value can be set to 3.0m, and the weighting coefficients for the normal maintenance loss, large error loss, and gating loss can be set to 0.35, 0.08, and 0.05, respectively. These parameters can be adjusted based on the training data distribution, target motion scale, and defense mission requirements.

[0148] In one specific embodiment, the method disclosed herein can be trained and validated using the MMAUD multimodal anti-drone dataset, which simultaneously provides lidar observations, millimeter-wave radar observations, and the three-dimensional true location of the drone.

[0149] Normal sequences corresponding to Mavic2, Mavic3, and Phantom4 can be selected as training data, and the P values ​​corresponding to the first residual, second residual, cross-modal residual, and motion residual can be calculated using these normal sequences. 95 With P 99. Sequences such as M300 and Avata can be used for testing.

[0150] In an experimental embodiment, the historical window length W is set to 12, L2 distance is adopted for four types of residuals, the maximum matching time difference between the observation of the first sensor and the observation of the second sensor is set to 0.3s, the trajectory risk threshold is set to 0.45, the normal retention threshold is set to 0.72, the large error threshold is set to 3.0m, the historical trajectory state is encoded by GRU, and the recovery network generates corresponding weights according to five types of candidate trajectories and obtains the recovered trajectory.

[0151] Attacks such as continuous fake target induction, cross-modal conflict and continuous trajectory hijacking can be added to the test sequence to verify the adaptability of the defense model to different trajectory deception modes. Continuous fake target induction can add a fake target with gradually changing position to the observation of a certain sensor through multiple consecutive frames, so that the original fused trajectory is gradually induced by the fake target. This scenario can verify whether the system can reduce the influence of attacked observations through single-modal trusted information, historical motion trusted information and dynamic candidate weights.

[0152] Cross-modal conflict can make the observation of the first sensor and the observation of the second sensor point to different target positions respectively. At this time, the cross-modal residual increases, the cross-modal trusted information decreases, and the trajectory risk increases. The recovery module can reduce the influence of abnormal candidates on the recovered trajectory, and refer more to trusted single-modal candidates, historical motion candidates or residual recovery candidates.

[0153] Continuous trajectory hijacking can gradually change the original fused trajectory through small position offsets of multiple consecutive frames. Since the change of a single frame may be small, single-frame anomaly detection may not be able to identify it in time. The present disclosure calculates the historical motion reference position and historical motion trusted information in combination with the historical trajectory state, and can identify the phenomenon that the trajectory gradually deviates from the recent motion trend from the perspective of temporal continuity.

[0154] Refer Figure 2 , Figure 3 and Figure 4 As shown, in the continuous trajectory hijacking scenario, the average displacement error (ADE) of the method of the present disclosure is 5.739, the root mean square error (RMSE) is 7.052, and the P90 error is 12.165. In the cross-modal conflict scenario, ADE is 1.594, RMSE is 2.017, P90 error is 2.223, and the trajectory coverage rate Overlap@3m within the 3m tolerance reaches 0.969. In the continuous fake target induction scenario, ADE is 2.129, RMSE is 2.717, P90 error is 3.401, and Overlap@3m is 0.817. The above results show that multi-source trusted information, historical trajectory review, dynamic candidate trajectory recovery and normal retention gating can jointly reduce the influence of abnormal observations on the final output trajectory.

[0155] The method disclosed herein does not require modification of the original fused trajectory for every frame. Taking a normal frame as an example, both the first and second observation centers are close to the original fused trajectory, the first and second support confidence information are high, the cross-modal residual between the two observation centers is small, the cross-modal confidence information is high, and the original fused trajectory is also close to the historical motion reference position, with high historical motion confidence information. At this time, the trajectory risk is low, and the normal hold gating can also output a high normal hold probability, and the system directly outputs the original fused trajectory.

[0156] Taking an attack on the first sensor as an example, the first observation center gradually deviates from the original fused trajectory and the second observation center, leading to a decrease in the first support credibility information and cross-modal credibility information. The candidate trajectory selection network can reduce the weight of the first single-modal observation candidate and increase the weight of the second single-modal observation candidate, historical motion candidate, or residual recovery candidate based on the second support credibility information, historical motion credibility information, and the effectiveness of other candidates. When the trajectory risk reaches a high level or the probability of normal maintenance decreases, the system outputs the recovered trajectory.

[0157] Taking a collision between two sensors as an example, both the first and second support confidence information may have certain values, but the distance between the first and second observation centers increases significantly, and the cross-modal confidence information decreases significantly. The trajectory risk increases accordingly. The system further determines a more reliable recovery source based on the relationship between each single-modal candidate and the historical trajectory state, avoiding the fixed use of a sensor simply because it produces observation results.

[0158] Taking continuous trajectory drift as an example, the changes in the original fused trajectory within a single frame may remain relatively smooth, and there may not be any drastic conflict between the first and second observation centers. However, the original fused trajectory gradually deviates from the historical motion reference position obtained based on recent historical trajectory states. The motion residual continuously increases, the reliability of historical motion information continuously decreases, and the trajectory risk increases. The system can enhance the role of historical motion candidates and residual recovery candidates to mitigate the cumulative impact of continuous drift on the final output trajectory.

[0159] Please refer to Figure 5 As shown, this disclosure also provides a dual-modal trajectory anti-spoofing defense system 500, which includes an acquisition module 501, an evaluation module 502, a recovery module 503, and a selection module 504.

[0160] The acquisition module 501 is used to acquire the original fused trajectory generated based on the observations of the first sensor and the second sensor, as well as the corresponding historical trajectory state. The acquisition module 501 can receive the original fused trajectory output by the original dual-modal trajectory reproduction model, or simultaneously read the corresponding first and second sensor observations from the sensor data interface or the cache unit. The historical trajectory state can be stored in a historical state cache and updated in a sliding window manner with the arrival of a new frame of data.

[0161] The evaluation module 502 is used to determine the multi-source reliable information and trajectory risk information of the original fused trajectory based on the observations of the first sensor, the observations of the second sensor, the original fused trajectory, and the historical trajectory status. The evaluation module 502 may include an observation center determination unit, a residual calculation unit, a reliability mapping unit, a historical motion modeling unit, and a risk determination unit. The observation center determination unit determines the first observation center and the second observation center. The residual calculation unit calculates the first residual, the second residual, the cross-modal residual, and the motion residual. The reliability mapping unit maps different types of residuals to first support reliable information, second support reliable information, cross-modal reliable information, and historical motion reliable information. The historical motion modeling unit obtains the historical motion reference position based on the historical trajectory status. The risk determination unit determines the trajectory risk information based at least on the cross-modal reliable information and the historical motion reliable information.

[0162] The recovery module 503 generates multiple candidate trajectories based on multiple different information sources, and determines the weights corresponding to each candidate trajectory according to the multi-source reliable information to obtain the recovered trajectory. The recovery module 503 can generate original fused trajectory candidates, first single-mode observation candidates, second single-mode observation candidates, historical motion candidates, and residual recovery candidates. The recovery module 503 determines the weights corresponding to the five types of candidate trajectories according to the multi-source reliable information, historical trajectory status, and candidate validity, and calculates the weighted result to obtain the recovered trajectory.

[0163] Selection module 504 is used to select either the original fused trajectory or the recovered trajectory as the output trajectory based on trajectory risk information and normal maintenance information of the original fused trajectory. Selection module 504 may include a normal maintenance gating branch. The normal maintenance gating branch outputs the normal maintenance probability based on historical trajectory status, multi-source trusted information, and the status of multiple candidate trajectories. When the normal maintenance probability meets a preset normal maintenance condition and the trajectory risk meets a preset low-risk condition, selection module 504 outputs the original fused trajectory. Otherwise, selection module 504 outputs the recovered trajectory.

[0164] The acquisition module 501, evaluation module 502, recovery module 503, and selection module 504 can be implemented as software modules, or by the processor executing corresponding program instructions, or in combination with programmable logic devices, application-specific integrated circuits, or other data processing hardware. Each module can be set independently, or two or more modules can be integrated into the same functional module.

[0165] The disclosed dual-modal trajectory anti-spoofing defense system 500 can be set on the trajectory output side of the original dual-modal trajectory reproduction model, which is responsible for generating the original fused trajectory based on the observations of the first and second sensors. This disclosed dual-modal trajectory anti-spoofing defense system can further verify the credibility of the original fused trajectory without replacing the original fusion network and trajectory tracking process, and generate a recovered trajectory when a high trajectory risk is identified.

[0166] Please refer to Figure 6 As shown, embodiments of this disclosure also provide an electronic device 600, which includes at least one processor 601, a memory 602 (e.g., non-volatile memory), a main memory 603, and a communication interface 604, wherein the at least one processor 601, the memory 602, the main memory 603, and the communication interface 604 are connected together via an internal bus 605. The at least one processor 601 is configured to invoke at least one program instruction stored or encoded in the memory 602 to cause the at least one processor 601 to perform various operations and functions of the bimodal trajectory anti-spoofing defense method described in various embodiments of this specification.

[0167] The electronic device 600 can communicate with lidar, millimeter-wave radar, and other trajectory reproduction devices, and can also be integrated into anti-drone sensing devices, edge computing devices, or servers. After receiving dual-modal observations and the original fused trajectory, the electronic device 600 determines multi-source reliable information and trajectory risk information according to the above steps, generates a recovered trajectory, and selects either the original fused trajectory or the recovered trajectory as the final output trajectory.

[0168] In the embodiments of this specification, electronic device 600 may include, but is not limited to: personal computer, server computer, workstation, desktop computer, laptop computer, notebook computer, mobile electronic device, smartphone, tablet computer, cellular phone, personal digital assistant (PDA), handheld device, messaging device, wearable electronic device, consumer electronic device, etc.

[0169] This disclosure also provides a machine-readable medium carrying program instructions that, when executed by a processor, can be used to implement various operations and functions of the dual-modal trajectory anti-spoofing defense method described in the various embodiments of this specification.

[0170] The machine-readable medium in this disclosure can be a machine-readable signal medium or a machine-readable storage medium, or any combination thereof. A machine-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a machine-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable machine disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a machine-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0171] In this disclosure, the machine-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying machine-readable program instructions. The propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The machine-readable signal medium may also be any machine-readable medium other than a machine-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program instructions contained on the machine-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0172] This disclosure also provides a computer program product, which includes program instructions that, when executed by a processor, can implement the above-described dual-modal trajectory anti-spoofing defense method.

[0173] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, systems, and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by program instructions. These program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0175] It will be apparent to those skilled in the art that this disclosure is not limited to the details of the exemplary embodiments described above, and that this disclosure can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of this disclosure is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this disclosure. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0176] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A dual-modal trajectory anti-spoofing defense method, characterized in that, include: Obtain the original fused trajectory generated based on the observations of the first sensor and the second sensor, as well as the corresponding historical trajectory status; Based on the observations from the first sensor, the observations from the second sensor, the original fused trajectory, and the historical trajectory status, the multi-source reliable information and trajectory risk information of the original fused trajectory are determined; Multiple candidate trajectories are generated based on multiple different information sources, and the weights corresponding to each candidate trajectory are determined according to the multi-source credible information to obtain the recovered trajectory. Based on the trajectory risk information and the normal maintenance information of the original fused trajectory, either the original fused trajectory or the recovered trajectory is selected as the output trajectory.

2. The dual-modal trajectory anti-spoofing defense method according to claim 1, characterized in that, The first sensor is a lidar, and the second sensor is a millimeter-wave radar; The multi-source trusted information includes first support trusted information characterizing the degree of support of the first sensor observation for the original fused trajectory, second support trusted information characterizing the degree of support of the second sensor observation for the original fused trajectory, cross-modal trusted information characterizing the degree of consistency between the first sensor observation and the second sensor observation, and historical motion trusted information characterizing the degree of consistency between the original fused trajectory and the historical trajectory state. The trajectory risk information is determined at least based on the cross-modal reliable information and the historical motion reliable information.

3. The dual-modal trajectory anti-spoofing defense method according to claim 2, characterized in that, Determining the multi-source reliable information of the original fusion trajectory includes: The first observation center corresponding to the observation of the first sensor and the second observation center corresponding to the observation of the second sensor are determined respectively; Determine the first residual between the first observation center and the original fused trajectory, the second residual between the second observation center and the original fused trajectory, the cross-modal residual between the first observation center and the second observation center, and the motion residual between the original fused trajectory and the historical motion reference position obtained based on the historical trajectory state; Based on the first residual, the second residual, the cross-modal residual, and the motion residual, the first support credibility information, the second support credibility information, the cross-modal credibility information, and the historical motion credibility information are determined respectively.

4. The dual-modal trajectory anti-spoofing defense method according to claim 3, characterized in that, For different types of residuals, a first reference boundary and a second reference boundary higher than the first reference boundary are determined based on the statistical distribution of the corresponding type of residuals in the normal training sequence. Based on the relationship between the current residual and the first reference boundary and the second reference boundary, the current residual is mapped to the corresponding confidence score.

5. The dual-modal trajectory anti-spoofing defense method according to claim 1, characterized in that, The historical trajectory status includes the trajectory status and evidence information corresponding to multiple consecutive historical frames; The historical motion reference position corresponding to the current frame is determined based on the historical trajectory state. Historical motion reliability information is determined based on the motion deviation between the original fused trajectory and the historical motion reference position. Historical motion candidates are generated from the multiple candidate trajectories based on the historical motion reference position.

6. The dual-modal trajectory anti-spoofing defense method according to claim 1, characterized in that, The plurality of candidate trajectories includes: Original fusion trajectory candidates generated based on the original fusion trajectory; The first single-mode observation candidate is generated based on the observations from the first sensor; The second single-mode observation candidate is generated based on the observations from the second sensor; Historical motion candidates generated based on the historical trajectory states; Residual recovery candidates are generated by applying a local correction to the original fusion trajectory.

7. The dual-modal trajectory anti-spoofing defense method according to claim 6, characterized in that, The weights corresponding to each candidate trajectory are determined based on the multi-source credible information, the historical trajectory status, and the validity of each candidate trajectory. When the first sensor observation or the second sensor observation is missing or the corresponding confidence level is reduced, the weight of the corresponding single-mode observation candidate is reduced. When the consistency between the first sensor observation and the second sensor observation decreases and / or the consistency between the original fused trajectory and the historical trajectory state decreases, the weight of at least one of the historical motion candidate and the residual recovery candidate is increased.

8. The dual-modal trajectory anti-spoofing defense method according to claim 1, characterized in that, The normal maintenance information includes the normal maintenance probability determined based on the historical trajectory status, the multi-source trusted information, and the status of the multiple candidate trajectories; When the normal maintenance probability meets the preset normal maintenance condition and the trajectory risk represented by the trajectory risk information meets the preset low risk condition, the original fused trajectory is selected as the output trajectory. Otherwise, the recovered trajectory is selected as the output trajectory.

9. A dual-modal trajectory anti-spoofing defense system, characterized in that, include: The acquisition module is used to acquire the original fused trajectory generated based on the observations of the first sensor and the second sensor, as well as the corresponding historical trajectory status. The evaluation module is used to determine the multi-source reliable information and trajectory risk information of the original fused trajectory based on the observations of the first sensor, the observations of the second sensor, the original fused trajectory, and the historical trajectory status. The recovery module is used to generate multiple candidate trajectories based on multiple different information sources, and determine the weight corresponding to each candidate trajectory according to the multi-source credible information, so as to obtain the recovered trajectory; The selection module is used to select either the original fused trajectory or the recovered trajectory as the output trajectory based on the trajectory risk information and the normal maintenance information of the original fused trajectory.

10. An electronic device, characterized in that, It includes a memory, a processor, and program instructions stored in the memory and executable on the processor, wherein the processor, when executing the program instructions, implements the dual-modal trajectory anti-spoofing defense method according to any one of claims 1 to 8.

Images