A multi-source perception redundancy identification method for single-mode failure

Abstract

The application discloses a kind of multi-source perception redundancy identification methods for single mode failure, it is related to multi-source perception data processing and reliability guarantee technical field, including obtaining at least two kinds of heterogeneous sensor source data and carrying out time scale alignment, frame loss compensation and external parameter coordinate unification, candidate redundancy pair is constructed and the observable state quantity set is recorded;The modal health degree is obtained by calculating observation quality component and consistency component based on difference residual, and single mode failure is determined according to health degree continuous anomaly and residual anomaly proportion;Redundancy priority is determined by redundancy score calculation and sorting based on coverage score, stability score and cost score, complete fusion weight smoothing reconstruction and on-demand output degradation and confidence output;Through failure label rule matching and index online evaluation before and after reconstruction update threshold and weight form scene parameter table, to reduce false detection and missed detection and trajectory jitter under complex working conditions, inhibit positioning drift accumulation and improve compensation success rate and continuity.

Description

Technical Field

[0001] This invention relates to the field of multi-source sensing data processing and reliability assurance technology, and in particular to a multi-source sensing redundancy identification method for single-modal failure, applicable to multi-sensor sensing fusion, fault-tolerant reconstruction and output degradation in scenarios such as autonomous driving, mobile robots, drones and intelligent equipment. Background Technology

[0002] Autonomous and advanced driver-assistance systems (ADAS) vehicles commonly employ a combination of sensors, including cameras, millimeter-wave radar, lidar, and inertial devices, working collaboratively for target recognition, environmental modeling, and driving decisions. Mobile robots and drones are also increasingly reliant on multi-source information, such as vision, inertial sensors, wheel speed data, and radar, for localization, mapping, and obstacle avoidance. The direct benefit of multi-source perception is richer information; theoretically, even if one sensor performs poorly, other information can compensate, thereby improving the system's reliability and safety in complex environments.

[0003] However, in practical applications, the perception capability of a single path can suddenly deteriorate due to environmental changes, and this deterioration is often difficult to detect in a timely manner. For example, in autonomous driving scenarios, rain, fog, backlight, and dirt can cause camera images to appear grayish, overexposed, or lose details, leading to false or missed detections in target recognition. If the fusion still "trusts" the camera output based on the original fixed weights, phenomena such as target jumps and trajectory jitter may occur. In mobile robot or drone scenarios, visual odometry is prone to tracking failure or decreased accuracy on weakly textured ground, highly reflective walls, and rapidly changing lighting conditions. Positional errors accumulate over time, affecting mapping quality and obstacle avoidance judgment, and in severe cases, path deviation or local loss of control may occur.

[0004] The key reason for the above problems is that although multi-source systems appear to have redundancy, this redundancy is not inherently usable. On the one hand, different sensors degrade in different ways; some experience increased noise, some data loss, and some output delay or drift. On the other hand, many existing solutions only perform simple fusion or weighted averaging, lacking real-time assessment of the reliability of each data source, and lacking a mechanism to quickly switch to an alternative information source when a particular source deteriorates significantly. As a result, the system superficially continues to output fused results, but in reality, it continues to mix unreliable data together, amplifying errors.

[0005] Therefore, when a single mode fails or degrades significantly under specific conditions, the system lacks a redundant identification mechanism that can be executed in real time to promptly identify which path has a problem, what other information can be used to compensate, and what output granularity and reliability should be used to continue providing sensing results after compensation, thereby avoiding risks such as false detection, missed detection, trajectory jitter, and cumulative positioning drift. Summary of the Invention

[0006] This invention aims to address the problem that multi-source sensing systems continue to output according to a fixed fusion strategy when single-modal sensors fail or degrade under conditions such as rain, fog, backlight, and weak texture reflection, resulting in false detections, missed detections, trajectory jitter, and accumulated positioning drift. It provides a redundancy identification method that can determine single-modal failure in real time, identify alternative redundant links, and complete fusion reconstruction and output degradation, thereby improving the continuous stability and security of sensing and positioning under controllable computing power and bandwidth costs.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A multi-source sensing redundancy identification method for single-mode failure includes the following steps: S1, Data Alignment and Candidate Redundancy Construction. Acquire multi-source sensing data from at least two heterogeneous sensor sources. Add timestamps to the data from each sensor source and unify them to the same time scale. The unification process includes generating an intermediate frame to complete interpolation alignment when the time interval between adjacent frames exceeds the time scale alignment threshold, and using the previous valid frame and combining it with relative motion estimation from other sensor sources to perform time extrapolation to obtain compensated observations when frame loss does not exceed the frame loss compensation upper limit. Unify the observations from each sensor source to the same spatial coordinate system. The unification process includes rotating the observation position vector according to the extrinsic parameter rotation relationship and translating it according to the extrinsic parameter translation relationship. When the observation is at a distance... When the azimuth and distance are given, they are first converted to Cartesian coordinates before rotation and translation, and the validity of the external parameters is verified online based on the registration error of the common-view object. A candidate redundancy set is constructed based on the aligned dataset. The candidate redundancy set includes candidate redundancy pairs for target perception and candidate redundancy pairs for positioning odometry. The candidate redundancy pairs for target perception are obtained by cross-modal object pairing with time and space thresholds, and the candidate redundancy pairs for positioning odometry are obtained by mode combination that can output pose increment or velocity increment. The set of observable state variables of each candidate redundancy pair is recorded.

[0008] S2, Modal health calculation. For each sensor source, the observation quality component and consistency component are calculated to obtain the modal health. The observation quality component includes at least one of image sharpness and exposure saturation, echo effective point ratio and measurement noise estimation, and inertial zero-bias drift estimation. The consistency component is determined by the statistics of cross-modal difference residuals. The difference residuals are calculated following the process of object pairing, unified coordinates, construction of state difference vectors, distance measurement, and residual output. Object pairing is filtered by time difference and spatial distance thresholds. The state difference vectors include at least position difference vectors and may further include velocity, size, and attitude difference vectors. The attitude difference is normalized to the range of -π to π. The distance measurement uses Euclidean residuals or Mahalanobis residuals. Euclidean residuals are the magnitudes of the difference vectors and can be combined by weighted components. Mahalanobis residuals are obtained by taking the square root of the quadratic form of the difference vector and the difference covariance matrix. The difference covariance matrix is ​​obtained by adding the covariances of the two modes or by adding the statistical covariances of the sliding window, and is guaranteed to be invertible by diagonal regularization. The anomaly thresholds are obtained through offline calibration. Offline calibration selects at least three typical working conditions to collect training data and takes the 99th percentile of the difference residual distribution.

[0009] S3, Single-modal failure determination. Failure or degradation is determined based on the temporal continuity of modal health and difference residuals. When the target modal health is below the health threshold and continues for a certain duration, and the proportion of samples with difference residuals exceeding the abnormality threshold reaches the abnormality threshold, the target modality is determined to be failed or degraded.

[0010] S4, Redundancy Identification and Redundancy Scoring. When a failed or degraded mode is determined to exist, alternative information sources are identified from the candidate redundancy set, and a redundancy score is calculated. The redundancy score consists of a coverage score, a stability score, and a cost score. The coverage score is generated item by item based on the set of state variables required for the task and the modal capability table, and then normalized and summed according to importance weights. Directly observable coverage is marked as one, and unobservable coverage is marked as zero. The indirectly derived coverage is the availability coefficient obtained from offline evaluation. The stability score is obtained by aggregating the health of redundant pairs within a sliding window and combining the window mean and normalized variance. The upper bound of the variance is taken as the 99th percentile of offline statistics. The cost score is calculated by the average processing time of newly added processing links and the average number of words... The occupancy ratio and bandwidth occupancy ratio are calculated separately for each segment and weighted to obtain the overall occupancy ratio. The cost score is then obtained by subtracting the overall occupancy ratio from one. The three scores are weighted and summed according to coverage weight, stability weight, and cost weight to obtain the redundancy score. The redundancy priority is obtained by sorting the redundancy scores. When the scores are close, the one with higher coverage is given priority, followed by the one with higher cost. The three weights are determined through grid experiments on at least three types of scenario datasets. The grid experiments evaluate the success compensation rate and system latency. Feasible weights are first screened using the lower limit of the success compensation rate and the upper limit of the system latency. Then, the optimal weight is selected by the comprehensive target value of success compensation rate multiplied by compensation coefficient minus system latency multiplied by latency coefficient, and a scenario weight table is formed.

[0011] S5, Redundant Link Reconstruction and Output Degradation. Based on redundancy priority, redundant links are selected to replace failed or degraded modes in the fusion process. Reconstruction includes resetting the fusion weight of failed or degraded modes to zero or below the minimum guaranteed weight, increasing the fusion weight of redundant links to above the minimum takeover weight, and scaling the weights of the remaining non-failed modes proportionally to make their sum equal. To avoid jitter caused by sudden weight changes, weights are updated at fixed steps within a weight smoothing window, causing the weights of failed modes to decrease monotonically and the weights of redundant links to increase monotonically. When redundant links cannot cover all the required state quantities for output, output degradation is performed. For target perception tasks, the result is downgraded from a 3D bounding box to a 2D occupied grid or risk area; for positioning odometry tasks, the output is downgraded from full pose output to planar displacement increments or heading angle increments, and the confidence level obtained from piecewise linear mapping of the redundancy score is output. Degradation is lifted when the redundancy score continuously meets the recovery conditions and the health of the failed modes recovers to the recovery threshold.

[0012] S6, Failure Label Generation and Self-Learning Update. Failure labels are obtained by rule matching based on the continuous anomaly combination of observation quality components and consistency components. Failure labels include at least occlusion type, low signal-to-noise type, drift type, and time delay misalignment type. When redundant link reconstruction is triggered, the baseline indicators before reconstruction are recorded and an online evaluation window is started. The baseline indicators include at least false positive rate, false negative rate, trajectory jitter amplitude, and positioning drift rate. The false positive rate and false negative rate are obtained by statistically analyzing the near-true values ​​constructed based on multimodal consistency. The trajectory jitter amplitude is obtained by statistically analyzing the deviation of the output trajectory from the smooth trajectory. The positioning drift rate is obtained by dividing the error increment of the positioning output relative to the reference positioning by the time increment. After reconstruction is completed, the compensated indicators are calculated using the same caliber, and the improvement magnitude is obtained. When the improvement magnitude of at least three indicators reaches the improvement threshold, the threshold and weight of the corresponding scenario are recorded as a new parameter set and written into the scenario parameter table for subsequent use in the same scenario.

[0013] Compared with the prior art, the beneficial effects of the present invention are: 1. Time-stamp alignment, frame loss compensation, and unified extrinsic coordinates are used to establish comparable datasets, reduce mismatches and residual false highs caused by cross-modal misalignment, improve the consistency of multi-source data, and reduce false triggering. 2. The differential residuals adopt Euclidean or Mahalanobis metric and are combined with the offline 99th percentile outlier threshold, which can more accurately characterize the degree of inconsistency between modes, identify single-modal degradation earlier and more accurately, and reduce false alarms. 3. The continuous health assessment and abnormality threshold are used to suppress transient noise triggering, avoid frequent switching caused by short-term fluctuations, and improve output stability; 4. The redundancy capability is quantified and ranked based on the scores of coverage, stability, and cost. Redundant links are selected, and the best alternative source is selected first under the control of computing power and bandwidth. 5. Weighted and smooth redundant link reconstruction, combined with on-demand output degradation and output confidence, maintains continuous and reliable output during failure and significantly reduces the risk of trajectory jitter. 6. The online evaluation of indicators before and after failure tag rule matching and reconstruction is used to update thresholds and weights. The system can adapt and take over more quickly under similar working conditions, and the compensation success rate continues to improve. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the overall process of the multi-source sensing redundancy identification method for single-modal failure according to the present invention. Figure 2 This is a flowchart illustrating the differential residual calculation process and the logic block diagram of Euclidean and Mahalanobis distance metrics of this invention. Figure 3 This is a flowchart illustrating the calculation of coverage score, stability score, cost score, and redundancy priority determination for this invention. Detailed Implementation

[0015] In the specific implementation of this application, the execution order of each step can be adjusted according to the system's real-time requirements, sensor update frequency, and task type. Without changing the overall technical concept of multi-source sensing data alignment, single-modal failure determination, redundancy identification and redundant link reconstruction, output degradation, and threshold and weight self-updating, any equivalent transformations or substitutions made by those skilled in the art regarding the order, parallel relationships, and data interaction methods between the steps should fall within the protection scope of this application.

[0016] General Implementation This implementation is applicable to multi-source sensing systems. When at least one mode fails or degrades, it can identify alternative information sources and complete redundant link reconstruction and output degradation. At the same time, it updates thresholds and weights through online evaluation.

[0017] Step S1: Data Alignment and Candidate Redundancy Construction 1.1 Data Acquisition and Unified Time Standard The system operates cyclically according to a unified sampling period. Each sensor output carries a data acquisition timestamp, which comes from the same system clock or a clock synchronized with a unified time.

[0018] Configuration parameter one: Upper limit of timescale alignment error. Configuration method: Take one-tenth to one-fifth of the sampling period. Configuration basis: Divide the allowed timescale position error upper limit by the maximum speed of the scene target to obtain the allowed time error, and then take the larger of the allowable time error and the 99th percentile of the real-time scheduling jitter statistics as the upper limit of error.

[0019] 1.2 Interpolation to generate intermediate frames When the difference between the timestamps of two adjacent frames from the same sensor source exceeds the upper limit of the time stamp alignment error, an intermediate frame is generated to align to the unified sampling time.

[0020] For target observation, the same object in two adjacent frames is first obtained based on the object identifier or tracking association. Then, the position and velocity are linearly interpolated according to the time ratio. The interpolated velocity is obtained by interpolating the velocity of two adjacent frames according to the same time ratio. The interpolated position is obtained by interpolating the position of two adjacent frames according to the same time ratio.

[0021] For inertial observations, angular velocity and acceleration are discretely integrated over the interpolation time interval. Attitude is obtained by integrating angular velocity, velocity by integrating acceleration, and position by integrating velocity.

[0022] 1.3 Frame Drop Compensation A frame is considered lost when a sensor does not output data in the current sampling period.

[0023] Configuration parameter two: Short-term frame drop compensation upper limit. Configuration method: Use two to five sampling periods. Configuration basis: Analyze the growth curve of extrapolation error as the duration of frame drop based on offline playback statistics, and take the duration corresponding to the inflection point where the error begins to increase significantly.

[0024] When frame loss does not exceed the short-term frame loss compensation limit, time extrapolation is performed using the observation from the previous valid frame and the relative motion estimate output from at least one other sensor source. The time extrapolation process involves transforming the position of the previous valid frame observation to the current time using the pose increment obtained from the relative motion estimate, and updating the velocity using acceleration or velocity increments to obtain the compensated observation. When frame loss exceeds the short-term frame loss compensation limit, the sensor source is marked as unavailable and prohibited from subsequent fusion.

[0025] 1.4 Unified Coordinate System Transformation The extrinsic parameters of rotation and translation of each sensor source relative to a unified coordinate system are obtained in advance. The extrinsic parameters are obtained through offline calibration. The coordinate transformation is performed by first rotating and then translating.

[0026] When the input observation is in polar coordinates, the distance and azimuth are first converted into Cartesian coordinates in the sensor source coordinate system, and then rotation and translation are performed.

[0027] Configuration parameter three: Extrinsic parameter verification error threshold. Configuration method: Take the 99th percentile of the registration residual during offline calibration, and add a safety margin. Configuration basis: Ensure that updates are not frequently triggered under normal operating conditions, and that the system can be disabled or updated promptly when the extrinsic parameter drifts significantly.

[0028] Online verification method: Select common viewing objects, register the positions of objects after different modes are transformed to a unified coordinate system, and calculate the registration error as the modulus of the difference between the two positions; when the mean or 90th percentile of the registration error within the sliding window exceeds the external parameter verification error threshold, trigger the external parameter update process or disable the coordinate transformation result of that mode.

[0029] 1.5 Constructing a Candidate Redundancy Set The alignment data is divided into a target-aware subset and a positioning odometry subset according to the task. Cross-modal object association and pairing are performed in the target-aware subset.

[0030] Configuration parameter four: Pairing time threshold. Configuration method: Take two to three times the upper limit of the time scale alignment error. Configuration basis: Consider a conservative time window resulting from the superposition of interpolation error and timing error.

[0031] Configuration parameter five: Pairing space threshold. Configuration method: Take the 99th percentile of the target positioning error plus a safety margin. Configuration basis: Offline playback statistics of the target positioning error distribution for each mode to ensure that the same target is not misassigned.

[0032] The association pairing process involves first filtering candidate pairs whose time difference does not exceed the pairing time threshold, and then filtering candidate pairs whose positional distance under the same coordinate system does not exceed the pairing space threshold. The pairing results satisfying both criteria are considered to be the same object.

[0033] All modal pairs that can form stable pairings are denoted as target perception candidate redundant pairs.

[0034] In the odometry subset, the mode combinations that can output pose increments or velocity increments are denoted as odometry candidate redundancy pairs.

[0035] For each candidate redundancy pair, its set of observable state variables is recorded. This set is obtained from the modal capability table, which is defined by the sensor output field or the drive interface.

[0036] II. Step S2 Modal Health Calculation and Index Calculation In step S2, some indicators are calculated from the measured quantities. The following provides the feasible calculation process and supplements the statistical scope.

[0037] 2.1 Sharpness Indicators It is calculated from the grayscale of image pixels. For each pixel, the absolute value of the horizontal grayscale difference and the absolute value of the vertical grayscale difference are calculated, and the two are added together to obtain the edge response intensity. The mean of the edge response intensity is calculated for the whole frame or block area. Then, the deviation of the edge response intensity from the mean is calculated. The deviation is obtained by taking the square root of the average of the squares of the deviation. This deviation is used as a sharpness index.

[0038] 2.2 Exposure Saturation Index Calculated from image pixel grayscale. The highlight saturation range is defined as the range where grayscale values ​​are greater than or equal to the maximum grayscale value minus a threshold, and the low-brightness saturation range is defined as the range where grayscale values ​​are less than or equal to the minimum grayscale value plus a threshold. The threshold is a configurable parameter.

[0039] Configuration parameter six, saturation range threshold. Configuration method: Set to 0.5% to 2% of the maximum grayscale range based on the image quantization bit depth.

[0040] Configuration basis: The proportion of highlights and shadows at the tail of normal exposure frames is statistically analyzed offline, and a threshold is set to make the proportion of the tail of normal frames lower than the target upper limit.

[0041] The percentage of pixels falling into the high-brightness saturation range and the percentage of pixels falling into the low-brightness saturation range are statistically analyzed, and the sum or one of these percentages is used as the exposure saturation index.

[0042] 2.3 Echo effective point ratio index Calculations are made based on the distance, intensity, and validity indicators of radar or laser echo points. An effective distance range and intensity threshold are set. Points are checked one by one to determine if the distance is within the range, the intensity is not less than the threshold, and the validity indicator is valid. If these conditions are met, the point is counted as a valid point. The percentage of valid points is obtained by dividing the number of valid points by the total number of points.

[0043] Configuration parameter seven: Intensity threshold and effective distance range. Configuration method: Use sensor calibration parameters or factory default parameters as initial values, and fine-tune them during offline playback to minimize false detection and false negative rates. Configuration basis: Ensure sufficient effective points are retained in low signal-to-noise environments while eliminating obvious noise points.

[0044] 2.4 Measurement Noise Estimation Index Calculated from a sequence of measurements of the same object or the same grid cell within the sliding window.

[0045] Configuration parameter eight: Noise estimation sliding window length. Configuration method: Use a sampling number of 0.5 to 2 seconds. Configuration basis: The window needs to cover multiple echo cycles to statistically analyze fluctuations, but it should not be too long to avoid masking sudden degradation.

[0046] The calculation process involves averaging the measurements within the window to obtain the window mean, subtracting the window mean from each measurement to obtain the deviation. The deviation is a scalar with its absolute value and a vector with its magnitude. The deviation is then averaged or its root mean square within the window, and the result is used as the measurement noise estimation index.

[0047] 2.5 Zero-bias drift estimation index It is obtained from inertial angular velocity, acceleration, and the determination of being at rest.

[0048] Configuration parameter nine: Length of the stillness detection window. Configuration method: Use a sampling period of one to three seconds. Configuration basis: Sufficient time is needed to resist instantaneous jitter; the window length with the lowest false alarm rate for offline statistical stillness segment detection is required.

[0049] Configuration parameter ten: stationary speed threshold and stationary angular velocity threshold. Configuration method: The speed threshold is taken as the 99th percentile of the wheel speed or odometer noise, and the angular velocity threshold is taken as the 99th percentile of the inertial angular velocity noise. Configuration basis: To ensure that the stationary judgment is valid when the object is actually stationary, and to avoid misjudgment during low-speed movement.

[0050] A stationary state is determined when the velocity amplitude and angular velocity amplitude are both consistently not greater than the velocity threshold within the stationary state determination window.

[0051] Within a stationary window, the angular velocity is averaged along the three axes to obtain the zero bias of angular velocity, and the acceleration is averaged along the three axes and the gravity projection is subtracted to obtain the zero bias of acceleration. The magnitude of the zero bias vector between adjacent stationary windows is used as the drift amount, which serves as the zero bias drift estimation index.

[0052] 2.6 Discrepancy Residuals and Consistency Components The difference residuals cannot be directly measured. They are calculated from the cross-modal pairing results and the unified coordinate state variables. The process is as follows: object pairing, unified coordinates, construction of state difference vector, Euclidean or Mahalanobis distance metric, and residual output.

[0053] Object pairing uses pairing time thresholds and pairing space thresholds for filtering.

[0054] The state difference vector includes the position difference vector, and under certain conditions, it includes the velocity difference vector, size difference vector, and attitude difference vector. The attitude difference is normalized to the range of -π to π for each dimension.

[0055] The Euclidean residual is the magnitude of the difference vector, and multiple components can be weighted and summed to obtain the combined residual.

[0056] Configuration parameter eleven: Residual component weights. Configuration method: Increase the weights of position and size for target perception tasks, and increase the weights of position and velocity for localization tasks. Configuration basis: Offline playback is based on minimizing the combined target of false detection rate, false negative rate, trajectory jitter amplitude, and localization drift rate.

[0057] The Mahalanobis residuals are calculated by transposing the difference vector, multiplying it by the inverse of the covariance matrix, multiplying it by the difference vector again, and then taking the square root. The covariance matrix is ​​obtained by adding the covariances of the two modal uncertainties, or by summing the covariances after statistical analysis using a sliding window. When the covariance is not invertible, it is inverted after diagonal regularization.

[0058] Configuration parameter 12: Covariance diagonal regularization. Configuration method: Take one-thousandth to one-hundredth of the mean of the covariance diagonal. Configuration basis: Offline playback ensures numerical stability and minimizes the false alarm rate.

[0059] The distance type selection rule is to prioritize Mahalanobis residuals when the distance is available or can be stably statistically uncertain, otherwise use Euclidean residuals.

[0060] During offline calibration, at least three typical working conditions should be selected to collect training data, and the 99th percentile should be used as the anomaly threshold.

[0061] The consistency component calculates the window mean and dispersion of the differential residual sequence within the consistency sliding window. The dispersion is the root mean square of the residual's deviation from the mean. The consistency component is a weighted combination of the window mean and dispersion.

[0062] Configuration parameter thirteen: Consistency sliding window length and consistency combination weight. Configuration method: The window length is set to 0.5 seconds to 2 seconds, and the consistency combination weight is set to a value that ensures the false positive rate and false negative rate of failure determination meet the target constraints. Configuration basis: Offline playback is used to statistically analyze the residual distribution under different degradation types, and the combination that can distinguish between normal and degraded conditions is selected.

[0063] III. Step S3 Single-mode failure or degradation determination The system calculates modal health for each mode, which is determined by both the observed quality component and the consistency component. Failure or degradation is determined when the health is consistently below a threshold and the consistency component is abnormal and persistent.

[0064] Configuration parameter fourteen, health threshold. Configuration method: Offline statistics are used to analyze the distribution of health status under normal operating conditions, and the 1% quantile of the normal distribution is taken as the lower limit threshold. Configuration basis: Ensure that the false trigger rate is lower than the target upper limit under normal operating conditions.

[0065] Configuration parameter 15: Continuous judgment time length. Configuration method: 0.2 to 1 second. Configuration basis: Offline playback comparison of the duration distribution of instantaneous noise and actual degradation; selection of the length that can filter out instantaneous fluctuations.

[0066] Configuration parameter sixteen: Consistency anomaly threshold and anomaly percentage threshold. Configuration method: The consistency anomaly threshold is taken as the 99th percentile of the consistency component under normal operating conditions, and the anomaly percentage threshold is taken as 70% to 90%, with an example value of 80% for the anomaly percentage threshold. Configuration basis: A trade-off between false alarm rate and false negative rate, determined through offline playback.

[0067] When the health score is consistently below the health threshold and the duration of the judgment is reached, and the proportion of samples with consistency components exceeding the consistency anomaly threshold reaches the anomaly proportion threshold, the modality is judged to be invalid or degraded.

[0068] IV. Step S4: Redundancy Identification, Redundancy Scoring, and Redundancy Priority 4.1 Coverage Score First, determine the set of state variables required for the task, and then assign importance weights to each state variable.

[0069] Configuration parameter seventeen: Importance weight of state variables. Configuration method: In the positioning odometry task, the weight of pose increment or velocity increment is set to the maximum. In the perception task, the weights for target location and target existence are set to maximum. Configuration basis: Based on the degree of dependence of the task's safety constraints and control loop on state variables.

[0070] Each candidate redundancy pair is assigned a coverage label, which can be directly observed as 1 and cannot be 0, indirectly yielding the availability coefficient. The availability coefficient is obtained through offline evaluation, which involves reconstructing the state variable using candidate redundancy pairs on a labeled dataset and counting the proportion of reconstruction errors that satisfy a threshold.

[0071] Configuration parameter eighteen: Reconstruction error threshold. Configuration method: Set the reconstruction error to between the 95th and 99th percentiles of normal samples. Configuration basis: To ensure the reliability of indirect inference meets the requirements.

[0072] The coverage score is the sum of the products of the coverage marker and the importance weight, divided by the sum of the weights. For positioning odometry tasks, if pose or velocity increments are not available, the coverage score is set to zero.

[0073] 4.2 Stability Score Redundant health sequences are aggregated within a stability sliding window, and the health of redundant pairs is taken as the smaller value of the health of the two modalities or a weighted average.

[0074] Configuration parameter nineteen: Redundancy's health score synthesis method and weighting. Configuration method: Use the smaller value for safety-first scenarios, and a weighted average for balanced scenarios. Basis: A smaller value prevents the weakest link mode from dragging down overall reliability and being ignored.

[0075] Calculate the window mean and window variance. The upper bound of the variance is taken as the 99th percentile of the health variance in offline statistics. The normalized variance is the variance divided by the upper bound and truncated to zero or one. The stability score is the window mean multiplied by one minus the normalized variance.

[0076] 4.3 Cost Score The average processing time for newly added processing links is obtained by recording the difference between the start and end timestamps of processing and averaging the values ​​within a window. The utilization ratio is calculated by dividing the average processing time by the sampling period. The average number of bytes is the sum of the number of bytes per frame within the window divided by the number of frames. The bandwidth utilization ratio is the average number of bytes divided by the number of bytes that can be transmitted per unit period. The overall utilization ratio is the weighted sum of the two utilization ratios. The cost score is one minus the overall utilization ratio. If the overall utilization ratio is greater than one, the cost score is set to zero.

[0077] Configuration parameter 20: Cost-based weighting coefficient. Configuration method: Increase the weight of computational usage for platforms with computing power bottlenecks, and increase the weight of bandwidth usage for platforms with bandwidth bottlenecks. Configuration basis: Based on the resource type with the highest long-term usage in resource monitoring.

[0078] 4.4 Redundancy Scoring and Priority The redundancy score is a weighted sum of the coverage score, stability score, and cost score, weighted by the coverage weight, stability weight, and cost weight respectively, and the sum of the three weights is one.

[0079] Configuration parameter 21, redundancy scoring with three weights. Configuration method and basis: determined through trade-off experiments. The trade-off experiments select at least three types of scenario datasets, including rain / fog / backlight, weak texture / reflective, and normal scenes; construct single-modal failure event segments in each scenario, and statistically analyze the success compensation rate and system latency respectively. Perform grid experiments on the candidate weight set, first filtering out combinations with success compensation rates below the lower limit or system latency above the upper limit, then selecting the optimal combination from the feasible combinations using a comprehensive objective value. The comprehensive objective value is success compensation rate multiplied by the compensation coefficient minus system latency multiplied by the latency coefficient. The compensation coefficient and latency coefficient are configurable parameters, configured based on the business's preference for security and real-time performance. Finally, a scenario weight table is formed and stored.

[0080] Configuration parameter twenty-two: lower limit of success compensation rate and upper limit of system latency. Configuration method: directly given by safety requirements and platform real-time constraints. Configuration basis: the lower limit of success compensation rate is taken as the value that meets the minimum safety redundancy requirements, and the upper limit of system latency is taken as the maximum end-to-end latency acceptable for the control cycle.

[0081] Candidate redundancy pairs are sorted by redundancy score to obtain redundancy priority. When the scores are close, the pair with higher coverage is given priority, and then the pair with higher cost score is given priority.

[0082] Configuration parameter 23: The score is close to the judgment threshold. Configuration method: Take the range of 5% to 10% of the redundant score distribution. Configuration basis: To avoid frequent switching due to minor differences.

[0083] V. Step S5: Redundant Link Reconstruction and Output Degradation 5.1 Redundant Link Reconfiguration For failed or degraded modes, weight suppression is applied, resetting their weights to zero or reducing them to a minimum weight. For redundant links with the highest redundancy priority, weight enhancement is applied, and the weights of the remaining non-failed modes are scaled proportionally to ensure that the sum of their weights is one. The scaling process involves first calculating the total remaining weight, which equals one minus the target weight of the redundant links, then minus the target weight of the failed modes. The remaining weight is then divided by the sum of the original weights of the remaining non-failed modes to obtain the scaling factor. Finally, the scaling factor is multiplied by the original weights of the remaining non-failed modes to obtain the scaled weights.

[0084] To avoid output jitter, a smooth transition is made to the weight changes. The weights are updated in fixed steps within the weight smoothing window, so that the weights of the failure modes monotonically decrease to the target value and the weights of the redundant links monotonically increase to the target value.

[0085] Configuration parameter 24: Failure mode safety margin and minimum takeover margin for redundant links. Configuration method: The safety margin weight is between 0% and 10%, and the takeover margin is no less than 30%. Configuration basis: The safety margin weight is used for rapid recovery in case of misjudgment, and the takeover margin ensures that the redundant links have sufficient impact on the fusion result.

[0086] Configuration parameter 25: Weighted smoothing window length. Configuration method: Use three to ten sampling periods.

[0087] Configuration basis: Offline playback minimizes trajectory jitter and ensures takeover latency does not exceed the upper limit. During reconstruction, the time-scale alignment and coordinate transformation of step S1 are repeatedly performed on the redundant link output, and it is paired with the current fused tracking object according to the pairing time threshold and pairing space threshold. If pairing cannot be achieved within the continuous decision window, the redundant link is marked as unavailable and the next priority is switched.

[0088] Configuration parameter 26: Length of the redundancy link unavailability determination window. Configuration method: Use two to five sampling periods. Configuration basis: To avoid misjudging unavailability based on a single pairing failure, while ensuring timely handover.

[0089] 5.2 Output Degradation First, determine whether the set of observable states for enabling redundant links covers the set of states required for the current output. If it does, maintain the original output granularity.

[0090] Degradation if not covered: In target perception tasks, 3D bounding box-level target results are downgraded to 2D occupancy grids or risk regions. The occupancy grid generation process involves projecting the target position in a unified coordinate system onto the ground plane, then discretizing it into grid cells according to grid resolution and assigning each cell to an occupied area. The risk region generation process involves dilating the occupancy grid, with the dilution radius being the risk dilution radius.

[0091] In the positioning odometry task, when it is not possible to output both pose increment and attitude simultaneously, the output is downgraded to planar displacement increment or heading angle increment, and the type of missing state quantity is marked.

[0092] The confidence score is obtained by mapping the redundancy score, which is piecewise linear: the confidence score is zero when the redundancy score is not greater than the first threshold, one when it is not less than the second threshold, and calculated proportionally when it is between the two thresholds.

[0093] Configuration parameter 27: Grid resolution and risk expansion radius. Configuration method: Grid resolution is set to 10 cm to 50 cm, and the risk expansion radius is the spatial length corresponding to the safety buffer distance or braking distance. Configuration basis: Grid resolution is determined by the minimum obstacle avoidance resolution and computing power constraints, while the risk expansion radius is determined by the safety strategy.

[0094] Configuration parameter 28 maps confidence scores to a first threshold and a second threshold. Configuration method: The redundancy score distribution of offline statistical compensation failure events is used, with the 95th percentile as the first threshold; the redundancy score distribution of offline statistical compensation success events is used, with the 10th to 20th percentiles as the second threshold, ensuring the first threshold is less than the second threshold. Configuration rationale: To widen the confidence interval and avoid frequent fluctuations in the critical score that could lead to unstable confidence scores.

[0095] Once a downgrade is triggered, if the redundancy score remains at or above the second threshold and the health of the failure mode remains at or above the recovery threshold within the recovery determination window, the downgrade will be lifted.

[0096] Configuration parameter 29: Recovery decision window length and recovery threshold. Configuration method: The window length is three to ten sampling periods, and the recovery threshold is a fraction of the normal health distribution. Configuration basis: To avoid frequent upgrades and downgrades of boundary states.

[0097] VI. Step S6: Generation of Failure Tags and Update of Threshold Weights 6.1 Expiration Tag Rule Matching A sliding window sequence is formed for the observation quality component and the consistency component. The proportion of samples exceeding its threshold for each component is counted. If the proportion reaches the threshold for the proportion of persistently anomalous samples, that component is considered persistently anomalous. Label matching is based on anomaly combination. The image shows persistent abnormalities in sharpness, exposure saturation, and consistency components, classifying it as low visibility. The image shows persistent abnormal sharpness, inconsistent abnormal exposure saturation, and persistent abnormal consistency components, indicating an occlusion type. The echo valid point ratio is continuously abnormal, the measurement noise is continuously abnormal, and the consistency component is continuously abnormal, so it is judged to be a low signal-to-noise type. The measurement noise is continuously abnormal, the proportion of valid echo points is not continuously abnormal, and the consistency component is continuously abnormal, which is judged to be jitter noise type. If the zero-bias drift is consistently abnormal and the consistency component with respect to the positioning output is consistently abnormal, it is determined to be a drift type. If the consistency component is consistently abnormal and the difference residuals show a continuous increasing trend within the window, while the observation quality component is not consistently abnormal, it is judged as a time delay misalignment type. The continuous increasing trend is determined when the proportion of positive differences between adjacent residuals reaches the threshold for the proportion of residual increase.

[0098] When multiple rules are met simultaneously, the drift type takes precedence over the delay misalignment type, the delay misalignment type takes precedence over the low signal-to-noise type, the low signal-to-noise type takes precedence over the occlusion type, and the occlusion type takes precedence over the low visibility type output.

[0099] Configuration parameter 30: Threshold for the percentage of persistent anomalies and threshold for the percentage of residual increase. Configuration method: Set to 70% to 90%. Configuration basis: Offline playback ensures that the false positive rate is below the target upper limit and the false negative rate is acceptable.

[0100] 6.2 Online assessment and parameter updates When redundant link reconfiguration is triggered, the baseline metrics before reconfiguration within the evaluation window are recorded, and the compensated metrics are statistically analyzed using the same caliber after reconfiguration. The evaluation window length is the same as the online evaluation window length.

[0101] Configuration parameter 31: Online evaluation window length. Configuration method: Sample size over one to three seconds. Configuration basis: To ensure stable statistical indicators and timely updates.

[0102] Sources of online accuracy values ​​for false positive and false negative rates Online evaluation does not rely on human-generated true values; it uses multimodal consistency to construct near-true values.

[0103] Configuration parameter 32: Threshold for the number of modes with consistent quasi-true values. Configuration method: Set to more than half of the available modes. Configuration basis: Ensure reliable quasi-true values ​​and reduce the impact of single-mode errors.

[0104] When the same target is confirmed to exist in modalities other than the evaluated modality by at least a threshold of the number of quasi-true value consistent modalities within the pairing time threshold and the pairing space threshold, the target is recorded as having quasi-true value existence. Outputting a target but not confirmed by quasi-true values ​​is counted as a false detection, and outputting a target confirmed by quasi-true values ​​but not output is counted as a false negative. Dividing these values ​​by the corresponding totals yields the false detection rate and the false negative rate, respectively.

[0105] Statistical definition of trajectory jitter amplitude For the same target, the position sequence output is used to calculate a smooth trajectory, which is a moving average position sequence. The jitter deviation at each time step is the magnitude of the difference between the output position and the smooth position. The mean or root mean square of the jitter deviation within the evaluation window is taken as the trajectory jitter amplitude.

[0106] Configuration parameter 33: Length of the sliding average window for the smooth trajectory. Configuration method: Three to ten sampling periods. Configuration basis: Offline playback ensures the smooth trajectory does not lag excessively and filters out high-frequency jitter.

[0107] Reference positioning source for positioning drift rate Reference positioning is selected based on the scenario, prioritizing a high-reliability source: for outdoor autonomous driving, satellite positioning combined with inertial positioning or laser positioning is preferred; for indoor robots, laser mapping positioning is preferred. When the above references are unavailable, the multimodal positioning combination output with the highest redundancy score is used as the reference. The positioning drift rate is the increment of the displacement error of the positioning output relative to the reference positioning divided by the time increment.

[0108] Improvement Magnitude and Update Conditions of Indicators The improvement margin is calculated by subtracting the reconstructed margin from the pre-reconstruction margin. The improvement margin is calculated separately for the false positive rate, false negative rate, trajectory jitter amplitude, and positioning drift rate. The improvement threshold is determined by statistically analyzing the distribution of improvement margins through historical failure event replays, taking the median or a higher quantile.

[0109] Configuration parameter 34: Indicator improvement threshold. Configuration method: Take the median to the 75th percentile of the improvement magnitude distribution. Configuration basis: Update only when there is a clear effect to avoid parameter drift.

[0110] When the improvement of at least three indicators is not less than the corresponding improvement threshold, the threshold and weight under the current scenario category are recorded as a new parameter set and written into the scenario parameter table. This parameter set will be called first when the same scenario category or the same failure label is identified in the future.

[0111] Example 1 Autonomous driving or advanced driver assistance scenarios The system is configured with a forward-facing camera, millimeter-wave radar, and inertial devices, with optional lidar. The unified coordinate system is the vehicle's own coordinate system.

[0112] Normal driving initialization Complete the offline calibration of extrinsic parameters and write them into the extrinsic parameter settings. Set parameters such as the upper limit of timescale alignment error, pairing time threshold, pairing space threshold, stability window length, and consistency window length. The parameters are based on the corresponding configuration criteria in this embodiment. Establish a modal capability table and a candidate redundancy pair set. The target perception candidate redundancy pairs should include at least camera and millimeter-wave radar, camera and lidar, and millimeter-wave radar and lidar.

[0113] Rain, fog, and backlighting cause camera degradation The camera's clarity index decreases while its exposure saturation index increases, and the residual difference between the camera and the millimeter-wave radar at the target position continues to increase, with the consistency component exceeding the consistency anomaly threshold and persisting. Follow step S3 to determine if the camera is faulty or degraded. Follow step S6 to match it as a low-visibility type.

[0114] Redundancy scoring and reconstruction The set of state variables required for the target perception task is at least the target location and the target existence, and the importance weight is the highest among the target location and the target existence.

[0115] Millimeter-wave radar directly observes the target distance and radial velocity. After conversion to the vehicle coordinate system, the target position can be obtained, and the coverage score is high. Its health score in rain and fog scenarios is calculated by the window mean and variance.

[0116] The cost score is obtained by statistically analyzing the average processing time and average number of bytes of the newly added processing links.

[0117] Redundancy scores are calculated by weighting and summing the three categories of redundancy scores, and then sorted. The link with the highest redundancy priority is selected for takeover. Camera weights are reset to zero or reduced to a minimum weight, while redundant links have their weights increased and transitioned smoothly.

[0118] Output degradation and confidence If the radar link alone cannot provide three-dimensional dimensions and attitude, the system output is downgraded to a two-dimensional occupied grid and risk area. The grid resolution is configured according to the minimum resolution and computing power required for braking control, and the risk expansion radius is configured according to the braking distance and safety buffer. The confidence level is output using a piecewise linear mapping based on redundancy scores.

[0119] Online assessment and parameter updates Within the online evaluation window, the false positive rate and false negative rate are statistically calculated using multimodal consistency to construct near-true values, the jitter amplitude of the smoothed trajectory deviation is used to calculate the positioning drift rate using satellite inertial navigation or laser positioning as a reference. If the improvement of at least three indicators reaches the indicator improvement threshold, the consistency anomaly threshold, health threshold, stability window length, and redundancy score weights for this scenario are written into the rain, fog, and backlight scene parameter table, and will be used preferentially in subsequent similar scenarios.

[0120] Example 2 Mobile robot or drone scenario The system is configured with a camera, inertial devices, and wheel speed and odometer; a 2D LiDAR can be optionally added for indoor use. The unified coordinate system is the robot's body coordinate system.

[0121] Normal cruise initialization Configure upper limits for timescale alignment error, short-term frame loss compensation, pairing time threshold, pairing space threshold, stationary judgment window and threshold, difference residual calculation method, and covariance regularization. Establish candidate redundant pairs for positioning odometry, including at least camera and inertial, inertial and wheel speed, and camera and wheel speed.

[0122] Weak texture or strong reflection can cause visual odometry to degrade. The residual difference between the visual odometry and the inertial wheel speed combination continuously increases within the sliding window, and the consistency component remains abnormal. If the zero-bias drift estimation index also remains abnormal, it matches the drift type; otherwise, it matches the time delay misalignment type. Determine whether the visual odometry is faulty or degraded according to step S3.

[0123] Redundancy scoring and reconstruction The set of state variables required for the localization odometry task must be at least pose increments or velocity increments, which are mandatory. If a candidate redundancy pair does not have the mandatory options, the coverage score is set to zero. The combination of inertia and wheel speed usually satisfies the mandatory options, and the coverage score is non-zero. The stability score is calculated from the mean and variance of the health window of the redundancy pair. The cost score is obtained by statistically analyzing the new processing time and number of bytes. The system selects the link with the highest redundancy priority to take over, reduces the weight of visual odometry and increases the weight of inertial wheel speed, and gradually changes the window length according to the weights.

[0124] Output degradation and confidence When a reliable attitude is lacking, the positioning output is downgraded to a planar displacement increment or a heading angle increment, and the type of missing state variable is labeled. The confidence score is output using a redundancy score mapping.

[0125] Online assessment and parameter updates Indoor positioning is primarily based on laser mapping as a reference for drift rate calculation. If laser mapping is unavailable, the positioning combination with the highest redundancy score is used as a reference. The improvement in false positive rate, false negative rate, trajectory jitter amplitude, and positioning drift rate is statistically analyzed based on the online evaluation window. When update conditions are met, the window length, threshold, and redundancy score (for weak texture and reflective scenes) are weighted and written into the scene parameter table. Subsequent similar areas are then quickly triggered and stably output.

[0126] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. For those skilled in the art, various modifications, combinations or substitutions can be made to the sensor type, threshold range, sliding window length, scoring weight configuration method and output granularity form in the embodiments without departing from the spirit and substance defined by the claims of this application; all equivalent transformations and improvements made based on the technical concept of this application should fall within the protection scope of this application.

Claims

1. A multi-source sensing redundancy identification method for single-mode failure, characterized in that, Includes the following steps: S1, the data alignment and candidate redundancy construction step, is used to acquire multi-source sensing data from at least two heterogeneous sensor sources in each sampling period, and unify the multi-source sensing data to the same time scale and the same spatial coordinate system to generate an aligned dataset for subsequent consistency verification; wherein, unifying to the same time scale includes adding a collection timestamp to each sensor source data and performing interpolation or frame loss compensation to ensure that the alignment error is not greater than the configurable parameter, and unifying to the same spatial coordinate system includes transforming the observations of each sensor source to the vehicle coordinate system or the robot body coordinate system based on the sensor extrinsic parameter matrix; and constructing a candidate redundancy set based on the aligned dataset, wherein the candidate redundancy set includes at least: candidate redundancy pairs for target / obstacle perception and candidate redundancy pairs for localization / odometer; S2, Modal health calculation step, is used to quantify the observation quality and output stability of each sensor source in the current sampling period to obtain the corresponding modal health. The modal health is determined by both the observation quality component and the consistency component. The observation quality component includes at least: the sharpness and exposure saturation indices for image modes, the effective echo point ratio and measurement noise estimation indices for radar or laser modes, and the zero-bias drift estimation indices for inertial modes. The consistency component is determined by the difference residuals between the modal observation and other modal observations in the candidate redundancy set on the same object. The difference residuals are calculated by pairing the position, velocity, size, or attitude estimates of the same object in each mode and calculating their Euclidean or Mahalanobis distance in a unified coordinate system. The threshold used to determine whether the difference residuals are abnormal is obtained through offline calibration. Offline calibration includes collecting training data under at least three typical operating conditions, statistically analyzing the quantiles of the residual distribution, and using the 99% quantile as the abnormal threshold to reduce false alarms during operation. S3, Single-modal failure determination step, used to determine whether a single mode has failed or degraded based on the temporal continuity of the modal health and the difference residual; wherein, the determination rule for failure or degradation is: when the modal health of the target mode is lower than the health threshold and the duration reaches a configurable parameter, and the difference residual between the target mode and at least one other mode exceeds the abnormal threshold and exceeds 80% of the window length within the sliding window, the target mode is determined to be a failed or degraded mode; the sliding window length is a configurable parameter, and its value is determined by replaying and statistically analyzing historical false detections, missed detections, and positioning drift events, used to suppress transient noise triggering while ensuring response speed; S4, the redundancy identification and redundancy scoring step, is used to identify alternative information sources from the candidate redundancy set and generate a redundancy score when a failed or degraded mode is determined to exist. The redundancy score consists of at least a coverage score, a stability score, and a cost score. The coverage score represents the observability of the state quantities required by the alternative mode for the target task. The stability score is determined by the mean and variance of the health of the alternative mode within a sliding window. The cost score is determined by the computational cost and communication bandwidth usage. The redundancy scores are then weighted and summed to obtain a redundancy priority, where each weight is determined through trade-off experiments on success compensation rate and system latency in different scenarios to achieve a balance between stability improvement and cost. S5, the redundant link reconstruction and output degradation steps, are used to select at least one redundant link to replace the failed or degraded mode in the fusion based on the redundancy priority, and generate an output granularity and confidence level that matches the redundant link; wherein, the redundant link reconstruction includes resetting the weight of the failed or degraded mode in the fusion to zero or reducing it to no more than the configurable parameter, and increasing the fusion weight of the mode corresponding to the selected redundant link to no less than the configurable parameter; the output degradation includes: when the replacement mode cannot provide all state quantities, the output is downgraded from the three-dimensional box-level target result to the two-dimensional occupied grid or risk area result, and the output carries the confidence level obtained by the redundancy score mapping; S6, Fault Label Generation and Self-Learning Update Step, is used to generate fault labels and update thresholds and weights after each redundant link reconstruction. The fault labels include at least three types: occlusion type, low signal-to-noise type, and delay-drift type. These fault labels are obtained by rule matching of abnormal patterns in the observation quality component and consistency component. The thresholds and weights are updated by online evaluation of changes in task indicators before and after reconstruction. Task indicators include at least false positive rate, false negative rate, trajectory jitter amplitude, and positioning drift rate. When the improvement in the compensated indicators reaches a preset improvement threshold, the thresholds and weights for the corresponding scenario are recorded as a new parameter set, enabling the system to trigger redundancy identification more quickly and obtain more stable output under similar subsequent operating conditions.

2. The method according to claim 1, characterized in that, Step S1 includes the following hierarchical steps: S11. Acquire multi-source sensing data and add timestamps, and calculate the timestamp difference between two adjacent frames of data from the same sensing source. S12. When the timestamp difference is greater than the configurable parameter, interpolation is performed to generate an intermediate frame. The interpolation includes linear interpolation based on the state variables of the same observed object in two adjacent frames. The state variables include at least position and velocity. When the sensing source is an inertial mode, the interpolation further includes discrete integration of attitude and velocity based on angular velocity and acceleration to generate the inertial state at the intermediate moment. S13. When a sensor source does not output data in the current sampling period, it is determined that a frame loss has occurred. Under the condition that the duration of the frame loss does not exceed the configurable parameter, the observation results of the previous valid frame are used and combined with the relative motion estimation from at least one other sensor source to perform time extrapolation on the observation results to obtain a compensated observation. When the duration of the frame loss exceeds the configurable parameter, the sensor source is marked as unavailable. S14. Pre-acquire the extrinsic parameter matrix of each sensor source relative to the vehicle coordinate system or robot body coordinate system, and perform rigid body transformation on the observation point, target position or target bounding box of each sensor source to complete the coordinate transformation. The rigid body transformation includes at least rotation transformation and translation transformation. The extrinsic parameter matrix is ​​obtained through offline calibration and is verified online based on the registration error of the common-view object during operation. When the registration error exceeds the configurable parameter, the extrinsic parameter is updated or the coordinate transformation result of the sensor source is prohibited. S15. Construct a candidate redundancy set based on the aligned dataset after time-scale alignment and coordinate transformation. The construction includes: dividing the aligned dataset into a target-aware subset and a positioning odometry subset according to the task type; in the target-aware subset, associating the same target in different modalities based on spatial proximity rules and temporal consistency rules, wherein the spatial proximity rule is that the association distance is less than a configurable parameter, and the temporal consistency rule is that the time difference is less than a configurable parameter, and recording the modal pairs that satisfy the association as target-aware candidate redundancy pairs; in the positioning odometry subset, recording the modal combinations that can output pose increments or velocity increments as positioning odometry candidate redundancy pairs; and recording the common observable state quantity set for each candidate redundancy pair.

3. The method according to claim 1, characterized in that, The consistency component mentioned in step S2 is calculated through the following steps: S21. After completing time-scale alignment and coordinate transformation, the estimation results of the same object output by different modes within the same sampling period are associated and paired. The association pairing includes determining the same object based on the time difference not being greater than a configurable parameter and the spatial distance not being greater than a configurable parameter. S22. For each pair of paired objects, extract the state variables in a unified coordinate system based on the estimation results. The state variables include at least position and, when conditions permit, further include at least one of velocity, size, or attitude. S23. Calculate the magnitude of the difference between the state variables in different modes as the difference residual, wherein the magnitude of the position difference is the magnitude of the difference between the position vectors of the two modes, the magnitude of the velocity difference is the magnitude of the difference between the velocity vectors of the two modes, the magnitude of the size difference is the magnitude of the difference between the size vectors of the two modes, and the magnitude of the attitude difference is the magnitude of the difference between the attitude angle vectors of the two modes. S24. Statistically analyze the difference residuals within the sliding window to obtain a consistency component. The consistency component is a combination of the mean and dispersion of the difference residuals within the sliding window. The dispersion is a statistical value of the degree of deviation of the difference residuals from their window mean, used to characterize the degree of consistency between this mode and other modes in terms of temporal continuity.

4. The method according to claim 1, characterized in that, The difference residual is calculated from the measured quantities through the following process: Within the same sampling period, the estimation results of mode A and mode B for the same object are obtained respectively. First, they are filtered according to the time difference not being greater than the configurable parameter, and then they are filtered according to the spatial distance under the unified coordinate system not being greater than the configurable parameter, so as to determine the cross-modal pairing results of the same object. For the pairing results, the state variables of mode A and mode B are transformed to a unified coordinate system. The transformation includes rotating the position vector in the state variable according to the external parameter rotation relationship and then translating it according to the external parameter translation relationship. When the input is distance and azimuth, the distance and azimuth are first converted into Cartesian coordinate position vectors in the corresponding mode coordinate system, and then the rotation and translation are performed. Construct a difference vector for the state variables that are available to both parties. The difference vector includes at least a position difference vector, and further includes at least one of a velocity difference vector, a size difference vector, or an attitude difference vector when conditions are met. Each dimension of the attitude difference vector falls within the range of -π to π after angle normalization. When using Euclidean distance, the square root of the sum of the squared differences of each dimension of the difference vector is taken to obtain the Euclidean residual. When there are multiple difference vectors, the Euclidean residuals are added together in a weighted manner to obtain the combined residual, where the weights are configurable parameters. When using Mahalanobis distance, a single difference vector is selected or multiple difference vectors are concatenated in a fixed order to obtain a joint difference vector. The covariance matrix of the joint difference vector is then obtained or calculated. The covariance matrix of the joint difference vector is the sum of the covariance matrix of the output of mode A and the covariance matrix of the output of mode B, or the sum of the statistical covariance matrix obtained by averaging the outer product of the deviation vectors of the output of mode A relative to its window mean within a sliding window and the outer product of the deviation vectors of the output of mode B relative to its window mean. After inverting the covariance matrix of the joint difference vector to obtain the inverse matrix, the product of the transpose of the joint difference vector, the inverse matrix, and the joint difference vector is calculated to obtain a scalar. The square root of this scalar is then used to obtain the Mahalanobis residual. When the covariance matrix of the joint difference vector is not invertible, a configurable parameter is added to its diagonal before inverting it. The distance type is selected based on whether the covariance matrix of mode A and mode B can be obtained or stably statistically analyzed. When the covariance matrix of mode A and mode B can be obtained or stably statistically analyzed, Mahalanobis residuals are used; otherwise, Euclidean residuals are used.

5. The method according to claim 1, characterized in that, Step S4 includes the following hierarchical steps to obtain coverage score, stability score, and cost score, and to determine redundancy priority based on the redundancy score: S41. Determine the task type and generate the set of state variables required for the task. The task type includes target perception task and positioning odometry task. The set of state variables required for the target perception task includes at least target position and target existence. The set of state variables required for the positioning odometry task includes at least pose increment or velocity increment. S42. Establish a modal capability table and generate an observable state quantity set for each candidate redundancy pair, wherein the modal capability table is defined by the sensor output field or the driver interface, and the observable state quantity set is the intersection or union of the output state quantity sets of each modality in the candidate redundancy pair. S43. Calculate the coverage score. The calculation process includes assigning an importance weight to each state variable in the set of state variables required for the task. The importance weight is a configurable parameter. For each candidate redundancy pair, it is determined whether its observable state variable set contains the state variable. If it contains the state variable, the coverage mark for that state variable is 1; if it does not contain the state variable, the coverage mark is 0. If the state variable can be indirectly derived from the candidate redundancy pair, the coverage mark is an availability coefficient between 0 and 1. The availability coefficient is obtained through offline evaluation. The offline evaluation includes reconstructing the state variable using the candidate redundancy pair on the labeled dataset and calculating the proportion of reconstruction errors that meet the threshold. The proportion is used as the availability coefficient. The coverage mark of each state variable is multiplied by its importance weight, summed, and then divided by the sum of all importance weights to obtain the coverage score. When the task type is a positioning odometry task and the candidate redundancy pair cannot provide pose increments or velocity increments, the coverage score is set to 0. S44. Calculate the stability score. The calculation process includes setting the sliding window length as a configurable parameter and collecting the modal health sequence of candidate redundant pairs within the sliding window; at the same time, synthesizing the modal health of each candidate redundant pair into a redundant pair health score, either by taking the smaller value of each modal health score or by weighted average synthesis, where the weighting coefficient is a configurable parameter; calculating the window mean and window variance of the redundant pair health scores within the sliding window, where the window mean is the sum of the redundant pair health scores within the window divided by the number of samples, and the window variance is the sum of the squares of the differences between the redundant pair health scores and the window mean within the window divided by the number of samples; dividing the window variance by the upper bound of the variance and truncating it to 0 to 1 to obtain the normalized variance, where the upper bound of the variance is the 99th percentile of the health variance obtained offline; and determining the stability score as the window mean multiplied by one minus the normalized variance. S45. Calculate the cost score. The calculation process includes: calculating the average processing time of the candidate redundancy for the newly added processing link. The average processing time is obtained by recording the difference between the start time and end time of the processing within a sliding window and taking the average value; dividing the average processing time by the sampling period to obtain the occupancy ratio and truncating it to 0 to 1; calculating the average data transmission amount of the candidate redundancy within the sliding window. The average data transmission amount is the sum of the number of data bytes in each frame within the window and divided by the number of frames. The bandwidth utilization ratio is obtained by dividing the average data transmission volume by the number of bytes that can be transmitted per unit period corresponding to the available bandwidth and truncating it to 0 to 1; the calculated utilization ratio is added to the bandwidth utilization ratio in a weighted manner to obtain the overall utilization ratio, where the weighting coefficient is a configurable parameter; The cost score is determined as one minus the overall occupancy ratio. When the overall occupancy ratio is greater than 1, the cost score is set to 0. S46. Calculate redundancy scores and determine redundancy priorities. The calculation process includes normalizing the coverage score, stability score, and cost score to 0 to 1, then weighting and summing the coverage score, stability score, and cost score according to coverage weight, stability weight, and cost weight to obtain the redundancy score. The coverage weight, stability weight, and cost weight are configurable parameters, and their sum is 1. Redundancy priorities are obtained by sorting the redundancy scores from largest to smallest. When the difference between the redundancy scores of two candidate redundancies is not greater than the configurable parameter, the candidate with the higher coverage score is selected first; if they are still the same, the candidate with the higher cost score is selected. The coverage weight... Stability weights and cost weights are determined through trade-off experiments. These experiments involve selecting at least three types of scenario datasets and collecting or constructing data segments containing single-modal failure events in each scenario. Success compensation rate and system latency are used as evaluation metrics. A grid experiment is conducted on candidate weight sets that satisfy the conditions that coverage weight, stability weight, and cost weight are all between 0 and 1 and their sum is 1, to obtain the success compensation rate and system latency corresponding to each candidate weight. An upper limit for system latency and a lower limit for success compensation rate are set; these are configurable parameters. After eliminating candidate weights that exceed the upper limit for system latency or fall below the lower limit for success compensation rate, a feasible weight set is obtained. A comprehensive target value is calculated for the feasible weight set. The calculation process of the comprehensive target value is to multiply the success compensation rate by the compensation coefficient and subtract the result of multiplying the system delay by the delay coefficient, where the compensation coefficient and the delay coefficient are configurable parameters. The candidate weight with the largest comprehensive target value is selected as the coverage weight, stability weight and cost weight. The coverage weight, stability weight and cost weight are recorded according to the scenario category to form a scenario weight table for selection at runtime.

6. The method according to claim 1, characterized in that, The redundant link reconstruction in step S5 includes the following hierarchical steps: S51. After determining the existence of a failed or degraded mode in step S3, determine the failed or degraded mode to be removed and its current weight in the fusion, and obtain the redundant link with the highest redundancy priority from step S4. S52. Perform weight suppression on the failed or degraded mode. The weight suppression method is to reset the fusion weight of the failed or degraded mode to zero, or reduce the fusion weight of the failed or degraded mode to no greater than the configurable parameter. S53. Perform weight enhancement on the redundant link. The weight enhancement method is to increase the fusion weight of the corresponding mode of the redundant link to not less than the configurable parameter, and perform proportional scaling on the fusion weight of the remaining non-failed modes so that the sum of the fusion weights of all modes is one. The proportional scaling includes calculating a scaling factor. The calculation process of the scaling factor is to subtract the target fusion weight of the redundant link from one and then subtract the target fusion weight of the failed or degraded mode to obtain the remaining total weight. Then, divide the remaining total weight by the sum of the original fusion weights of the remaining non-failed modes to obtain the scaling factor. Finally, multiply the original fusion weight of each remaining non-failed mode by the scaling factor to obtain the scaled fusion weight. S54. To avoid output jitter caused by sudden weight changes, a smooth transition is performed on the fusion weight of the failed or degraded mode and the fusion weight of the redundant link. The smooth transition method is to update the weights in a fixed step size within a transition window with a configurable number of continuous sampling periods, so that the fusion weight of the failed or degraded mode monotonically decreases from the current value to the target value, and the fusion weight of the redundant link monotonically increases from the current value to the target value. S55. The observation results corresponding to the redundant link are connected to the fusion input. The connection method is to perform the same time-scale alignment and coordinate transformation as in step S1 on the output of the redundant link, and to perform association pairing with the current fusion tracking object. The association pairing satisfies that the time difference is not greater than the configurable parameter and the spatial distance is not greater than the configurable parameter. If the association pairing cannot be completed within the judgment window where the number of consecutive sampling periods is the configurable parameter, the redundant link is marked as unavailable and switched to the redundant link with the next redundancy priority. S56. After completing weight suppression and weight enhancement, output the fusion result and the currently enabled redundant link identifier, so that step S6 can record failure events and reconstruction strategies.

7. The method according to claim 1, characterized in that, The output degradation in step S5 includes the following hierarchical steps: S57. Determine the set of state variables required for the current task output, and based on the set of common observable state variables recorded in step S4 for candidate redundancy pairs, determine whether the enabled redundant link covers all state variables in the set of state variables. S58. When the enabled redundant link covers all state variables in the state variable set, maintain the original output granularity and output the full result. The original output granularity includes the three-dimensional bounding box level target result or the continuous pose estimation result. S59. When the enabled redundant links do not cover all state variables in the state variable set, output degradation is performed. The output degradation methods include: For target perception tasks, the output is downgraded from 3D bounding box-level target results to 2D occupancy grid or risk region results. The generation process of the 2D occupancy grid includes projecting the target position in a unified coordinate system onto the ground plane and discretizing it into the corresponding grid cells according to the grid resolution configurable parameters. The generation process of the risk region includes performing morphological dilation processing on the occupancy grid, and the dilation radius of the morphological dilation processing is a configurable parameter. For positioning odometry tasks, when the enabled redundant link cannot output pose increment and attitude at the same time, the output will be downgraded to at least one of planar displacement increment or heading angle increment, and the missing state variable type will be marked in the output. S60. Generate and output confidence scores, which are obtained by mapping redundancy scores. The mapping process is a piecewise linear mapping. When the redundancy score is not greater than the first threshold, the confidence score is zero. When the redundancy score is not less than the second threshold, the confidence score is one. When the redundancy score is between the first threshold and the second threshold, the confidence score is calculated by subtracting the first threshold from the redundancy score to obtain the difference, and then dividing the difference by the second threshold and subtracting the first threshold to obtain the confidence score. The first threshold and the second threshold are configurable parameters, and the first threshold is less than the second threshold. S61. When output degradation is triggered, within the recovery determination window where the number of consecutive sampling periods is a configurable parameter, if the redundancy score is continuously not less than the second threshold and the modal health of the failed or degraded mode is continuously not less than the recovery threshold, then the output degradation is lifted and the original output granularity is restored, wherein the recovery threshold is a configurable parameter.

8. The method according to claim 1, characterized in that, The failure labels in step S6 are obtained by rule matching of the abnormal patterns of the observation quality component and the consistency component, and the updates of the thresholds and weights are completed by online evaluation of the changes in task indicators before and after redundant link reconstruction. Specifically, it includes the following hierarchical steps: S81. Calculate the observation quality component and the consistency component in each sampling period, and form their time series within a sliding window. The observation quality component includes at least the sharpness index and exposure saturation index of the image mode, the echo effective point ratio index and measurement noise estimation index of the radar or laser mode, and the zero bias drift estimation index of the inertial mode. The consistency component is obtained by combining the window mean and dispersion of the difference residual. S82. Perform anomaly detection on the time series. Anomaly detection includes comparing the observation quality component and the consistency component with their respective thresholds, and counting the percentage of samples exceeding the threshold within the sliding window. When the percentage of samples exceeding the threshold is not lower than the configurable parameter, it is determined that the component has a continuous anomaly. S83. Failure label matching is performed based on the combined patterns of persistent anomalies, and the matching rules include at least the following: When the sharpness index is below the sharpness threshold and the exposure saturation index is above the saturation threshold, and the consistency component is above the consistency threshold, it is matched as a low visibility failure label. When the sharpness index is below the sharpness threshold and the exposure saturation index is not above the saturation threshold, while the consistency component is above the consistency threshold, the match is a masking failure label. When the effective point ratio index of the echo is lower than the effective point ratio threshold and the measurement noise estimation index is higher than the noise threshold, and the consistency component is higher than the consistency threshold, it is matched as a low signal-to-noise type failure label. When the measured noise estimation index is higher than the noise threshold and the echo valid point ratio index is not lower than the valid point ratio threshold, and the consistency component is higher than the consistency threshold, the match is a jitter noise type failure label. When the zero-bias drift estimation index is higher than the drift threshold and the consistency component is higher than the consistency threshold in the difference residual with respect to the positioning output, it is matched as a drift-type failure label. When the consistency component is higher than the consistency threshold and the difference residual shows a monotonically increasing trend within the sliding window, and the observation quality component does not show a continuous anomaly, it is matched as a time delay misalignment type failure label; wherein the determination rule for the monotonically increasing trend is that the proportion of positive differences between adjacent residuals within the sliding window is not less than the configurable parameter; When two or more matching rules are met at the same time, failure tags are output according to priority. The priority is: drift failure tags take priority over time delay misalignment failure tags, time delay misalignment failure tags take priority over low signal-to-noise failure tags, low signal-to-noise failure tags take priority over occlusion failure tags, and occlusion failure tags take priority over low visibility failure tags. S84. When triggering redundant link reconstruction, record the baseline indicators before reconstruction and start the evaluation window. The baseline indicators include at least the false detection rate, the missed detection rate, the trajectory jitter amplitude, and the positioning drift rate. The false detection rate is the proportion of the number of targets output as targets within the evaluation window but not confirmed by other modes in the true value or consistency check to the total number of output targets. The missed detection rate is the proportion of the number of targets confirmed by the true value or most modes within the evaluation window but not output to the total number of confirmed targets. The trajectory jitter amplitude is the statistical value of the deviation of the continuous output positions of the same target relative to its smooth trajectory within the evaluation window. The smooth trajectory is obtained by sliding the average value of the output position sequence of the same target. The deviation amplitude is the magnitude of the difference between the output position and the smooth trajectory position at each moment. The positioning drift rate is obtained by dividing the displacement error increment of the positioning output relative to the reference positioning by the time increment within the evaluation window. The reference positioning comes from a high-confidence mode combination or an offline calibration trajectory. The evaluation window length is a configurable parameter. S85. After completing the redundant link reconstruction, calculate the compensated indicators within the evaluation window using the same statistical caliber, and calculate the improvement magnitude of the indicators. The improvement magnitude of the false detection rate is the false detection rate before reconstruction minus the false detection rate after reconstruction; the improvement magnitude of the missed detection rate is the missed detection rate before reconstruction minus the missed detection rate after reconstruction; the improvement magnitude of the trajectory jitter amplitude is the trajectory jitter amplitude before reconstruction minus the trajectory jitter amplitude after reconstruction; and the improvement magnitude of the positioning drift rate is the positioning drift rate before reconstruction minus the positioning drift rate after reconstruction. S86. Compare the improvement magnitude of the indicator with the improvement threshold to determine whether to update the threshold and weight. The improvement threshold is a configurable parameter. It is determined by statistically analyzing the historical failure events to obtain the distribution of the improvement magnitude of each indicator, and taking the median or higher quantile of the distribution as the improvement threshold. When the improvement of at least three indicators is not less than the corresponding improvement threshold, the compensation is determined to be effective, and the threshold and weight under the current scenario category are recorded as a new parameter set; when the conditions for effective compensation are not met, the current threshold and weight remain unchanged. S87. The new parameter set is associated with the corresponding scene category and stored as a scene parameter table. In subsequent operation, when the same scene category or the same failure label is identified, the threshold and weight in the scene parameter table are called first to speed up the triggering and improve the stability of compensation.

Images