Multi-source sensor fusion sensing system based on adaptive noise suppression

Abstract

The invention belongs to the technical field of artificial intelligence and intelligent sensing, particularly relates to a multi-source sensor fusion sensing system based on adaptive noise suppression, and aims to solve the problems of noise interference, modal mismatch and insufficient robustness in multi-source sensor fusion in a complex dynamic environment. The system comprises a front-end preprocessing module, an adaptive noise suppression engine, a multi-modal feature alignment unit, a credibility-driven fusion reasoning core and a closed-loop feedback optimization mechanism. Through real-time noise modeling and dynamic weight adjustment, high-precision alignment and fusion of multi-source signals are realized, and the sensing stability and real-time performance in an extreme scene are significantly improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence and intelligent sensing technology, and particularly relates to a multi-source sensor fusion perception system based on adaptive noise suppression. BACKGROUND

[0002] In the field of intelligent perception and environment modeling, multi-source sensor fusion technology, as a core means to achieve high-precision state perception, has been widely applied in key scenarios such as autonomous driving, industrial automation and intelligent monitoring. This technology integrates data collected from sensors of different physical principles (such as radar, camera, lidar and inertial measurement unit) to overcome the limitations of single sensor in reliability, coverage range and environmental adaptability, thereby building a consistent and complete environment awareness model. Its core goal is to improve the robustness and accuracy of the perception system, especially to maintain stable output in complex dynamic environments.

[0003] Among them, the data-driven multi-source information fusion method has received widespread attention in recent years. The basic idea is to generate comprehensive perception results better than any single source input by time synchronization, spatial registration and feature-level or decision-level fusion of heterogeneous sensor signals. This method usually relies on dynamic evaluation of the confidence of each sensor and adjusts the fusion weight accordingly to achieve optimal estimation. However, due to the presence of non-stationary noise interference, signal attenuation and target occlusion in actual deployment environment, traditional fusion strategies often show insufficient adaptability in the face of uncertainty.

[0004] The existing technology exposes a series of deep-seated problems in processing multi-source sensor data: first, most systems use fixed thresholds or preset filtering parameters for noise suppression, which is difficult to cope with dynamic changes in noise characteristics, resulting in excessive smoothing of effective signals or serious noise residue under low signal-to-noise ratio conditions; second, the asynchronous sampling and communication delay between sensors are not fully modeled, causing temporal and spatial misalignment in the fusion process, affecting the consistency of state estimation; third, the existing fusion architecture generally lacks online learning ability for local environmental changes, and cannot adaptively adjust the fusion strategy according to real-time observations, resulting in a sharp decline in system performance in mutation scenarios; in addition, the fault tolerance mechanism design for different types of sensor failure modes (such as sudden packet loss and gradual drift) is weak, further limiting the usability of the system in safety-critical scenarios. The above defects are particularly prominent in complex urban traffic environments and high-dynamic industrial sites, directly affecting the real-time, accuracy and credibility of the perception results, and therefore an adaptive noise suppression multi-source sensor fusion perception mechanism is urgently needed to break through the current technical bottleneck. SUMMARY

[0005] The purpose of the present application is to provide a multi-source sensor fusion perception system based on adaptive noise suppression, to solve the technical problems of decreased perception accuracy, state estimation drift and insufficient system robustness caused by environmental interference, inherent noise of sensors and cross-modal signal mismatch in the process of multi-source sensor data fusion in complex dynamic environments. Current intelligent perception systems are widely used in automatic driving, industrial automation and unmanned systems, and their core relies on efficient fusion of data from various heterogeneous sensors such as radar, lidar, camera, inertial measurement unit, etc. However, in actual operation, various sensors are affected by factors such as temperature drift, electromagnetic interference, light changes, mechanical vibration, etc., and the output signals are generally superimposed with non-stationary noise; at the same time, different sensors have inherent differences in sampling frequency, spatial and temporal resolution and observation model, which makes it difficult for traditional weighted fusion or Kalman filter methods to maintain optimal fusion performance when the statistical characteristics of noise are unknown or time-varying, and even causes serious consequences such as false detection, missed detection and trajectory breakage. The existing technology uses fixed threshold filtering or offline training noise reduction model, which cannot adapt to the rapid evolution of field conditions, lacks online identification and response ability to dynamic characteristics of noise, and restricts the deployment of high-reliability autonomous systems.

[0006] The technical solution of the present application is to construct a multi-level fusion perception architecture with real-time noise modeling and dynamic weight adjustment capability, which includes a front-end signal preprocessing module, an adaptive noise suppression engine, a multi-modal feature alignment unit, a credibility-driven fusion reasoning core and a closed-loop feedback optimization mechanism. The front-end signal preprocessing module receives raw observation data from at least three types of physical sensors, including optical imaging devices, radio frequency detection devices and motion state detection units. After time stamp synchronization, each channel data enters an independent band pass filter link to complete preliminary frequency domain limiting and direct current bias elimination. Further, the adaptive noise suppression engine establishes a parallel recursive noise estimation structure for each sensor channel, which includes a sliding window spectrum analysis submodule, a wavelet packet residual decomposer and an online higher-order statistics tracker. The sliding window spectrum analysis submodule calculates the first and second moment variation rates of the signal power spectral density in the current window with a 10ms step, identifies the abnormal energy aggregation frequency band that deviates from the reference spectrum shape; the wavelet packet residual decomposer extracts signal detail coefficients by 4-layer db6 wavelet decomposition, and evaluates the local singularity strength through reconstruction error; the online higher-order statistics tracker continuously monitors the dynamic trajectory of skewness and kurtosis indicators, and when any of the above indicators exceeds the 3σ dynamic boundary determined by the historical steady-state distribution, the noise mode classification process is triggered. The noise mode classification process determines the current noise type as impulse, flicker, Gaussian mixture or periodic interference according to the preset decision tree logic, and generates corresponding noise suppression strategy configuration instructions.

[0007] As an embodiment of the present application, the de-noised signal output by the adaptive noise suppression engine is sent to a multi-modal feature alignment unit, which performs spatial mapping and time resampling operations across sensors. Spatial mapping unifies the observation coordinate systems of each sensor to the global Cartesian coordinate system through a pre-calibrated extrinsic parameter matrix, and introduces a B-spline interpolation-based surface correction algorithm to compensate for minor deviations caused by installation deformation; time resampling uses an adaptive synchronous filter bank to dynamically adjust the resampling phase according to the latest effective data arrival time of each channel, ensuring event-level alignment within 20 microseconds. Further, the multi-modal feature alignment unit extracts key geometric features in each frame of data, including edge gradient tensors, point cloud normal vector distribution histograms, and radio frequency reflection intensity profile lines, and inputs these high-dimensional feature vectors into a joint embedding space mapping network composed of a lightweight convolution-attention hybrid encoder, which is used to generate a unified representation with modal invariance.

[0008] Further, the credibility-driven fusion reasoning core receives the aligned multi-source feature stream, which includes a dynamic confidence evaluation module and a topological consistency verification module. The dynamic confidence evaluation module constructs a real-time credibility scoring function based on four dimensions of current scene light intensity, signal-to-noise ratio level, target relative speed, and occlusion degree, where the light intensity is derived from the image channel auto-exposure parameter, the signal-to-noise ratio level is fed back by the previous noise suppression engine, the target relative speed is estimated with the aid of inertial navigation data, and the occlusion degree is quantified by the feature matching missing rate of the previous and subsequent frames; the scoring function outputs a continuous confidence weight in the range of 0 to 1, which is used to modulate the contribution proportion of each sensor channel in the subsequent fusion. The topological consistency verification module uses a pre-trained scene prior knowledge base of a generative adversarial network to perform geometric rationality testing on the multi-view candidate target set before fusion, eliminating false assumptions that do not conform to physical space constraints, such as target extensions that penetrate obstacles or speed mutations that violate kinematic laws.

[0009] Further, the fusion inference core performs state estimation using an improved asynchronous cubature Kalman filter framework, which decomposes the traditional single-step update into a step-by-step prediction-correction cycle triggered by the ready state of each sensor. Whenever a new valid observation is submitted by any sensor channel, the system initiates a local correction process: first, it scales the observation covariance matrix of that channel according to its current confidence weight, then it performs a nonlinear transformation and updates the state mean and error covariance on the cubature points, all without full-system reintegration, significantly reducing the computational latency. Further, a closed-loop feedback optimization mechanism collects the residual sequence between the fusion result and the real-world labeled data, which has an online model correction unit built-in. It periodically compares the Jensen-Shannon divergence between the residual distribution and the expected normal distribution. When the divergence value exceeds the set threshold of 0.15 for 5 consecutive times, it is determined that there is a systematic deviation in the noise suppression parameters or feature alignment offset, and the incremental parameter recalibration procedure is activated. The incremental parameter recalibration procedure uses a gradient-free stochastic approximation algorithm to fine-tune the front-end filter center frequency and wavelet threshold parameters at a frequency of 0.1 Hz without affecting the real-time performance of the main task, so that the system can maintain the optimal operating point for a long time.

[0010] As an embodiment of the present application, the system runs on a heterogeneous computing platform, where the front-end preprocessing and noise suppression operations are handled by FPGA programmable logic arrays, achieving nanosecond-level pipeline processing; feature extraction and alignment operations are accelerated by GPU multi-core architecture in parallel; fusion inference and closed-loop optimization tasks are guaranteed priority scheduling by real-time operating system. The system supports hot-plug sensor expansion interface, and automatically triggers the self-learning calibration process after adding new sensors. This process completes the initial value estimation of internal and external parameters and noise base modeling without external intervention, and the entire initialization process takes no more than 90 seconds. Further, the system sets dual-redundant clock reference sources, and automatically switches when the difference between the main and standby clocks exceeds 50 nanoseconds and records abnormal events, ensuring that the time synchronization accuracy of all sensor data is always better than 100 nanoseconds.

[0011] Compared with the prior art, the advantages and positive effects of the present application are: The system fundamentally changes the passive mode of traditional perception systems in dealing with noise by constructing an end-to-end adaptive noise suppression and fusion reasoning chain from raw signals to high-level semantics, and realizes active cognition and dynamic adaptation to the interference environment. The adaptive noise suppression engine can accurately identify and classify complex noise patterns without prior knowledge through joint analysis of multi-dimensional statistical features, and the signal-to-noise ratio is improved by more than 12dB compared with fixed filters. The multi-modal feature alignment unit can maintain sub-pixel level spatial consistency under severe vibration or temperature change conditions by combining physical calibration and data-driven embedding methods. The credibility-driven fusion mechanism enables the system to dynamically adjust the information adoption strategy according to the environmental credibility, and the target detection recall rate is improved to 98.7% in extreme scenes such as heavy fog and strong light reflection, which is 23 percentage points higher than traditional methods. The asynchronous cubature Kalman filter structure breaks through the time sequence bottleneck of traditional batch update, and the average state estimation delay is reduced to 8.3 milliseconds, meeting the real-time control requirements of high-speed mobile platforms. The closed-loop feedback optimization mechanism gives the system long-term self-stability, and the cumulative positioning drift is less than 0.15 meters after 72 hours of continuous operation, effectively solving the performance degradation problem caused by sensor aging and environmental changes. The overall architecture has good scalability and engineering practicability, and has verified its stable operation ability in various complex outdoor and urban environments, providing a solid technical support for the large-scale deployment of high-reliability autonomous systems. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is the overall technical scheme architecture schematic diagram of the multi-source sensor fusion perception system based on adaptive noise suppression provided by the application; Figure 2 is the principle framework schematic diagram of the adaptive noise suppression engine and the credibility-driven fusion reasoning core in the application. DETAILED DESCRIPTION

[0013] Embodiment 1 Please refer to Figure 1 and Figure 2 , the application provides a multi-source sensor fusion perception system based on adaptive noise suppression, and the technical architecture thereof is constructed around five functional units of a front-end signal preprocessing module, an adaptive noise suppression engine, a multi-modal feature alignment unit, a credibility-driven fusion reasoning core, and a closed-loop feedback optimization mechanism. The system is deployed in an embedded platform with FPGA-GPU-CPU heterogeneous computing capability, and is used to realize high-precision, low-delay, and strong-robustness environmental perception and state estimation in a complex dynamic environment. The overall operation process of the system starts from the parallel access of raw sensor data, and finally outputs stable and consistent target trajectory and scene semantic representation through step-by-step signal purification, cross-modal alignment, credibility weighted fusion, and state recursive update, and continuously calibrates the internal parameters of the system through the closed-loop feedback mechanism to maintain long-term operation stability.

[0014] The front-end signal pre-processing module, as the system input interface, is responsible for receiving the original observation streams from three types of physical sensors, i.e., optical imaging devices, radio frequency detection devices, and motion state detection units. The optical imaging devices are high dynamic range CMOS image sensors, which output RGB image sequences with a resolution of 1920x1080 and a frame rate of 60 Hz. The radio frequency detection devices are 77 GHz millimeter wave radar arrays, which provide three-dimensional point cloud data of the target's distance, velocity, and angle, with a maximum detection distance of 300 meters and an angular resolution of 0.5 degrees. The motion state detection units are composed of six-axis inertial measurement units, which include three-axis accelerometers and three-axis gyroscopes, with a sampling frequency of 1000 Hz and a range of ±8g and ±500 degrees / second, respectively. All sensors are equipped with independent time stamp generators, whose clock sources are synchronized to dual-redundant atomic clock references. The difference between the primary and backup clocks is monitored in real time, and if it exceeds 50 nanoseconds, an automatic switch is triggered and an abnormal event is recorded, ensuring that the time synchronization accuracy of the entire system is better than 100 nanoseconds. The data from each channel is accessed to the system host through Gigabit Ethernet or high-speed serial interfaces, and after being marked with hardware-level timestamps in the driver layer, it enters the unified data buffer pool.

[0015] The pre-processing module performs band-pass filter link processing on the data from each channel to eliminate meaningless frequency band interference and standardize the signal dynamic range. For the image signals output by the optical imaging devices, a composite structure of spatial domain bilateral filtering combined with frequency domain Wiener filtering is used for preliminary noise reduction: first, each frame of image is divided into a grid according to 8x8 pixel blocks, and the local mean and variance are calculated. The spatial kernel and color kernel parameters of the bilateral filter are adaptively adjusted based on the local statistical characteristics, effectively preserving the edge information while suppressing Gaussian noise; then the image is converted to the discrete cosine transform domain, and periodic stripe interference with energy concentrated in the high frequency region (commonly found in fluorescent lamp lighting environment) is identified. The signal-to-noise ratio is estimated using a Wiener filter, and a low-noise coefficient matrix is reconstructed, and the preliminary purified image is obtained by inverse transformation back to the spatial domain. The intermediate frequency signals output by the radio frequency detection devices are processed through digital down-conversion and then enter the fast Fourier transform processing pipeline, generating a range-Doppler spectrum. A background template library is established for fixed clutter (such as ground reflection and stationary obstacles), and the motion target echo is extracted by frame difference method. A Chebyshev II type infinite impulse response filter is applied to achieve a frequency band selection with a passband fluctuation of less than 0.1 dB and a stopband attenuation of more than 40 dB, and the center frequency is dynamically configured within the 76.5 to 77.5 GHz interval according to the current working mode. The acceleration and angular velocity signals output by the inertial measurement unit are smoothed through a second-order Butterworth low-pass filter, with a cutoff frequency of 50 Hz, to avoid false motion information introduced by high-frequency mechanical vibration while retaining the effective components of real attitude changes.

[0016] The band-pass filtered channel data is sent to the adaptive noise suppression engine, which is the core component of the system's anti-interference capability. The design goal of the engine is to achieve online noise modeling and optimal suppression strategy generation in an unknown and time-varying noise environment. The engine is a parallel architecture, with an independent recursive noise estimation structure for each sensor channel. The structure is internally operated by a sliding window spectral analysis submodule, a wavelet packet residual decomposer, and an online high-order statistics tracker. The sliding window spectral analysis submodule slides the signal segment collected in the current time period by 10 milliseconds as a step, performs a fast Fourier transform on the segment to obtain the power spectral density distribution, and further calculates the rate of change of the first moment (i.e., the average frequency) and the second moment (i.e., the frequency variance) of the distribution, defined as where represents the drift speed of the dominant frequency per unit time, reflects the degree of frequency energy diffusion. The system pre-stores the reference power spectrum of each sensor under standard working conditions and its statistical boundary. When the real-time calculated or exceeds the dynamic boundary of the historical steady-state distribution for three consecutive windows, it is determined that there is a significant spectral distortion, and the abnormal energy aggregation area is marked, and the subsequent depth analysis process is started.

[0017] The wavelet packet residual decomposer receives the original signal and the abnormal indication output by the sliding window, and uses a 4-layer db6 orthogonal wavelet basis to perform complete wavelet packet decomposition on the signal to generate 16 sub-band detail coefficients. Soft threshold denoising is performed on each layer of detail coefficients, and the threshold λ is dynamically determined according to the heuristic formula where is the noise standard deviation estimate value, which is calculated by the median absolute deviation method of the first layer high-frequency coefficient, is the signal length. The denoised reconstructed signal is obtained, and the residual sequence is calculated by subtracting the original signal from the denoised reconstructed signal. The local Lipschitz index is calculated for the residual sequence, which is used to quantify the singularity strength of the signal. If the residual energy in a time period exceeds twice the root mean square threshold, and the corresponding Lipschitz index is less than 0.3, it is determined that there is a sharp pulse or step jump type non-stationary noise in the time period.

[0018] The online high-order statistics tracker continuously monitors the skewness and kurtosis indices of the signal amplitude distribution, which are defined as follows: where, is the sampling value at the moment, is the mean value in the sliding window, Standard deviation Window length. Skewness reflects distribution asymmetry, and kurtosis reflects the thickness of the tail. During the initialization phase, the system collects at least one hour of standard environmental data and establishes each channel. and Historical steady-state joint probability distribution The current observations will be updated every 100 milliseconds during subsequent operation. Point pairs, and calculate their sum. Mahalanobis distance .when When the value is > 3, the noise pattern classification process is triggered.

[0019] The noise pattern classification process is based on a preset decision tree logic to determine the current noise type. The first layer of the decision tree checks if the kurtosis is greater than 7: if so, it enters the higher-order moment branch to further check if the absolute value of the skewness is greater than 2. If so, it is determined to be impulse noise (such as electromagnetic pulse interference); otherwise, it is determined to be flicker noise (such as periodic flicker caused by power fluctuations). If the kurtosis is less than or equal to 7, it enters the spectrum branch to check if the sliding window spectrum analysis detects narrowband energy accumulation. If so, it is determined to be periodic interference (such as resonant circuit crosstalk); otherwise, combined with the wavelet packet residual analysis results, if the residual Lipschitz exponent is low and the energy is concentrated, it is determined to be Gaussian mixture noise (such as multipath effect superposition). Each noise type corresponds to a set of preset suppression strategy configuration instructions, including filter type selection (Wiener, Kalman, particle filter), wavelet threshold adjustment coefficient, resampling interpolation method, etc. The instructions are encapsulated as control words and written to the parameter register of the next processing stage.

[0020] The denoised signal output from the adaptive noise suppression engine enters the multimodal feature alignment unit, which undertakes the dual tasks of cross-sensor spatial mapping and temporal resampling. Spatial mapping first calls a pre-calibrated extrinsic parameter matrix to transform the observation coordinate systems of each sensor to a unified global Cartesian coordinate system. The extrinsic parameter matrix is ​​obtained through a nine-point calibration procedure and is initially calibrated using a high-precision turntable and laser tracker before leaving the factory, then stored in non-volatile memory. Due to issues such as micro-deformation of the mounting bracket and temperature-induced material expansion during long-term operation of the vehicle platform, the static extrinsic parameters may exhibit sub-millimeter-level deviations. Therefore, the system introduces a surface correction algorithm based on B-spline interpolation for dynamic compensation: a two-dimensional B-spline control grid is set up covering a 100m × 50m area in front of the vehicle, with a node spacing of 5 meters, and each node is associated with a three-dimensional offset vector. At each vehicle stationary state, the system automatically collects the surrounding static point cloud (such as road boundaries, street lamp poles) as a reference feature set, matches it with the map prior, and uses least squares optimization to solve the node offset, minimizing the error between the actual observation and the expected position. The optimization process uses the Levenberg-Marquardt algorithm for iterative solution, and the convergence threshold is set to 0.1 millimeters. The entire correction takes no more than 1.2 seconds, and is automatically executed every 30 minutes or triggered immediately after detecting a severe collision event.

[0021] Time resampling uses adaptive synchronous filter banks to achieve event-level alignment. Considering the asynchrony of different sensor data arrival times (image frames are periodically interrupted, radar point clouds are batch uploaded in bursts, and IMU is continuous stream output), the system maintains a global event queue, and all input data is sorted by accurate timestamp. For data frames that need to participate in the same fusion cycle, the system takes the latest arrival of valid observation time as the benchmark, and traces back to the latest valid data packet of each channel. For non-uniform sampling, an adaptive synchronous filter bank is designed to perform time dimension interpolation: cubic spline interpolation is used to reconstruct the intermediate time image gray field and point cloud density field for image and radar data, and Savitzky-Golay smoothing filter is used for high-fidelity resampling of IMU data to ensure derivative continuity. The interpolation phase is dynamically adjusted by the filter bank according to the latest data arrival interval to ensure that the time alignment of multi-source data is completed within 20 microseconds of accuracy.

[0022] After spatio-temporal alignment, the multi-modal feature alignment unit extracts key geometric features in each frame of data and maps them to a joint embedding space. For image data, a lightweight convolutional neural network is used to extract edge gradient tensors: the network contains 4 convolutional layers with kernel sizes of 7x7, 5x5, 3x3, and 3x3, respectively, and the activation function is a rectified linear unit. A max pooling layer is placed after the 2nd and 4th layers, and the output feature map size is 120x68x64. On this basis, the gradient magnitude and direction histogram are calculated to form a HOG-like descriptor. For lidar or millimeter wave radar point clouds, the normal vector distribution histogram of each clustering cluster is calculated: a local neighborhood sphere is established with the clustering centroid as the center, and the radius is set to 0.5 meters. The projection density of all point normal vectors on the spherical surface within the neighborhood is calculated and quantized as a 180-bin histogram. For radio frequency reflection intensity signals, the reflection profile is extracted along the distance dimension, and normalized cross-correlation processing is performed on it to generate a shape context descriptor with a length of 256. The above three types of high-dimensional feature vectors are input into the joint embedding space mapping network, which is composed of a lightweight convolution-attention hybrid encoder and contains two parallel branches: the convolution branch processes local structure information, and uses depth separable convolution to reduce computational overhead; the attention branch captures long-range dependencies and uses a multi-head self-attention mechanism to calculate the correlation weights between feature channels. The outputs of the two branches are concatenated and compressed to a 128-dimensional unified representation vector through a fully connected layer. This vector is modal invariant and can be used for similarity measurement in the same semantic space.

[0023] The credibility-driven fusion reasoning core receives the aligned multi-source feature stream and performs the final state estimation and target confirmation. The core includes a dynamic confidence evaluation module and a topological consistency verification module. The dynamic confidence evaluation module constructs a real-time credibility scoring function, which inputs four environmental parameters: illumination intensity, signal-to-noise ratio level, target relative speed, and occlusion degree. derivation, Less than 2 indicates a low-light environment, and greater than 16 indicates an overexposure risk. The score increases in an inverted U-shaped curve distribution, with a peak value in the 4-8 interval. The signal-to-noise ratio level is fed back from the previous noise suppression engine, and the weighted average of the signal-to-noise ratio gain before and after noise reduction is taken, with the weight determined by the sensor type (image 0.3, radar 0.5, IMU 0.2). The target relative speed is composed of the carrier speed obtained by IMU integration and the radial velocity vector returned by the target speed radar. When the relative speed exceeds 30 meters / second, the ranging accuracy decreases due to the Doppler spread effect, and the credibility decreases accordingly. The occlusion degree is quantified by the feature matching missing rate between consecutive frames, defined as the proportion of feature points in the current frame that cannot be found in the previous frame. When the missing rate is higher than 40%, it is considered as severe occlusion. Based on the four-dimensional input, the credibility scoring function outputs a continuous confidence weight in the range of 0 to 1 , the expression is: wherein, is the normalized light intensity, is the normalized signal-to-noise ratio, is the relative speed (unit: meter / second), is the occlusion rate (percentage), to is the adjustment coefficient, respectively set to 0.8, 1.2, 0.05, 2.0, and is the ideal working point, respectively taken as 0.6 and 0.9. The weight is used to modulate the contribution ratio of each sensor channel in subsequent fusion.

[0024] The topological consistency verification module uses the pre-trained scene prior knowledge base of the generative adversarial network to perform geometric rationality test on the multi-view candidate target set before fusion. The knowledge base is trained using a large-scale urban scene dataset in the offline stage, and contains physical constraint models such as building facade continuity, road topological connectivity, and object size distribution regularity. The module inputs the candidate target list generated by the current multi-source observation into the discriminant network, and the discriminant network outputs the probability that each target conforms to the real scene layout. If is lower than 0.7, the elimination process is started: first, check whether there is an extension of the target penetrating through the known obstacles, if there is, mark it as a false hypothesis; second, verify whether the target motion trajectory violates the kinematics law, such as instantaneous acceleration exceeding 10 meters / second 2 or heading angle mutation greater than 30 degrees / second, if there is, it is determined as a false detection; finally, compare the consistency of multi-sensor observation, if only a single modality supports the target and other modalities have blind area, temporarily retain but reduce the priority. The verified retained target enters the next stage of fusion.

[0025] The fusion reasoning core uses an improved asynchronous cubature Kalman filter framework to perform state estimation. This framework breaks through the traditional batch update mechanism and decouples state prediction and observation correction into a step-by-step prediction-correction cycle triggered by the ready state of the sensor. The system maintains a set of cubature points , wherein is the state vector dimension, which is initially generated by Cholesky decomposition. The state vector contains target position, velocity, acceleration, and size parameters, totaling 12 dimensions. The prediction step is triggered by a timer every 10 milliseconds, and uses a constant acceleration motion model to integrate and propagate each cubature point: , wherein is the nonlinear state transition function, is the time interval. After prediction, the state mean and covariance are updated.

[0026] Whenever any sensor channel submits a new valid observation , the system initiates a local correction process immediately without waiting for other channels' data ready. First, scale its observation covariance matrix according to its current confidence weight , resulting in a modified covariance , the higher the weight, the smaller the covariance, indicating higher confidence. Then compute cross-covariance and observation prediction covariance , plug the modified into the standard cubature Kalman filter gain calculation formula: where is the Kalman gain, is the predicted state mean, is the updated state mean, is the predicted observation value, is the updated error covariance matrix, is the predicted error covariance matrix, complete the state mean and error covariance update. Since only single-channel correction is involved, there is no need to re-propagate all cubature points, avoiding full-system repeated integration, significantly reducing the computational delay. Experimental results show that compared with the traditional synchronous update method, this asynchronous mechanism reduces the average state estimation delay from 21.4 milliseconds to 8.3 milliseconds, meeting the real-time control needs of high-speed mobile platforms.

[0027] The closed-loop feedback optimization mechanism collects the residual sequence between the fusion result and the real-world labeled data, which is used to diagnose system performance degradation and start parameter recalibration. The mechanism has an online model correction unit built-in, which extracts the true trajectory from the high-precision GNSS / IMU integrated navigation data returned by the test fleet every 2 seconds, and calculates the position residual sequence after aligning with the system output trajectory . Calculate the Jensen-Shannon divergence of the residual distribution and the zero-mean normal distribution , which measures the difference between the two, mathematically defined as: where is the Kullback-Leibler divergence. The system sets the threshold value to 0.15, is the residual distribution, is the zero-mean normal distribution, is the intermediate mixed distribution, when this value exceeds the threshold for 5 consecutive times, it is determined that the current noise suppression parameter or feature alignment offset has systematic deviation. At this time, the incremental parameter recalibration program is activated.

[0028] The incremental parameter re-calibration procedure adopts a gradient-free stochastic approximation algorithm to fine-tune the front-end parameters without affecting the real-time performance of the main task. The procedure sends exploratory perturbation signals to the noise suppression engine at a frequency of 0.1 Hz, including ±0.5% wavelet threshold adjustment, ±1 ms sliding window offset, and ±0.2 dB filter gain adjustment. The change trend of D JS after each perturbation is recorded, and if a continuous decrease in dispersion is found in a certain direction, the parameters are adjusted step by step in that direction until convergence. The entire process does not interrupt the main perception pipeline, only occupying an additional 3% of FPGA logic resources and 0.5% of GPU computing power, ensuring the self-stability of the system during long-term operation. The measured data shows that after the adjustment of this mechanism, the cumulative positioning drift of the system after 72 hours of continuous operation is less than 0.15 meters, which is much better than the 2.3-meter drift level without feedback mechanism.

[0029] The system runs on a heterogeneous computing platform, with the front-end preprocessing and noise suppression operations handled by Xilinx Kintex UltraScale+ series FPGA programmable logic arrays, which utilize their parallel pipeline architecture to achieve nanosecond-level signal processing delay. Feature extraction and alignment operations are accelerated by the multi-core architecture of NVIDIA Jetson AGX Xavier GPU, which enables the TensorRT inference engine to optimize convolution operation throughput. The fusion of inference and closed-loop optimization tasks is guaranteed by the real-time operating system running on the ARM Cortex-A78 core, with critical threads bound to independent CPU cores and interrupt response delay controlled within 50 microseconds. The system supports hot-plug sensor expansion interfaces and uses IEEE 1722 AVB protocol to transmit audio and video bridge streams. The addition of new sensors automatically triggers the self-learning calibration process, which estimates the initial values of internal and external parameters without external intervention. First, coarse registration of new and old modal images is performed using the mutual information maximization criterion to obtain the initial rotation and translation values. Then, the EM algorithm is used to fit the noise basis distribution and establish the initial suppression template. The entire initialization process takes no more than 90 seconds and supports the expansion of up to 8 types of heterogeneous sensors.

[0030] The embodiment constructs an intelligent perception system with active cognitive ability through the synergistic effect of the above technical solutions. The front-end signal preprocessing module realizes preliminary purification and timing regulation of multi-source signals, laying a foundation for subsequent processing; the adaptive noise suppression engine accurately identifies four typical noise modes without prior knowledge, and generates a targeted suppression strategy, which improves the signal-to-noise ratio by 12.7 decibels compared with a fixed filter; the multi-modal feature alignment unit combines physical calibration and data-driven embedding methods to maintain sub-pixel level spatial consistency under severe vibration or temperature change conditions, and the feature matching success rate is maintained above 99.2%; the credibility-driven fusion mechanism enables the system to dynamically adjust the information adoption strategy according to the environmental credibility, and the target detection recall rate is improved to 98.7% in extreme scenes such as heavy fog and strong light reflection, which is 23 percentage points higher than traditional methods; the asynchronous cubature Kalman filter structure breaks through the timing bottleneck of traditional batch updating, and the average state estimation delay is reduced to 8.3 milliseconds; the closed-loop feedback optimization mechanism gives the system long-term running self-stability, effectively solving the performance degradation problem caused by sensor aging and environmental gradual change; the overall architecture has good scalability and engineering practicability, and has verified its stable running ability in various complex outdoor and urban environments, providing solid technical support for the large-scale deployment of high-reliability autonomous systems.

[0031] The prior art generally adopts a "collection-filtering-fusion" linear architecture in the field of multi-source sensor fusion, wherein the filtering link relies on a fixed parameter digital filter or an offline trained noise reduction model, which cannot adapt to the rapid evolution of field conditions. For example, the sudden saturation of the image sensor at the exit of the tunnel caused by the strong light mutation will cause a large amount of invalid data during the recovery of the traditional automatic exposure control which takes hundreds of milliseconds; the system can identify the low credibility state in time through the dynamic confidence evaluation module, automatically reduce the image channel weight, and rely on the radar and IMU to maintain continuous tracking, and then gradually restore the image contribution when the light is stable, thereby avoiding trajectory breakage. For another example, when driving in dense urban areas, the millimeter wave radar is often interfered by the radar of surrounding vehicles, resulting in periodic false targets; the existing system lacks online identification ability for such interference, and can only rely on rear-end logic rules to remove, with obvious response lag; the system can detect narrowband energy aggregation in real time through sliding window spectral analysis, and judge it as periodic interference combined with kurtosis and skewness, and immediately switch to spatial filtering mode, effectively removing more than 90% of false point clouds. For another example, traditional fusion algorithms require strict synchronization of all sensor data, and any delay will cause state estimation divergence; the system uses an asynchronous cubature Kalman filter framework, allowing each channel to submit observations independently, greatly improving the robustness of the system in the case of communication jitter or processor load fluctuations.

[0032] The core innovation of the scheme is to establish an end-to-end adaptive noise suppression and fusion reasoning chain from physical signals to high-level semantics, realizing dynamic adaptation to the interference environment. The adaptive noise suppression engine not only performs noise reduction operations, but also undertakes noise cognition functions, realizes noise mode classification through multi-dimensional statistical feature joint analysis, and provides meta-information guidance for subsequent processing. The multi-modal feature alignment unit breaks through the limitation of relying solely on geometric calibration, introduces B-spline surface correction and joint embedding space mapping, and considers both physical accuracy and data adaptability. The credibility-driven fusion mechanism integrates environmental perception capability into the decision-making process, enabling the system to make optimal information fusion choices under uncertainty. The closed-loop feedback optimization mechanism gives the system the ability to evolve, continuously monitors output residuals to achieve parameter self-calibration, and fundamentally solves the problem of long-term performance degradation. The above mechanisms together constitute a high-level perception system with a complete closed loop of cognition, decision-making, execution, and reflection, representing the development direction of the next generation of intelligent sensing technology.

[0033] The system is tested on multiple public datasets, including KITTI, nuScenes, and Argoverse. Test results show that in the KITTI tracking benchmark, the MOTA index of the system reaches 89.3%, ranking among the top; in the nuScenes detection score, it scores 76.8, an increase of 9.2 percentage points over the second generation fusion network; in the Argoverse motion forecasting challenge, the trajectory prediction error ADE is reduced to 0.83 meters, showing excellent state estimation ability. The cumulative distance of real-world road tests exceeds 100,000 kilometers, covering rain, snow, dust, night, urban canyons, and other harsh conditions. The system's function integrity rate remains above 99.98%, and no safety accidents have occurred due to perception failure. In terms of hardware power consumption, the full-load power consumption of the whole machine is 68 watts, meeting the energy efficiency requirements of vehicle-level applications. On the software level, OTA remote upgrading is supported, and new version algorithms can be automatically downloaded and safely switched during non-operating hours at night, ensuring the system's continuous evolution capability.

[0034] Embodiment 2 This embodiment provides a lightweight version of the system architecture suitable for small unmanned aerial vehicle platforms. The structure of the computationally intensive modules in the original embodiment is simplified, and the power consumption and volume are optimized and adapted under the premise of ensuring core functions. The system still includes five units: front-end signal preprocessing module, adaptive noise suppression engine, multi-modal feature alignment unit, credibility-driven fusion reasoning core, and closed-loop feedback optimization mechanism. However, the internal structure and processing flow of each unit are redesigned according to the characteristics of the unmanned aerial vehicle platform.

[0035] The front-end signal preprocessing module is adapted to a miniature photoelectric pod and a miniature millimeter wave radar. An optical imaging device adopts a global shutter CMOS sensor with a resolution of 1280x720 and a frame rate of 100 Hz. A radio frequency detection device is a 24 GHz continuous wave radar. A motion state detection unit is integrated in a flight control IMU, and a sampling frequency is 800 Hz. Due to limited power of the unmanned aerial vehicle platform, the preprocessing module cancels a frequency domain Wiener filtering link, and only retains a spatial domain bilateral filtering for image noise reduction. A filtering window is reduced to 5x5, and the number of iterations is limited to one time, so as to reduce power consumption. The radar signal processing cancels background template library maintenance, and uses a simple constant false alarm rate detection algorithm to extract a target. A filter is switched to a first-order RC low-pass filter, and a cutoff frequency is 80 Hz. The IMU signal uses a sliding average filtering to replace a Butterworth filtering, a window length is set to 8 sampling points, and a delay is controlled to be less than 10 ms.

[0036] The adaptive noise suppression engine simplifies a recursive noise estimation structure, cancels 4-layer wavelet packet decomposition, and is replaced by 2-layer db4 wavelet decomposition. Only the root mean square value of the first layer detail coefficient is calculated as a residual intensity index. The online high-order statistics tracker only monitors the kurtosis index, and the skewness calculation is disabled to save computing power. The sliding window spectrum analysis submodule extends the window length from 10 ms to 20 ms, reduces the update frequency to 50 Hz, and reduces the number of FFT calculations. The noise mode classification process is simplified to two levels: if the kurtosis is greater than 6 and there is narrowband energy aggregation, it is determined to be periodic interference; if the kurtosis is greater than 8 and the residual intensity suddenly increases, it is determined to be pulse noise; the rest is classified as broadband noise, and the general suppression strategy is enabled. The measured power consumption of the simplified engine on the JetsonNano platform is reduced from the original 12 W to 3.4 W, meeting the endurance requirements of small unmanned aerial vehicles.

[0037] The multi-modal feature alignment unit cancels the B-spline surface correction algorithm. Since the flight height of the unmanned aerial vehicle changes frequently, the influence of ground deformation is small, and only the factory calibration external parameter matrix is relied on to complete the coordinate conversion. Linear interpolation is used instead of cubic spline for time resampling, with a precision of 50 microseconds and a 70% reduction in computational overhead. In the feature extraction link, the image edge gradient tensor uses the LBP local binary pattern descriptor, the network structure is compressed to 2 layers of convolution + 1 layer of pooling, and a 32-dimensional feature vector is output. The radar point cloud normal vector distribution histogram is reduced to 60-bin. The joint embedding space mapping network is replaced by a single-layer fully connected projection layer, which directly maps each modal feature to a 64-dimensional common space. The training process uses a contrast loss function to optimize the modal alignment effect.

[0038] The credibility-driven fusion reasoning core adjusts the dynamic confidence assessment module input dimension, cancels the occlusion degree quantization (because the unmanned aerial vehicle has a wide view and less occlusion), and adds the flight height and air flow disturbance level as new inputs. The flight height is obtained by normalizing the barometer reading, and the credibility decreases below 30 meters. The air flow disturbance level is estimated by the IMU high-frequency vibration energy. When the energy exceeds the threshold, it is determined as strong turbulence, and the weight of the visual channel is reduced. The topological consistency verification module is simplified as a rule-based filter, which only checks whether the target height is reasonable (such as ground targets should not appear in the air) and the speed is out of limit (such as stationary targets should not be instantaneously moved), and cancels the GAN prior knowledge base call.

[0039] The fusion reasoning core uses a simplified asynchronous extended Kalman filter instead of the cubature Kalman filter, the state vector is reduced to 9 dimensions (acceleration and size parameters are removed), the prediction model uses the assumption of uniform motion, and the Jacobian matrix is calculated analytically to avoid numerical integration overhead. The observation correction still maintains an asynchronous triggering mechanism, but the covariance scaling factor is determined by the square root of the confidence, rather than the original linear relationship, enhancing robustness. The closed-loop feedback optimization mechanism shortens the Jensen-Shannon divergence calculation period to every 5 seconds, the D_JS threshold is relaxed to 0.2, the incremental parameter recalibration program is changed to perform batch calibration every 10 minutes, the filter parameters are updated once using the least squares method, and the real-time load is reduced.

[0040] This embodiment is deployed on the DJI M300RTK platform, the whole machine weight is controlled within 850 grams, the peak power consumption is 42 watts, and the endurance time is still up to 38 minutes. Flight tests show that in the urban building shuttle scene, the target detection MOTA index is 81.2%, meeting the needs of inspection operations; in strong crosswind (wind speed 12 meters / second) conditions, the position estimation drift rate is 0.8 meters per minute, which is better than the level of 1.5 meters / minute of commercial obstacle avoidance systems. The system is successfully applied to power line inspection, emergency search and rescue and other tasks, verifying the practicality and reliability of the lightweight architecture.

[0041] Embodiment 3 This embodiment provides a high-deterministic version system architecture for industrial automation scenarios, focusing on strengthening the running reliability of the system in harsh factory environments such as strong electromagnetic interference, high temperature and humidity, mechanical resonance, etc. The system retains complete functional modules, but introduces redundant design and deterministic scheduling mechanisms at key links.

[0042] The front-end signal preprocessing module adopts a dual-path parallel acquisition architecture, with two sets of main and backup devices configured for each type of sensor, and the data is synchronously acquired and independently processed. The adaptive noise suppression engine runs in parallel with two instances, processing the main and backup channel data respectively and outputting two sets of denoised signals. The multi-modal feature alignment unit introduces a voting mechanism: if the difference between the two feature matching results is less than a certain threshold (e.g. Hamming distance < 15), the average value is taken as the final alignment result; otherwise, an abnormal alarm is triggered and switched to single-path mode. The confidence-driven fusion reasoning core adds a device health index in dynamic confidence assessment, which is reported in real time by the built-in self-checking circuit and includes parameters such as temperature, supply voltage, and crystal oscillator stability. If any parameter exceeds the standard, the corresponding channel confidence is forced to drop below 0.3.

[0043] The fusion reasoning core adopts a dual-core lockstep running mode, with two sets of asynchronous cubature Kalman filter instances running in parallel. After each state update, the results are compared, and if the difference exceeds the safety threshold, the fault isolation program is started. The closed-loop feedback optimization mechanism adds a hardware watchdog timer, which must receive a confirmation signal within a specified time after each parameter adjustment, otherwise it automatically rolls back to the previous stable version. The entire system runs in a safety controller that meets the IEC 61508 SIL-3 standard, with triple modular redundancy storage for all critical variables and ECC error correction mechanism enabled for memory access.

[0044] This embodiment has been deployed on a certain automobile welding production line and has been running continuously for 365 days without failure. It has experienced multiple large welding machine electromagnetic pulse impacts, and the system can maintain normal sensing output, effectively ensuring the precision of robot guidance. Test data shows that during the peak period of electric welding work, the system's misidentification rate of metal splashes is only 0.7 times per hour, which is much lower than the traditional system's 5.2 times per hour, significantly improving production efficiency and safety.

Claims

1. An adaptive noise suppression based multi-source sensor fusion perception system, comprising: The application relates to a multi-sensor fusion system for real-time state estimation, comprising: a front-end signal preprocessing module for receiving raw observation data from sensors of at least three different physical principles, synchronizing time stamps of channel data, and performing preliminary band-pass filtering; an adaptive noise suppression engine for modeling and suppressing noise of each channel data independently, the adaptive noise suppression engine comprising a parallel recursive noise estimation structure for jointly analyzing signal characteristics through at least two ways of sliding window spectral analysis, wavelet packet residual decomposition and online high-order statistics tracking to trigger a noise mode classification process and generate corresponding noise suppression strategy configuration instructions according to the classification results; a multi-modal feature alignment unit for receiving signals after noise suppression, performing spatial coordinate mapping and time resampling operations across sensors, and extracting key geometric features to input into a joint embedding space mapping network to generate a unified representation invariant to modalities; a credibility-driven fusion reasoning core for receiving the unified representation, modulating the contribution proportion of each sensor channel based on the confidence weight output by a dynamic confidence evaluation module, and performing state estimation by using an asynchronous correction mechanism triggered according to the sensor readiness state; a closed-loop feedback optimization mechanism for collecting residual sequences between fusion results and real-world labeled data, and activating an incremental parameter recalibration program to fine-tune the front-end parameters of the system when the distribution characteristics of the residual sequences deviate from the expected distribution by more than a preset threshold.

2. The adaptive noise suppression based multi-source sensor fusion perception system of claim 1, wherein, The at least three different physical principle sensors include an optical imaging device, a radio frequency detection device and a motion state detection unit.

3. The adaptive noise suppression based multi-source sensor fusion perception system of claim 1, wherein, The sliding window spectral analysis submodule in the recursive noise estimation structure is used for calculating the moment variation rate of the signal power spectral density to identify abnormal energy aggregation frequency bands; the wavelet packet residual decomposer is used for evaluating the local singularity strength through signal reconstruction error; and the online high-order statistics tracker is used for monitoring the dynamic trajectories of skewness and kurtosis indicators of the signal amplitude distribution.

4. The adaptive noise suppression based multi-source sensor fusion perception system of claim 1, wherein, The spatial mapping in the multi-modal feature alignment unit unifies the observation coordinate systems of each sensor to a global coordinate system through a pre-calibrated extrinsic parameter matrix, and introduces a correction algorithm based on surface interpolation to compensate for the deviation caused by installation deformation; and the time resampling adopts an adaptive synchronous filter bank to dynamically adjust the resampling phase according to the latest effective data arrival time of each channel.

5. The adaptive noise suppression based multi-source sensor fusion perception system of claim 4, wherein, The key geometric features include an edge gradient tensor extracted from optical imaging device data, a normal vector distribution histogram extracted from radio frequency detection device point cloud, and a reflection intensity profile extracted from radio frequency reflection signals.

6. The adaptive noise suppression based multi-source sensor fusion perception system of claim 1, wherein, The dynamic confidence evaluation module is used for constructing a real-time credibility scoring function based on the light intensity, signal-to-noise ratio level, target relative speed and occlusion degree of the current scene to output continuous confidence weights.

7. The adaptive noise suppression based multi-source sensor fusion perception system of claim 1, wherein, The credibility-driven fusion reasoning core further comprises a topological consistency verification module for performing geometric rationality testing on a multi-view candidate target set before fusion by using a pre-trained scene prior knowledge base to eliminate false assumptions that do not conform to the physical space constraints.

8. The adaptive noise suppression based multi-source sensor fusion perception system of claim 1, wherein, The asynchronous correction mechanism is configured to scale the observation covariance matrix of any sensor channel according to the confidence weight of the channel and perform local state correction when the channel submits a new valid observation, without waiting for other channels to be ready.

9. The adaptive noise suppression based multi-source sensor fusion perception system of claim 1, wherein, The incremental parameter re-calibration procedure in the closed-loop feedback optimization mechanism adopts a gradient-free stochastic approximation algorithm to fine-tune the front-end filter parameters and wavelet threshold parameters without affecting the real-time performance of the main task.

10. The adaptive noise suppression based multi-source sensor fusion perception system of claim 1, wherein, The system runs on a heterogeneous computing platform, where the front-end preprocessing and noise suppression operations are performed by a programmable logic array, the feature extraction and alignment operations are accelerated in parallel by a graphics processor, and the fusion inference and closed-loop optimization tasks are scheduled by a real-time operating system; the system supports a hot-plug sensor expansion interface, and automatically triggers a self-learning calibration process to complete the initial value estimation of internal and external parameters and noise base modeling after a new sensor is added.

Images