Heterogeneous sensing early warning system and method based on decoupling perception and robust learning adversarial

Abstract

The invention discloses a heterogeneous sensing early warning system based on decoupling perception and adversarial robust learning, and the system comprises a feature extraction module which processes heterogeneous sensor original data collected in real time through a multi-layer decoupling encoder, separates target related features and environment interference features, and suppresses noise pollution from the source; the multi-dimensional collaborative fusion module adopts a cross-domain adversarial robustness learning framework to carry out space-time sequence alignment and deep fusion on decoupling features to generate high-robustness joint representation, and a data missing problem is processed through a cross-modal generative feature completion mechanism; and the cognitive enhancement closed-loop decision module constructs a cognitive heuristic confidence evaluation model based on joint representation, realizes graded early warning by combining real-time quality scoring and behavior prediction, and dynamically optimizes system parameters through a feedback mechanism. According to the method, the problems of poor target detection robustness, high delay and low accuracy in a complex dynamic environment are solved, the detection precision is remarkably improved, the false alarm rate is reduced, and the all-weather adaptive capacity is enhanced.

Description

Technical Field

[0001] The technical field to which the present invention relates is the field of artificial intelligence technology, and specifically refers to a heterogeneous sensing early warning system and method based on decoupled perception and adversarial robust learning. Background Art

[0002] In today's technological landscape, with the rapid development of cutting-edge applications such as autonomous driving, intelligent robotics, and security surveillance, the demand for ultra-low latency, high-precision, and highly robust real-time detection of targets in complex and dynamic environments is becoming increasingly urgent. In particular, with the widespread application of multi-source, heterogeneous sensors (such as radar, vision, ultrasound, and lidar) in areas such as intelligent transportation and industrial automation, the key challenges facing current technologies are how to efficiently and accurately process and integrate this massive amount of heterogeneous data, while ensuring stable system operation in harsh and even unknown environments.

[0003] When processing multimodal data collected by integrated sensors and deploying it to edge devices for ultra-low-latency real-time detection, existing modal perception technologies suffer from poor robustness to complex environmental interference, leading to contamination of raw features. Traditional radar and visual data processing typically extracts features directly from raw sensor input and performs simple early or late fusion. For example, patents such as CN119933827A propose a control method for a steam turbine reheat system based on multimodal data fusion. However, this method suffers from raw feature contamination and insufficient feature decoupling, failing to dynamically decouple the complex interactions between multimodal features to improve prediction and control accuracy. For example, patent CN120083597A proposes a real-time automobile exhaust flow monitoring system based on multimodal sensors and edge computing. However, this system lacks a deep decoupling mechanism for target-related features and environmental interference features in multimodal data, failing to effectively separate interference features to improve monitoring accuracy in complex dynamic environments. As a result, during the subsequent fusion and recognition process, contaminated raw features accumulate and propagate errors, severely impacting model accuracy and system reliability in harsh environments.

[0004] Secondly, after deployment, existing systems will experience a sharp decline in performance when faced with environmental interference that does not match the distribution of training data or is even completely unseen, resulting in high false positives or high missed negatives, and are unable to fundamentally solve the system's adaptive problems in unknown or extremely complex environments. Although some technologies attempt to improve model robustness through data enhancement, these methods often rely on preset interference types and intensities, making it difficult to generalize to the complex, changeable, and unpredictable interference patterns in the real world. For example, patent CN120096565A proposes a vehicle adaptive cruise control method based on multimodal sensor fusion and neural networks, but it lacks an intrinsic robustness learning mechanism and fails to effectively respond to unforeseen complex environmental interference through adaptive feature invariance learning.

[0005] Furthermore, traditional real-time detection systems typically perform target detection and tracking based on sensor data from the current frame, with processing and transmission paths often fixed or simple dynamic. However, in highly dynamic applications like autonomous driving, targets (such as vehicles and pedestrians) exhibit complex motion trajectories, potentially involving high-speed movement, sudden changes in direction, or multi-target interactions. For example, patent CN119960465A proposes a control method for pointing machines based on multimodal sensors and dynamic path planning. However, this method lacks multi-target behavior prediction and resource focusing capabilities, failing to dynamically adapt to the multi-target trade-offs and resource allocation requirements in complex traffic scenarios. Existing technologies fail to effectively predict future target behavior or feed this prediction information into the feature extraction and fusion process. This makes it difficult to achieve timely information transmission and decision-making when dealing with high-speed, densely packed, or occluded targets, compromising the continuity of recognition and transmission. Furthermore, the lack of the ability to dynamically focus computing resources and attention on key targets and prediction areas results in computational overhead being disproportionate to performance gains, making it difficult to meet ultra-low latency requirements.

[0006] Therefore, the invention of a heterogeneous sensing warning system based on decoupled perception and adversarial robust learning can overcome the key technical difficulties faced by current multimodal intelligent perception systems in the face of various complex dynamic environments, such as original feature pollution, insufficient environmental generalization ability, and low efficiency in identifying and transmitting high-dynamic and non-rigid obstacles and low decision-making accuracy, thereby significantly improving driving safety, which has become a technical problem that needs to be solved urgently. Summary of the Invention

[0007] The purpose of the present invention is to provide a heterogeneous sensing warning system and method based on decoupled perception and adversarial robust learning. The present invention can solve the problem of feature contamination caused by the raw data of sensors being easily affected by environmental interference, the problem of poor generalization ability of the model for unknown or complex interference and high false alarm rate, and the problem of large recognition transmission delay and high missed detection rate in high dynamic scenes. It realizes deep decoupling of environmental noise, cross-domain adversarial robust learning and predictive decision-making closed loop.

[0008] To achieve this goal, the present invention designs a heterogeneous sensing warning system based on decoupled perception and adversarial robust learning, which includes: The feature extraction module is used to collect the raw sensor data from heterogeneous sensors in real time. Through the training of the set multi-layer decoupling encoder, the decoupled sensor target related features and environmental interference features are obtained. The multi-dimensional collaborative fusion module is used to align and fuse the decoupled sensor target related features through a cross-domain adversarial robust learning framework to obtain the fused joint representation features. The fused joint representation features are then subjected to multi-level fine semantic recognition to obtain fine semantic recognition results. At the same time, cross-modal generative feature completion is performed on the fused joint representation features to obtain the completed joint representation features. The cognitive-enhanced closed-loop decision-making module is used to construct a cognitive-inspired confidence assessment model based on the results of refined semantic recognition, the completed joint representation features, and the historical trajectory information of the sensor target. Based on the cognitive-inspired confidence assessment model, real-time quality scores are performed on the raw sensor data sensed by heterogeneous sensors, the decoupled sensor target-related features, and the environmental interference features. According to the real-time quality scores and the sensor target behavior prediction results, graded warnings are issued for the sensor target behavior.

[0009] Preferably, the cross-domain adversarial robustness learning framework of the multi-dimensional collaborative fusion module in the multi-dimensional collaborative fusion module specifically includes: an interference domain generator for generating simulated environmental interference features, a cross-domain discriminator for determining whether a feature is an interfered feature, a spatial alignment module and a deep temporal alignment module for deep feature alignment, and a deep fusion module based on a neural network architecture; The interference domain generator is obtained by training a generative adversarial network generator. It uses real-time environmental parameters and random noise provided by the environmental adaptive enhancement system as input to generate simulated environmental interference features. These interference features are then superimposed with the target-related features output by the modal decoupling and noise-resistant feature extraction module and fed into the fusion network as contaminated input for robustness training. The specific method of obtaining the cross-domain discriminator is: training a discriminator whose input is the fused joint feature representation. The cross-domain discriminator is used to distinguish whether the fused joint representation features are the original sensor data distribution sensed by heterogeneous sensors, or the data distribution after the fusion of contaminated target related features containing environmental interference features. At the same time, the feature fusion network of the deep fusion module attempts to deceive the discriminator through adversarial training, so that its output composite feature representation remains invariant under different interference domains.

[0010] Preferably, the specific method for obtaining the refined semantic recognition result in the multi-dimensional collaborative fusion module is: based on the fused joint representation features, a neural network model for high-precision target detection is used to perform target classification optimization and bounding box regression optimization to identify the specific category of the obstacle; The specific method to obtain the completed joint representation features is: when the data quality of a certain modality cannot meet the fusion reliability, the effective information of other modalities is used to complete the effective features of the modality through conditional generative adversarial networks or diffusion models.

[0011] Preferably, it also has a compensation strategy generation module, including an environment performance impact model, a combination of reinforcement learning and meta-learning methods, and a strategy smooth transition mechanism; Environmental performance impact model: Build a multi-layer perceptron network that takes as input the real-time and predicted environmental parameters obtained by the environmental adaptive enhancement system and outputs the expected performance of each modal sensor and recommended adjustment values ​​for specific system parameters. Combining reinforcement learning and meta-learning: A reinforcement learning agent observes the current environment state and system performance, outputs parameter adjustment values, and uses a reward function designed to maximize detection accuracy and minimize latency, while penalizing false positives and false negatives. By introducing a meta-learning strategy into reinforcement learning training, the agent can quickly learn the optimal compensation strategy from a small amount of new environmental data, enabling proactive environmental exploration and adaptation. Strategy smooth transition mechanism: When a new compensation strategy is generated, the system does not switch immediately, but gradually transitions within a set number of frames through linear interpolation.

[0012] Preferably, the multi-dimensional collaborative fusion module also includes real-time data quality assessment and adaptive adjustment strategy. The method of real-time data quality assessment is: constructing a lightweight evaluation network to perform real-time modality quality scoring on the original data of each modality of each frame and the decoupled target-related features; The adaptive adjustment strategy is to dynamically adjust the contribution weight and processing priority of each modality in the fusion based on the real-time quality score, the prediction results of the behavior prediction network and the real-time environmental assessment of the environmental adaptive enhancement system. The contribution weight can be calculated in real time by a lightweight multi-layer perceptron according to the real-time modality quality score and the real-time environmental parameters obtained by the environmental adaptive enhancement system.

[0013] Preferably, a heterogeneous sensing warning method based on decoupled perception and adversarial robust learning includes: Collect raw data in real time and obtain decoupled target-related features and environmental interference features through the set multi-layer decoupling encoder training; Based on the target-related features and environmental interference features, feature alignment and fusion are performed through a cross-domain adversarial robustness learning framework to obtain the fused joint representation features. The fused joint representation features are then subjected to multi-level fine semantic recognition to obtain fine semantic recognition results. At the same time, cross-modal generative feature completion is performed to obtain the completed joint representation features. A cognitive-inspired confidence assessment model is constructed based on the results of fine-grained semantic recognition, the completed joint representation features, and the collected historical trajectory information. Real-time quality scoring is performed based on the original data, decoupled target-related features, and decoupled environmental interference features. Graded warnings are issued based on the real-time quality scores and behavior prediction results, and the sensor detection warning results are fed back.

[0014] Beneficial Effects of the Invention: This invention proposes a heterogeneous sensor warning system based on decoupled perception and adversarial robust learning. By introducing a multi-layer decoupled encoder in the feature extraction module, it deeply separates target-related features from environmental interference features in the raw sensor data. Combined with the cross-domain adversarial robust learning framework in the multidimensional collaborative fusion module, the model is forced to learn feature invariance under simulated or real-world interference, suppressing contamination of the raw data by interference such as illumination changes, mechanical vibration, and electromagnetic noise. The multidimensional collaborative fusion module uses an adaptive spatial transformation network and an improved iterative closest point algorithm for spatial alignment. Combined with a dynamic time warping algorithm based on deep temporal alignment, the system corrects for spatiotemporal differences in target-related features of different sensor types. Furthermore, the module utilizes a cross-modal attention mechanism to achieve multi-level feature fusion and combines it with a conditional generative adversarial network or diffusion model for cross-modal generative feature completion, automatically completing missing features when a single modality fails. Combined with the adaptive adjustment strategy of the real-time data quality assessment network, the system dynamically optimizes the contribution weight and processing priority of each modality, focusing computing resources on key target areas to meet the ultra-real-time warning requirements of highly dynamic scenarios such as rail transit. A cognitive-inspired confidence assessment model is constructed through a cognitive-enhanced closed-loop decision-making module, and a graph neural network is combined to analyze multimodal consistency, semantic scene relevance, and multi-dimensional uncertainty to quantify the reliability of the decision. Based on confidence scoring and real-time multi-target behavior prediction, the system implements hierarchical warnings, and dynamically adjusts sensor parameters and fusion weights by combining the environmental performance impact model of the compensation strategy generation module and the reinforcement learning meta-learning mechanism. Through the above-mentioned end-to-end closed-loop optimization architecture, the present invention provides a set of efficient, robust, and adaptive solutions for heterogeneous sensor warnings in complex dynamic environments, solving the problems of existing technologies in terms of environmental interference robustness, feature fusion efficiency, and decision-making accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a structural schematic diagram of the present invention; Figure 2 Schematic diagram of the process of the model of the present invention; Figure 3 This is the flow chart of the modal decoupling and anti-noise feature extraction module; Figure 4 This is the flowchart of the adversarial domain-invariant multi-dimensional collaborative fusion module; Figure 5 The flowchart of the cognitive enhanced closed-loop decision engine. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1 A heterogeneous sensing warning system based on decoupled perception and adversarial robust learning, such as Figure 1 As shown, it includes: The feature extraction module is used to collect the raw sensor data from heterogeneous sensors in real time. Through the training of the set multi-layer decoupling encoder, the decoupled sensor target related features and environmental interference features are obtained. The multi-dimensional collaborative fusion module is used to align and fuse the decoupled sensor target related features through a cross-domain adversarial robust learning framework to obtain the fused joint representation features. The fused joint representation features are then subjected to multi-level fine semantic recognition to obtain fine semantic recognition results. At the same time, cross-modal generative feature completion is performed on the fused joint representation features to obtain the completed joint representation features. The cognitive-enhanced closed-loop decision-making module is used to construct a cognitive-inspired confidence assessment model based on the results of refined semantic recognition, the completed joint representation features, and the historical trajectory information of the sensor target. Based on the cognitive-inspired confidence assessment model, real-time quality scores are performed on the raw sensor data sensed by heterogeneous sensors, the decoupled sensor target-related features, and the environmental interference features. According to the real-time quality scores and the sensor target behavior prediction results, graded warnings are issued for the sensor target behavior.

[0018] In the above technical solution, the environmental interference features are used in the perception decoupling and noise-resistant feature extraction module to supervise the decoupling process (as input to the discriminator and used to calculate the environmental supervision loss) to ensure the purity of the features. In addition, they are used as supplementary information in the adversarial resilience predictive collaborative fusion module. During cross-modal generative feature completion, when the system detects extremely poor quality data from a certain modality in a complex environment, resulting in unreliable target features, it uses valid information from other modalities and combines it with the interference features corresponding to that modality to generate supplementary features that are more realistic and more consistent with the current environment.

[0019] Data preprocessing is required for collecting raw sensor data from heterogeneous sensors. Some optimized technical solutions include: For radar data, the system converts the original millimeter-wave radar echo signal into a range Doppler map through FFT (Fast Fourier Transform), and then detects the target through the CFAR (Constant False Alarm Rate) algorithm to generate the original point cloud. , where RCS is the radar cross section, is the X coordinate of the i-th radar point in three-dimensional space, is the Y coordinate of the i-th radar point in three-dimensional space, is the Z coordinate of the i-th radar point in three-dimensional space, is the radial velocity of the i-th radar point; For visual data, the system converts visible light images and close-range infrared images Distortion correction and noise removal preprocessing (such as guided filtering or non-local mean filtering) are performed. Infrared images provide additional assistance in nighttime and low-light environments.

[0020] For ultrasonic data, the system converts the original ultrasonic echo signal Extract accurate distance information after bandpass filtering, envelope extraction and threshold detection .

[0021] For lidar data, the system converts the original lidar point cloud After preprocessing such as voxel downsampling and ground filtering, background points are removed.

[0022] For environmental data, the system converts the light intensity , ambient temperature , vibration acceleration , electromagnetic noise intensity Environmental sensor data is collected in real time and normalized.

[0023] In the above technical solution, the multi-layer decoupling encoder set in the feature extraction module specifically includes a visual decoupling encoder for visible light images and close-range infrared images, and a multimodal point cloud and signal decoupling encoder for the original point cloud, original ultrasonic echo signal, and original lidar point cloud. The obtained sensor target related features are divided into visual sensor target related features and point cloud signal sensor target related features, and the obtained environmental interference features are divided into visual environmental interference features and point cloud signal environmental interference features. Among them, the visual decoupling encoder for visible light images and close-range infrared images The lightweight mobile network (MobileNet-V3) structure improved based on the lightweight convolutional neural network (CNN) consists of a shared backbone network and two separate disentanglement heads. The backbone network extracts common features, and the decoupling heads are mapped to the target domain and the environment domain respectively. The network depth is 5 layers and the output feature dimension is 64. In its decoupling mechanism, the adversarial disentanglement strategy is adopted to realize dynamic game by introducing two discriminators. The two discriminators are the discriminators that receive the relevant features of the visual sensor target and the discriminators that receive the relevant features of the visual sensor target. and a discriminator that receives visual environment interference features , which is used to force the visual sensing target-related features to remain invariant to environmental changes, force the visual environment interference features to capture environmental information, and optimize feature extraction as a whole through the visual loss function, where the specific formula of the visual loss function is: ; ; ; ; ; ; Where, For reconstruction loss, (such as L1 or L2 norm), ensure that the decoupled features can be reconstructed back to the original image through the decoder , maintaining information integrity. Decoupled encoder for vision and the discriminator The adversarial loss between them, usually binary cross entropy loss, is used to train the encoder and the discriminator The game between. For encoder and the discriminator The confrontation loss between It is a preliminary target classification loss (such as cross entropy) based on the relevant features of the visual sensing target to ensure the validity and classifiability of the target features. is the true label, is the environmental supervision loss, such as mean squared error or cross entropy, used to supervise and actual environmental parameters The correlation between the two ensures that it can effectively encode environmental interference information. is the weight coefficient, which is used to balance various losses. are the various loss function values ​​in the visual loss function, The total visual loss function, is the original visual image, is the reconstruction loss, For the first fight loss, is the preliminary target classification loss, Losses due to environmental supervision, is the visual sensing target related feature, is the visual environment interference feature, is the visual decoder, is the true label, is the normalized environment data, k is the number of visual loss functions calculated; Multimodal point cloud and signal decoupling encoder for raw point cloud, raw ultrasonic echo signal and raw lidar point cloud , using a customized lightweight network, using a lightweight version of PointNet++ for point cloud processing (PointNet++ is a deep learning network for processing point cloud data), combining one-dimensional convolution and Transformer Encoder for time series signals; its decoupling mechanism uses the Mutual Information Minimization strategy to force point cloud signals to sense target-related features and point cloud signal environment interference characteristics The statistical independence between them is achieved by optimizing the feature extraction through the point cloud signal loss function. The specific formula of the point cloud signal loss function is: ; ; ; ; Where, is the total loss function of the point cloud signal, is based on The target detection loss is is the true detection label, To measure the mutual information and the degree of dependence between two random variables, in practice, mutual information is usually estimated by methods such as MINE (Mutual Information Neural Estimation) or InfoMax principle, whose goal is to minimize the shared information between the target feature and the interference feature. is the environmental supervision loss for this modality, are the various loss function values ​​in the point cloud signal loss function, is the weight coefficient, m is the number of loss functions calculated, is the target detection loss based on the relevant features of the sensing target, is mutual information, Losses due to environmental supervision, is the true detection label.

[0024] In the above technical solution, the discriminator take over , trying to predict whether it contains environmental information (such as light intensity, fog, raindrops, etc.). Attempt to deceive , making it impossible to Extract environmental information from Remains invariant to environmental changes. In addition, During training, it also receives normalized environment data from the "multi-source data preprocessing" link , as an additional supervisory signal, ensuring Can accurately capture and normalize environmental data Related environmental interference patterns.

[0025] In the above technical solution, PointNet++ is a deep learning network for processing point cloud data. Its core concept is hierarchical feature learning. It iteratively samples, groups, and extracts local features from point clouds, gradually abstracting higher-level global features from local details. First, a set of representative points is selected from the input point cloud data as centroids to construct local regions. Then, based on the centroids sampled in the previous step, a local neighborhood is defined for each centroid, and nearby points are aggregated into a group. Feature learning is then performed independently on each local neighborhood group to capture the geometric and semantic characteristics of the local region. Finally, by stacking multiple "collective abstraction" modules, the receptive field of the features is gradually increased, and higher-level global features are aggregated from local features. Through this hierarchical and iterative mechanism, PointNet++ effectively learns robust features with multi-scale perception and structural information from raw point cloud data.

[0026] In the above technical solution, if Figure 3 As shown in the figure, the feature extraction module uses adversarial decoupling (vision) and mutual information minimization (point cloud) to achieve source noise separation. The loss function design ensures that the interference features accurately capture the real environment parameters, providing high-quality input for subsequent fusion to improve feature purity, reduce computational overhead, and ensure cross-modal target feature consistency.

[0027] In the above technical solution, the cross-domain robustness learning framework of the multi-dimensional collaborative fusion module in the multi-dimensional collaborative fusion module specifically includes: an interference domain generator for generating interference features of the simulated environment, a cross-domain discriminator for determining whether a feature is an interfered feature, a spatial alignment module and a deep temporal alignment module for deep feature alignment, and a deep fusion module based on a neural network architecture; Interference Field Generator The specific way to obtain is: train a generator of a generative adversarial network , which uses the real-time environmental parameters provided by the environment adaptive enhancement system and random noise As input, generate simulated environmental interference characteristics , these interference features will be combined with the target related features output by the modal decoupling anti-noise feature extraction module The superposition is fed into the fusion network as the contaminated input for robustness training; Cross-origin discriminator The specific way to obtain is: train a discriminator , whose input is the fused joint feature representation , cross-domain discriminator It is used to distinguish whether the fused joint representation feature is the original sensor data distribution sensed by the heterogeneous sensor or the data distribution after the fusion of the polluted target related features containing the environmental interference features. At the same time, the feature fusion network of the deep fusion module Through adversarial training, we try to fool the discriminator , so that its output Remains invariant under different interference domains.

[0028] In the above technical solution, the environmental adaptive enhancement system is an independent module that can collect and process environmental data in real time and predict environmental changes. It is a software module that provides real-world conditional input for the interference domain generator to simulate interference.

[0029] In the above technical solution, random noise is sampled from a standard distribution (such as Gaussian distribution or uniform distribution), and its dimension matches the input requirements of the generator.

[0030] In the above technical solution, the interference domain generator is used to simulate unknown environmental interference, and the cross-domain discriminator is combined to enforce feature invariance, thereby solving the problem of insufficient environmental generalization ability, enhancing the zero-sample generalization ability, and suppressing the possibility of false target false alarms.

[0031] In the above technical solution, the steps of the multi-dimensional collaborative fusion module are as follows: Figure 4 As shown in the figure, the specific method of feature alignment and fusion in the multi-dimensional collaborative fusion module through the cross-domain adversarial robustness learning framework is as follows: The spatial alignment module and the deep temporal alignment module align the relevant features of the sensor targets. The spatial alignment module uses an adaptive spatial transformation network and an improved iterative closest point algorithm based on Kalman filtering to correct the spatial differences of the relevant features of different types of sensor targets. The deep temporal alignment module uses an improved dynamic time warping algorithm to correct the temporal differences of the relevant features of different types of sensor targets. The specific method of the adaptive spatial transformer network STN (Spatial Transformer Network) is as follows: for the visual sensing target related features, the affine transformation parameters in the STN layer No longer fixed, but real-time environmental parameters provided by the environmental adaptive enhancement system and the behavior prediction results obtained by the real-time multi-target behavior prediction network Dynamically generated, the specific calculation formula is: ; This enables the correction of image features to adapt in real time to changes in sensor pose, target motion, and the nonlinear errors of the sensor itself; The specific method of the improved iterative closest point algorithm based on Kalman filtering is as follows: in the fine registration link, the Kalman filter is used to predict the initial transformation matrix of the current frame , significantly reducing the number of iterations of the iterative closest point method, and using the point cloud signal sensing target-related features to iteratively optimize the iterative closest point method to obtain more accurate registration , the Kalman filter prediction formula is: ; Where, is the state vector at time t, is the state transition matrix, is the state vector at time t-1, is the control input matrix, is the control vector at time t, is the process noise vector at time t; The loss function of the iterative closest point method is: ; Where, is the source point cloud, is the corresponding point of the target point cloud, is the loss function value of the iterative closest point method, is the total number of point pairs successfully matched in the current iteration, is the i-th pair of matching points; The method of deep time series alignment is to use an improved dynamic time warping algorithm. The specific method is: a real-time dynamic time warping algorithm based on a sliding window is used to reduce the computational complexity from , optimized to ,in, is the complexity, W is the sliding window size, M and N represent the lengths of the two time series involved in the alignment. In real-time applications, M is usually the new incoming shorter sequence (the data of the current frame), and N is the historical reference sequence (or the longer template sequence). The sliding window size of the dynamic time warping algorithm The similarity measurement parameters are dynamically adjusted by the environment adaptive enhancement system according to the environment stability and target motion speed; The aligned sensor target-related features are then fused using a deep fusion method. Based on the cross-modal attention mechanism of the neural network architecture (Transformer-based), the target-related features of each modality learn the mutual dependence of different modalities at different spatial and temporal scales, and output the fused joint representation features. In the cross-modal attention mechanism, the similarity scores are normalized into probability distributions, and the specific formula for generating attention weights is: ; Where, is the query matrix, is the bond matrix, is the value matrix, is the dimension of the key, Softmax is the normalized exponential function, is the matrix transpose operator.

[0032] In the above technical solution, dynamic STN and lightweight DTW algorithms are used to achieve precise alignment to address the problems of sensor posture offset (vehicle vibration) and timing asynchrony, thereby improving spatial registration accuracy, reducing computational complexity, and optimizing the efficiency of high-dynamic target tracking.

[0033] In the above technical solution, the specific method for obtaining fine semantic recognition results in the multi-dimensional collaborative fusion module is: Based on the fused joint representation features, a neural network model for high-precision object detection (such as an improved ViT or ResNet-based detection head) is used to perform object classification optimization and bounding box regression optimization to identify specific categories of obstacles (such as children, adults, suitcases, strollers, umbrellas, pets, etc.); The specific method to obtain the completed joint representation features is: When the data quality of a certain modality cannot meet the fusion reliability requirements, the effective information of other modalities is used to supplement the effective features of the modality through conditional generative adversarial networks or diffusion models.

[0034] In the above technical solutions, the situation where fusion reliability cannot be met can be reflected in the following examples: 1. Missing key information: For example, the key recognition area of ​​the target in the visual modality (such as the face or limbs) is completely blocked or blurred to the point of being unrecognizable; 2. Feature extraction failure: The decoupled encoder cannot extract valid target-related features from the modal data.

[0035] In the above technical solution, multi-level fine semantic recognition and cross-modal generative feature completion also include cGAN and diffusion model. The specific implementation method of cGAN and diffusion model is: train a completion generator with effective modal features and environmental interference features of the modality to be completed. As a condition, a simulated, high-quality completion feature is generated. The completion discriminator is responsible for distinguishing the generated features from the true corresponding modality features.

[0036] The above technical solution solves the problems of difficulty in identifying non-rigid obstacles and vulnerability to modal failure. It enhances robustness through the ViT classification head and conditional generation completion mechanism, improves the recognition rate of complex obstacles, and avoids missed detections caused by modal failure.

[0037] In the above technical solution, the cognitive enhancement closed-loop decision module is as follows: Figure 5 As shown, the specific method of constructing a cognitive heuristic confidence evaluation model and performing real-time quality scoring based on the cognitive heuristic confidence evaluation model is as follows: Perform multimodal consistency modeling and represent the completed joint features , fine semantic recognition results and historical trajectory information of the sensor target Build as a graph , using graph neural networks to learn the consistency between joint representation features, fine semantic recognition results and historical trajectory information of sensor targets. For example, by calculating the node embedding similarity output by GNN, the degree of consensus of different modalities on the same target is quantified. Similarity in graph neural networks The formula is as follows: ; Where, For graph neural networks, GNN can use graph convolutional networks (GCN) or graph attention networks (GAT). Node features are the location, size, and category probability of the target, and edge features are the distance and relative speed between targets. Perform semantic scene correlation analysis and combine it with pre-built high-precision semantic maps ,Evaluate the rationality of detected obstacles and current target positions,status in high-precision semantic maps; Calculating confidence scores : in, is the learnable weight of similarity, is the learnable weight of uncertainty, is the learnable weight of semantic scene matching, is the activation function, Score the semantic scene matching, is the uncertainty of the target position, uncertainty in class identification, is the target motion uncertainty, and the confidence score is the real-time quality score.

[0038] When performing semantic scene relevance analysis, in some preferred embodiments of the present invention, a high-precision semantic map of the rail transit platform area and the interior of the train body is combined with the pre-built , evaluate the rationality of the detected obstacles (such as human body, luggage) and the current position and posture in the platform door scene. For example, when a "foreign object" is detected in the door gap and its size matches the door gap, its confidence will be based on The regional semantic information in the image is improved.

[0039] In the above technical solution, the historical trajectory information is the original sensor data stream directly obtained through various heterogeneous sensors (such as radar, vision, ultrasound, lidar, etc.).

[0040] In the above technical solution, is the uncertainty of the target position, The uncertainty of the category identification and These uncertainties are obtained through methods such as Bayesian neural network (BNN) or Monte Carlo Dropout.

[0041] In the above technical solution, the confidence score is calculated Finally, it also includes real-time multi-target behavior prediction and dynamic attention mechanism, the purpose of which is to improve the recognition and transmission robustness of highly dynamic, multi-target, and non-rigid obstacles, and optimize resource utilization. Its behavior prediction network uses the Graph Transformer Network (GTN), which uses the multiple targets identified by the adversarial domain invariant multi-dimensional collaborative fusion module as nodes of the graph, and the spatial distance, relative speed and historical motion trajectory between them as edges. GTN learns the complex interactions between nodes through a multi-head self-attention mechanism, and predicts the possible position, speed, posture and probability of entering the door gap area of ​​each target in the next N frames (for example, N=10, corresponding to 200ms), which is the prediction result of the behavior prediction network. The specific formula is as follows: in It's the past The historical trajectory sequence of frames, is the completed joint representation feature, and t is the current time point.

[0042] In the above technical solution, the behavior prediction result also includes dynamic attention guidance, that is, the predicted behavior information It is used to dynamically adjust the priorities and computing resource allocation of various modules within the system. For example, if the GTN predicts that a passenger will approach a door gap within the next few frames, the Transformer-based attention mechanism of the adversarial domain-invariant multidimensional collaborative fusion module will give higher weight to the passenger's relevant radar, ultrasonic, and close-range visual features when performing feature fusion. Simultaneously, this attention information (in the form of masks or weights) is passed through a feedback path to the modal decoupling and noise-resistant feature extraction module, guiding it to perform more refined feature decoupling or sampling of the high-risk area in the next frame.

[0043] In the above technical solution, by quantifying decision uncertainty (position / classification / motion variance), the human cognitive process is simulated to solve the problem of inaccurate decision-making for highly dynamic targets, which is conducive to dynamically adjusting the alarm threshold and effectively supporting hierarchical response.

[0044] The above technical solution also includes a compensation strategy generation module, including an environmental performance impact model, a combination of reinforcement learning and meta-learning, and a strategy smooth transition mechanism; Environmental Performance Impact Model: Build a multi-layer perceptron (MLP) network that takes as input the real-time and predicted environmental parameters obtained by the environmental adaptive enhancement system. It then outputs the expected performance of each modality sensor (e.g., visual clarity, radar ranging accuracy) and recommended adjustments to specific system parameters (e.g., camera exposure time, gain, radar transmit power, ultrasonic pulse frequency, signal processing threshold, and initial weights for each modality in fusion). Reinforcement learning (RL) and meta-learning combined method: Based on the reinforcement learning agent observing the current environment state and system performance, output parameter adjustment value, reward function Designed to maximize detection accuracy and minimize latency, while penalizing false positives and false negatives; introducing meta-learning strategies (such as MAML, Model-Agnostic Meta-Learning) into reinforcement learning training enables reinforcement learning agents to quickly learn optimal compensation strategies from a small amount of new environmental data, enabling active environmental exploration and adaptation; Strategy smooth transition mechanism: When a new compensation strategy is generated, the system does not switch immediately, but gradually transitions within the set number of frames through linear interpolation. The specific calculation formula for the compensation strategy value is: ; Where, It represents the compensation strategy (or adjusted parameter value) actually applied by the system at the current time point t, and is a dynamically changing quantity. Indicates the total number of frames for policy transition. This is a pre-set integer, for example, 5 frames, representing the number of processing cycles required to complete a smooth transition from the old policy to the new policy. Indicates the old compensation strategy (or old parameter value) that the system is currently using when the transition starts. represents the target compensation policy (or target parameter value) newly generated by the reinforcement learning / meta-learning method. t represents the number of frames from the start of the transition ( t = 0 ) to the current moment.

[0045] In the above technical solutions, the expected performance is obtained through pre-modeling or training. The details are as follows: Offline modeling: During the system development phase, extensive experiments and data collection are conducted to test the actual performance indicators (such as detection accuracy, false alarm rate, signal-to-noise ratio, and robustness of feature extraction) of each modal sensor (and its corresponding processing module) under various environmental conditions (for example, varying light intensity, fog, rainfall, and electromagnetic interference levels). Build a regression model: Use these experimental data as a training set, input "environmental parameters," and output "modal performance under the corresponding environment." A multilayer perceptron network is used to learn this mapping relationship between "environmental parameters" and "modal performance."

[0046] In the above technical solution, sensor parameters are dynamically adjusted based on the environmental performance prediction model and RL agent to achieve active environmental adaptability compensation, effectively improve parameter tuning efficiency, and avoid strategy switching oscillations.

[0047] In the above technical solution, the multi-dimensional collaborative fusion module also includes real-time data quality assessment and adaptive adjustment strategies. The real-time data quality assessment method is to build a lightweight evaluation network to perform real-time modality quality scoring (for example, visual image clarity, radar point cloud density, ultrasonic echo signal-to-noise ratio) on the raw data of each modality and the decoupled target-related features of each frame. Adaptive adjustment strategy is based on real-time quality scoring , prediction results of behavior prediction network Real-time environmental assessment of the environmental adaptive enhancement system and dynamic adjustment of the contribution weight of each modality in the fusion and processing priorities, e.g. when Below the threshold and When the performance is good, the system automatically switches to the radar-dominated "radar-dominated mode", increasing the radar's weight in the fusion and possibly skipping some visual fine processing to save computing resources. Lightweight multi-layer perceptron (MLP) can be used to score the modality quality in real time and real-time environmental parameters acquired by the environmental adaptive enhancement system Real-time calculation.

[0048] In the above technical solution, the lightweight evaluation network dynamically allocates modal weights to resolve the contradiction between high-speed target processing delay and modal failure vulnerability, realize intelligent resource focusing, and elastically support degradation processing solutions.

[0049] When the present invention performs graded warning on the sensor target behavior based on the real-time quality score and the sensor target behavior prediction result, in some preferred embodiments, the graded alarm signal is generated specifically as follows: 1. Warning: When (such as 0.1) and the target is close to the door, a voice prompt is issued; 2. Level 1 alarm: When (such as 0.95) and (e.g. 0.5), an audible and visual alarm will be sounded immediately and a door stop command will be sent; 3. Second level alarm: When (e.g. 0.99) and an obstacle is confirmed to be caught in the door gap, an emergency alarm is issued and a reset command is sent; The decision result feedback is to feed back the final decision (including confidence, decision type, alarm level, environmental assessment, etc.) to the feature extraction module and multi-dimensional collaborative fusion module in real time: 1. Feature extraction module: This module guides the system to perform finer feature decoupling of high-risk areas or adjust sensor parameters (for example, radar scanning frequency in a specific area) in the next frame. 2. Multi-dimensional collaborative fusion module: used for initial estimation of predictive processing (for example, directly determining the region of interest (ROI) based on predicted behavior) and dynamically adjusting the fusion weights and feature completion strategies of each modality; 3. Feedback-based environmental adaptive enhancement system: Based on the sensitivity of decisions to environmental interference, fine-tune the environment and performance impact model and optimize the compensation strategy.

[0050] In the above technical solution, the basis for dynamic adjustment can be divided into the following parts: (1) Detailed information on the behavior prediction results: such as the distance between the target and the door gap, the probability of the target entering the door gap, the target's predicted trajectory and movement trend.

[0051] (2) Refined dimension of real-time quality scoring (comprehensive confidence): Identify which modality has poor data quality (for example, visual images are blurred due to low lighting, and radar data is noisy due to rain and snow). If the data quality of a key modality is poor, more resources may be required to process feature completion or noise reduction for that modality. In addition, a confidence score is obtained for each detected target. If the confidence score is low, multiple resources are required for verification.

[0052] (3) Specific indicators of environmental parameters: For example, low-light environments such as nighttime and tunnels may require enhanced computing resources for visual processing.

[0053] (4) System resource status: One is CPU / GPU utilization and memory usage. If system resources are close to saturation, a more lightweight processing flow will be adopted or the highest-risk targets will be prioritized. Another is real-time latency requirements: Based on the strict latency requirements of the application scenario, the computing load is dynamically adjusted to ensure timely warnings.

[0054] In the above technical solution, the environmental adaptive enhancement system works in parallel with the core perception and decision-making process, providing real-time environmental status perception and system parameter adaptive adjustment capabilities, including an environmental feature extraction module and a compensation strategy generation module.

[0055] The environmental feature extraction module monitors multi-dimensional environmental parameters in real time: light intensity ,temperature ,humidity , vibration acceleration (obtained by accelerometer), electromagnetic interference intensity (obtained by a special sensor), ambient noise sound pressure level (obtained by microphone). At the same time, the environmental feature extraction module uses wavelet transform to and The time series signal is decomposed at multiple scales to extract high-frequency mutation features (such as abnormal vibration) and low-frequency trend features. Wavelet transform can accurately identify more than 95% of environmental mutation events, allowing the system to respond and adjust 200ms in advance. At the same time, the environmental feature extraction module uses the LSTM network to establish an environmental change prediction model. Estimate the environmental change trend at k moments in the future. For example, predict that the light intensity will drop by 30% in the next 10 seconds. It is an LSTM network specifically designed for environmental change prediction. For Environmental prediction value over time.

[0056] Example 2 A heterogeneous sensing warning method based on decoupled perception and adversarial robust learning, such as Figure 2 As shown, it includes: Collect raw data in real time and obtain decoupled target-related features and environmental interference features through the set multi-layer decoupling encoder training; Based on the target-related features and environmental interference features, feature alignment and fusion are performed through a cross-domain adversarial robustness learning framework to obtain the fused joint representation features. The fused joint representation features are then subjected to multi-level fine semantic recognition to obtain fine semantic recognition results. At the same time, cross-modal generative feature completion is performed to obtain the completed joint representation features. A cognitive-inspired confidence assessment model is constructed based on the results of fine-grained semantic recognition, the completed joint representation features, and the collected historical trajectory information. Real-time quality scoring is performed based on the original data, decoupled target-related features, and decoupled environmental interference features. Graded warnings are issued based on the real-time quality scores and behavior prediction results, and the sensor detection warning results are fed back.

[0057] Example 3 A computer program product includes a computer program, characterized in that when the computer program is executed by a processor, the steps of the method described in Example 2 are implemented.

[0058] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

Claims

1. A heterogeneous sensing warning system based on decoupled perception and adversarial robust learning, characterized by: It includes: The feature extraction module is used to collect the raw sensor data from heterogeneous sensors in real time. Through the training of the set multi-layer decoupling encoder, the decoupled sensor target related features and environmental interference features are obtained. The multi-dimensional collaborative fusion module is used to align and fuse the decoupled sensor target related features through a cross-domain adversarial robust learning framework to obtain the fused joint representation features. The fused joint representation features are then subjected to multi-level fine semantic recognition to obtain fine semantic recognition results. At the same time, cross-modal generative feature completion is performed on the fused joint representation features to obtain the completed joint representation features. The cognitive-enhanced closed-loop decision-making module is used to construct a cognitive-inspired confidence assessment model based on the results of refined semantic recognition, the completed joint representation features, and the historical trajectory information of the sensor target. Based on the cognitive-inspired confidence assessment model, real-time quality scores are performed on the raw sensor data sensed by heterogeneous sensors, the decoupled sensor target-related features, and the environmental interference features. According to the real-time quality scores and the sensor target behavior prediction results, graded warnings are issued for the sensor target behavior.

2. A heterogeneous sensing warning system based on decoupled perception and adversarial robust learning according to claim 1, characterized in that: The multi-layer decoupling encoder set in the feature extraction module specifically includes a visual decoupling encoder for visible light images and close-range infrared images, and a multimodal point cloud and signal decoupling encoder for the original point cloud, original ultrasonic echo signal, and original lidar point cloud. The obtained sensor target related features are divided into visual sensor target related features and point cloud signal sensor target related features, and the obtained environmental interference features are divided into visual environmental interference features and point cloud signal environmental interference features. Among them, the visual decoupling encoder for visible light images and close-range infrared images , a lightweight mobile network structure based on an improved lightweight convolutional neural network, consisting of a shared backbone network and two separate decoupling heads; In its decoupling mechanism, an adversarial decoupling strategy is adopted to realize dynamic game by introducing two discriminators. The two discriminators are respectively a discriminator for receiving the relevant features of the visual sensor target and a discriminator for detecting the target. and a discriminator that receives visual environment interference features , which is used to force the visual sensing target-related features to remain invariant to environmental changes, force the visual environment interference features to capture environmental information, and optimize feature extraction as a whole through the visual loss function, where the specific formula of the visual loss function is: ； ； ； ； ； ； Where, is the reconstruction loss, Decoupled encoder for vision and the discriminator The confrontation loss between For encoder and the discriminator The confrontation loss between is the preliminary target classification loss based on the relevant features of the visual sensing target, is the true label, Losses due to environmental supervision, is the weight coefficient, are the various loss function values ​​in the visual loss function, The total visual loss function, is the original visual image, is the reconstruction loss, For the first fight loss, is the preliminary target classification loss, Losses due to environmental supervision, is the visual sensing target related feature, is the visual environment interference feature, is the visual decoder, is the true label, is the normalized environment data, k is the number of visual loss functions calculated; Multimodal point cloud and signal decoupling encoder for raw point cloud, raw ultrasonic echo signal and raw lidar point cloud , using a customized lightweight network; in its decoupling mechanism, a mutual information minimization strategy is adopted to force the point cloud signal to sense target-related features and point cloud signal environment interference characteristics The statistical independence between them is achieved by optimizing the feature extraction through the point cloud signal loss function. The specific formula of the point cloud signal loss function is: ； ； ； ； Where, is the total loss function of the point cloud signal, is based on The target detection loss is is the true detection label, is the mutual information and measures the degree of dependence between two random variables, is the environmental supervision loss for this modality, are the various loss function values ​​in the point cloud signal loss function, is the weight coefficient, m is the number of loss functions calculated, is the target detection loss based on the relevant features of the sensing target, is mutual information, Losses due to environmental supervision, is the true detection label.

3. A heterogeneous sensing warning system based on decoupled perception and adversarial robust learning according to claim 2, characterized in that: The cross-domain robustness learning framework of the multi-dimensional collaborative fusion module specifically includes: an interference domain generator for generating interference features of simulated environments, a cross-domain discriminator for determining whether a feature is an interfered feature, a spatial alignment module and a deep temporal alignment module for deep feature alignment, and a deep fusion module based on a neural network architecture; Interference Field Generator The specific way to obtain is: train a generator of a generative adversarial network , which uses the real-time environmental parameters provided by the environment adaptive enhancement system and random noise As input, generate simulated environmental interference characteristics , these interference features will be combined with the target related features output by the modal decoupling anti-noise feature extraction module The superposition is fed into the fusion network as the contaminated input for robustness training; Cross-origin discriminator The specific way to obtain is: train a discriminator , whose input is the fused joint feature representation , cross-domain discriminator It is used to distinguish whether the fused joint representation feature is the original sensor data distribution sensed by the heterogeneous sensor or the data distribution after the fusion of the polluted target related features containing the environmental interference features. At the same time, the feature fusion network of the deep fusion module Through adversarial training, we try to fool the discriminator , so that its output Remains invariant under different interference domains.

4. A heterogeneous sensing warning system based on decoupled perception and adversarial robust learning according to claim 3, characterized in that: The specific method of feature alignment and fusion in the multi-dimensional collaborative fusion module through the cross-domain adversarial robustness learning framework is as follows: The spatial alignment module and the deep temporal alignment module align the relevant features of the sensor targets. The spatial alignment module uses an adaptive spatial transformation network and an improved iterative closest point algorithm based on Kalman filtering to correct the spatial differences of the relevant features of different types of sensor targets. The deep temporal alignment module uses an improved dynamic time warping algorithm to correct the temporal differences of the relevant features of different types of sensor targets. The specific method of the adaptive spatial transformation network is as follows: for the visual sensing target related features, the affine transformation parameters in the STN layer No longer fixed, but real-time environmental parameters provided by the environmental adaptive enhancement system and the behavior prediction results obtained by the real-time multi-target behavior prediction network Dynamically generated, the specific calculation formula is: ； The specific method of the improved iterative closest point algorithm based on Kalman filtering is as follows: in the fine registration link, the Kalman filter is used to predict the initial transformation matrix of the current frame , significantly reducing the number of iterations of the iterative closest point method, and using the point cloud signal sensing target-related features to iteratively optimize the iterative closest point method to obtain more accurate registration , the Kalman filter prediction formula is: ； Where, is the state vector at time t, is the state transition matrix, is the state vector at time t-1, is the control input matrix, is the control vector at time t, is the process noise vector at time t; The loss function of the iterative closest point method is: ； Where, is the source point cloud, is the corresponding point of the target point cloud, is the loss function value of the iterative closest point method, is the total number of point pairs successfully matched in the current iteration, is the i-th pair of matching points; The method of deep time series alignment is to use an improved dynamic time warping algorithm. The specific method is: a real-time dynamic time warping algorithm based on a sliding window is used to reduce the computational complexity from , optimized to ,in, is the complexity, W is the sliding window size, M and N represent the lengths of the two time series involved in the alignment, and the sliding window size of the dynamic time warping algorithm The similarity measurement parameters are dynamically adjusted by the environment adaptive enhancement system according to the environment stability and target motion speed; The aligned sensor target-related features are then fused using a deep fusion method. Based on the cross-modal attention mechanism of the neural network architecture, the target-related features of each modality learn the mutual dependencies of different modalities at different spatial and temporal scales, and output the fused joint representation features. In the cross-modal attention mechanism, the similarity scores are normalized into probability distributions, and the specific formula for generating attention weights is: ； Where, is the query matrix, is the bond matrix, is the value matrix, is the dimension of the key, Softmax is the normalized exponential function, is the matrix transpose operator.

5. A heterogeneous sensing warning system based on decoupled perception and adversarial robust learning according to claim 1, characterized in that: The specific method for obtaining fine semantic recognition results in the multi-dimensional collaborative fusion module is as follows: Based on the fused joint representation features, a neural network model for high-precision target detection is used to perform target classification optimization and bounding box regression optimization to identify the specific category of obstacles; The specific method to obtain the completed joint representation features is: When the data quality of a certain modality cannot meet the fusion reliability requirements, the effective information of other modalities is used to supplement the effective features of the modality through conditional generative adversarial networks or diffusion models.

6. A heterogeneous sensing warning system based on decoupled perception and adversarial robust learning according to claim 1, characterized in that: Construct a cognitive heuristic confidence evaluation model. The specific method for real-time quality scoring based on the cognitive heuristic confidence evaluation model is as follows: Perform multimodal consistency modeling and represent the completed joint features , fine semantic recognition results and historical trajectory information of the sensor target Build as a graph ,Using graph neural network to learn the consistency between joint representation features, fine semantic recognition results and sensor target historical trajectory information, similarity in graph neural network The formula is as follows: ； Where, It is a graph neural network; Perform semantic scene correlation analysis and combine it with pre-built high-precision semantic maps ,Evaluate the rationality of detected obstacles and current target positions,status in high-precision semantic maps; Calculating confidence scores : in, is the learnable weight of similarity, is the learnable weight of uncertainty, is the learnable weight of semantic scene matching, is the activation function, Score the semantic scene matching, is the uncertainty of the target position, uncertainty in class identification, is the target motion uncertainty, and the confidence score is the real-time quality score.

7. A heterogeneous sensing warning system based on decoupled perception and adversarial robust learning according to claim 1, characterized in that: It also has a compensation strategy generation module, including an environment performance impact model, a combination of reinforcement learning and meta-learning, and a strategy smooth transition mechanism; Environmental performance impact model: Build a multi-layer perceptron network that takes as input the real-time and predicted environmental parameters obtained by the environmental adaptive enhancement system and outputs the expected performance of each modal sensor and recommended adjustment values ​​for specific system parameters. Combining reinforcement learning and meta-learning: A reinforcement learning agent observes the current state of the environment and system performance, outputs parameter adjustment values, and the reward function is designed to maximize detection accuracy and minimize latency, while penalizing false positives and false negatives. Introducing a meta-learning strategy into reinforcement learning training enables the reinforcement learning agent to quickly learn the optimal compensation strategy from a small amount of new environmental data, thus achieving active environmental exploration and adaptation. Strategy smooth transition mechanism: When a new compensation strategy is generated, the system does not switch immediately, but gradually transitions within a set number of frames through linear interpolation.

8. A heterogeneous sensing warning system based on decoupled perception and adversarial robust learning according to claim 1, characterized in that: The multi-dimensional collaborative fusion module also includes real-time data quality assessment and adaptive adjustment strategies. The real-time data quality assessment method is to build a lightweight evaluation network to perform real-time modality quality scoring on the raw data of each modality and the decoupled target-related features of each frame; The adaptive adjustment strategy is to dynamically adjust the contribution weight and processing priority of each modality in the fusion based on the real-time quality score, the prediction results of the behavior prediction network and the real-time environmental assessment of the environmental adaptive enhancement system. The contribution weight can be calculated in real time by a lightweight multi-layer perceptron according to the real-time modality quality score and the real-time environmental parameters obtained by the environmental adaptive enhancement system.

9. A heterogeneous sensing warning method based on decoupled perception and adversarial robust learning, characterized in that: It includes: Collect raw data in real time and obtain decoupled target-related features and environmental interference features through the set multi-layer decoupling encoder training; Based on the target-related features and environmental interference features, feature alignment and fusion are performed through a cross-domain adversarial robustness learning framework to obtain the fused joint representation features. The fused joint representation features are then subjected to multi-level fine semantic recognition to obtain fine semantic recognition results. At the same time, cross-modal generative feature completion is performed to obtain the completed joint representation features. A cognitive-inspired confidence assessment model is constructed based on the results of fine-grained semantic recognition, the completed joint representation features, and the collected historical trajectory information. Real-time quality scoring is performed based on the original data, decoupled target-related features, and decoupled environmental interference features. Graded warnings are issued based on the real-time quality scores and behavior prediction results, and the sensor detection warning results are fed back.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claim 9 are implemented.

Images