An enhanced recognition method in a multi-source heterogeneous scene

Abstract

The application discloses an enhanced recognition method in a multi-source heterogeneous scene, and relates to the technical field of intelligent sensing, comprising the following steps: S1, dynamic acquisition and preprocessing of multi-source data; S2, dynamic modal weight distribution; S3, cross-modal feature enhancement fusion; S4, lightweight model reasoning and optimization; S5, dynamic environment adaptive learning. The enhanced recognition method in the multi-source heterogeneous scene effectively solves the problems of heterogeneous data time alignment and format unification by synchronously collecting multi-modal data in real time and performing standardization processing. In cooperation with the unique dynamic modal weight distribution mechanism, the reliability of each modal data can be evaluated in real time according to the scene change, and the fusion weight can be dynamically adjusted, so as to further ensure the accuracy and adaptability of data fusion. In addition, more discriminative joint features are generated through cross-modal feature enhancement fusion, which significantly improves the recognition accuracy and robustness.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensing technology, specifically to an enhanced recognition method for multi-source heterogeneous scenarios. Background Technology

[0002] With the deep integration of artificial intelligence and Internet of Things (IoT) technologies, multi-source heterogeneous data recognition technology has become an important research direction in the field of intelligent sensing. This technology integrates heterogeneous data from different sensors and modalities (such as images, text, radar, and LiDAR) to achieve comprehensive perception and accurate recognition of complex scenes, demonstrating significant application value in scenarios such as autonomous driving, intelligent security, and industrial inspection. However, existing technologies still face many challenges in recognition tasks under multi-source heterogeneous scenarios.

[0003] Current mainstream recognition technologies mainly fall into three categories: single-modal recognition, traditional fusion methods, and deep learning fusion. Single-modal methods, such as CNN-based image recognition or point cloud processing, perform well in specific scenarios, but rely on a single data source and cannot fully utilize complementary information from multiple sources, resulting in limited recognition accuracy and difficulty in handling complex environments. Traditional fusion methods achieve information integration through data-level, feature-level, or decision-level fusion, but they struggle to adaptively adjust the weights of different modalities and are sensitive to missing data or noise, exhibiting low fusion efficiency and poor adaptability.

[0004] While deep learning fusion methods offer superior performance, existing deep learning models fall short in handling spatiotemporal alignment of heterogeneous data and data heterogeneity (such as differences in sampling rate and resolution). Furthermore, these methods also face challenges such as high computational complexity and insufficient real-time performance.

[0005] Therefore, in practical applications such as smart cities and intelligent manufacturing, there is an urgent need for an enhanced recognition method that can effectively integrate multi-source data, adapt to dynamic environments, and is computationally efficient. Summary of the Invention

[0006] The purpose of this invention is to provide an enhanced recognition method for multi-source heterogeneous scenarios to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an enhanced recognition method for multi-source heterogeneous scenarios, comprising the following steps: S1. Dynamic acquisition and preprocessing of multi-source data: Real-time synchronous acquisition of multimodal data and standardized processing to achieve time alignment and format unification of heterogeneous data; S2, Dynamic Modal Weight Allocation: Real-time evaluation of the reliability of each modal data based on scene changes, and dynamic adjustment of fusion weights; S3, Cross-modal Feature Enhancement Fusion: Mining complementary information among multimodal data to generate more discriminative joint features; S4. Lightweight Model Inference and Optimization: By means of model compression, quantization acceleration and hardware adaptation, the computational complexity and memory consumption of the model are reduced, while the recognition accuracy of the model is maintained or improved as much as possible, and the model can achieve efficient inference on edge devices (such as NVIDIA Jetson series, ARM CPU, etc.). S5. Dynamic Environment Adaptive Learning: It retains knowledge from old tasks by using incremental learning, identifies outlier data based on anomaly detection, and dynamically updates weights or triggers model fine-tuning through feedback mechanisms to adapt to environmental changes.

[0008] Furthermore, in step S1, the standardization process includes the following operations: Time synchronization: Hardware-level synchronization (such as the PTP protocol) or software interpolation algorithms are used to align the sampling timestamps of different sensors to millisecond-level accuracy; among them, hardware synchronization: the synchronous generation of sensor trigger signals is implemented through FPGA; software interpolation: cubic spline interpolation is used to align the timing data. Data cleaning: noise removal (e.g., Kalman filtering to smooth radar data), filling in missing values ​​(e.g., image inpainting based on neighborhood interpolation); Format standardization: Using tools such as OpenCV (image) and PCL (point cloud), point cloud, image and other data are uniformly converted into tensor format for easy subsequent processing.

[0009] Furthermore, in step S2, the reliability of each modality data is quantified by calculating its confidence level, specifically including the confidence level calculation of visual images, radar signals, text data, and LiDAR point clouds.

[0010] Furthermore, the specific method for calculating the confidence level of the visual image is as follows: Step 1, Sharpness Assessment: Extract image quality features using a lightweight CNN (such as MobileNet) and calculate a sharpness score. The sharpness score can be based on indicators such as edge strength, contrast, or texture information of the image. ; in, Rate the clarity. and The images are respectively in and Gradient of direction, Number of image patches; Step 2, Noise Level Assessment: Quantify noise intensity using image gradient or frequency domain analysis (such as Fourier transform); ; in, To score the noise level, For image frequency domain representation, This is the frequency domain representation after low-pass filtering. The number of frequency domain points; Step 3, Overall Confidence Level: ; in, For visual image confidence, Rate the clarity. To score the noise level, and These are weighting coefficients and need to be adjusted according to the scenario.

[0011] Furthermore, the specific calculation method for the radar signal confidence level is as follows: Step 1, Signal-to-Noise Ratio (SNR) Evaluation: Calculate the signal-to-noise ratio of the radar signal, which is the ratio of signal power to noise power, to evaluate its quality. The signal-to-noise ratio reflects the relative strength of effective information and noise in the signal. in, The signal-to-noise ratio is scored. and These represent the power of the signal and the power of the noise, respectively. Step 2, Doppler frequency shift stability assessment: Analyze the variance of the target velocity estimate; ; in, To score the stability of the Doppler frequency shift, The variance of the velocity estimate; Step 3, Overall Confidence Level: ; in, For radar signal confidence level, The signal-to-noise ratio is scored. To score the stability of the Doppler frequency shift, and These are the weighting coefficients.

[0012] Furthermore, the specific method for calculating the confidence level of the text data is as follows: Step 1, Language Model Scoring: Use models such as BERT to calculate the semantic coherence and grammatical correctness of the text; ; in, Score the language model. For language model prediction The probability of each word The length of the text; Step 2, Keyword Matching: Analyze the frequency of scene-related keywords in the text; ; in, Score the keyword matching degree; Step 3, Overall Confidence Level: ; in, Confidence level for text data Score the language model. Score the keyword matching degree. and These are the weighting coefficients.

[0013] Furthermore, the specific calculation method for the LiDAR point cloud confidence score is as follows: Step 1, Point Cloud Density: Calculate the number of points per unit volume; ; in, Point cloud density, For points, The volume occupied by the point cloud; Step 2, Uniformity of reflection intensity: Analyze the variance of reflection intensity in the point cloud; ; in, For uniformity of reflection intensity, The variance of the reflection intensity; Step 3, Overall Confidence Level: in, Confidence of LiDAR point clouds Point cloud density, For uniformity of reflection intensity, and These are the weighting coefficients.

[0014] Furthermore, in step S2, the weights are dynamically calculated using the entropy method or the attention mechanism, and then normalized to ensure that the sum of all modal weights is 1, thus avoiding fusion bias. Entropy method: ; in, For the first The weights of each modality For the first Information entropy of a modality For the k-th sample in the th... The confidence percentage of each modality; Attention mechanism: ; in, For the first The weights of each modality Its confidence level.

[0015] Furthermore, step S3 includes the following sub-steps: S31. Feature Extraction: Use modality-specific networks (such as ResNet-50 for image processing and PointNet for point cloud processing) to extract high-level semantic features; S32, Feature Alignment: Mapping features of different modalities to a shared semantic space using a projection matrix (e.g., using t-SNE dimensionality reduction to visualize the alignment effect). S33. Feature Fusion: Use weighted splicing or graph neural networks (GNN) to aggregate cross-modal information, such as modeling intermodal relationship graphs using GNN.

[0016] Furthermore, the specific operation of step S4 is as follows: Model compression: This involves reducing the number of parameters using knowledge distillation (e.g., the Teacher-Student model) or channel pruning (e.g., the NetAdapt algorithm). Knowledge distillation utilizes a large, pre-trained teacher model (e.g., ResNet-101) to guide the learning of a small, untrained student model (e.g., MobileNetV3). By mimicking the output of the teacher model, the student model can significantly reduce the number of parameters and computational cost while maintaining high accuracy. Channel pruning uses algorithms such as NetAdapt to automatically identify and remove unimportant channels or layers from the model, thereby reducing model complexity. The pruning process is typically evaluated based on the contribution of each channel to the model output.

[0017] Quantization acceleration: Converting floating-point operations to 8-bit integer operations (INT8 quantization) improves inference speed by 3-4 times. Specifically, quantization tools such as PyTorch Quantization or TensorFlow Lite are used to implement mixed-precision quantization, which can automatically handle precision loss during the quantization process.

[0018] Hardware adaptation: Optimize the model's computation graph for the characteristics of the target edge platform (such as the GPU architecture of NVIDIA Jetson or the instruction set of ARM CPUs). For example, use the TensorRT engine to optimize model inference on NVIDIA Jetson, and accelerate the execution of key operators through CUDA kernel fusion or the NEON instruction set. Ensure the efficiency of model deployment and operation on edge devices, including optimizations in memory management, thread scheduling, and power consumption control.

[0019] Furthermore, in step S5, incremental learning employs the Elastic Weight Consolidation (EWC) algorithm to retain knowledge from previous tasks and avoid catastrophic forgetting.

[0020] Anomaly detection: Identify outlier data based on reconstruction error (such as autoencoders) or statistical methods (such as the 3σ principle), specifically using Isolation Forest or One-Class SVM to detect outliers.

[0021] Feedback and Adjustment: Dynamically update modal weights or trigger model fine-tuning based on recognition results, such as updating weights every 100 frames, and achieving real-time parameter updates through ROS (Robot Operating System). This invention provides an enhanced recognition method for multi-source heterogeneous scenarios, which has the following beneficial effects: 1. This invention effectively solves the problems of time alignment and format unification of heterogeneous data by real-time synchronous acquisition of multimodal data and standardized processing. Combined with its unique dynamic modality weight allocation mechanism, it can assess the reliability of each modality's data in real time according to scene changes and dynamically adjust the fusion weights, thereby further ensuring the accuracy and adaptability of data fusion. Furthermore, through cross-modal feature enhancement fusion, it further mines complementary information between multimodal data, generating more discriminative joint features, significantly improving the accuracy and robustness of recognition.

[0022] 2. This invention achieves lightweight model design through model compression, quantization acceleration, and hardware adaptation, thereby enabling efficient inference on edge devices. This not only reduces the model's computational complexity and memory footprint but also maintains or improves the model's recognition accuracy, meeting the needs of applications with high real-time requirements. Simultaneously, the dynamic environment adaptive learning mechanism utilizes incremental learning to retain knowledge from previous tasks and identifies outlier data based on anomaly detection. Through a feedback mechanism, it dynamically updates weights or triggers model fine-tuning, enabling the method to quickly adapt to environmental changes and maintain long-term stable operation. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating the steps of an enhanced recognition method for multi-source heterogeneous scenarios according to the present invention. Detailed Implementation

[0024] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0025] Example 1: Target Detection and Recognition in Autonomous Driving Scenarios In autonomous driving scenarios, vehicles need to perceive their surroundings in real time, including other vehicles, pedestrians, road signs, etc. This embodiment utilizes multi-source heterogeneous data (such as camera images, LiDAR point clouds, and radar signals) for target detection and recognition.

[0026] Step S1: Dynamic acquisition and preprocessing of multi-source data: The system utilizes a front-facing camera (1080P resolution), a 32-line LiDAR, and a forward-facing millimeter-wave radar to simultaneously acquire camera images, LiDAR point clouds, and radar signals in real time. The camera captures a single frame of RGB image with a resolution of 1920x1080; the LiDAR captures point cloud data containing 10,000 points; and the radar captures distance, velocity, and angle information for five targets.

[0027] The sampling timestamps of the camera, LiDAR, and radar were aligned to millisecond-level precision using the PTP protocol. Then, data cleaning was performed: image denoising (e.g., Gaussian filtering), outlier removal from the LiDAR point cloud, and smoothing filtering of the radar data. Finally, format normalization was performed: images were converted to tensor format (1x3x1080x1920), LiDAR point clouds to tensor format (1x10000x3), and radar data to tensor format (1x5x3).

[0028] Step S2, Dynamic Modal Weight Allocation: First, calculate the confidence level of each modality: Visual image confidence score: Sharpness and noise level are evaluated using a lightweight CNN to calculate the overall confidence score. In this embodiment, the sharpness score is 0.9 (based on edge strength calculation), and the noise level score is 0.85 (based on frequency domain analysis). The weighting coefficients for sharpness score and noise level are 0.6 and 0.4, respectively. Therefore, the overall confidence score is calculated as: 0.6 * 0.9 + 0.4 * 0.85 = 0.875.

[0029] LiDAR point cloud confidence score: The confidence score is calculated based on point cloud density and reflection intensity uniformity. In this embodiment, point cloud density = 0.92 (a large number of points per unit volume), reflection intensity uniformity = 0.88 (a small variance), and the weighting coefficients for point cloud density and reflection intensity uniformity are 0.7 and 0.3, respectively. Therefore, the overall confidence score is calculated as: 0.7 * 0.92 + 0.3 * 0.88 = 0.9.

[0030] Radar signal confidence level: Confidence level is assessed based on signal-to-noise ratio (SNR) and Doppler frequency shift stability. In this embodiment, the SNR score is 0.95 (high signal quality), and the Doppler frequency shift stability score is 0.9 (small velocity estimation variance). The weighting coefficients for the SNR score and the Doppler frequency shift stability score are 0.6 and 0.4, respectively. Therefore, the overall confidence level is calculated as: 0.6 * 0.95 + 0.4 * 0.9 = 0.925.

[0031] Then, dynamic weight calculation: the weights of each modality are calculated using the entropy method, with visual image weight = 0.3, LiDAR point cloud weight = 0.4, and radar signal weight = 0.3.

[0032] Step S3, Cross-modal feature enhancement and fusion: Feature extraction: Image features (2048 dimensions) were extracted using ResNet-50, point cloud features (1024 dimensions) were extracted using PointNet, and features (128 dimensions) were obtained by simple processing of radar data.

[0033] Feature alignment: Different modal features are mapped to a shared semantic space (512 dimensions) through a projection matrix.

[0034] Feature fusion: Features are fused using a weighted concatenation method to obtain joint features (512 dimensions).

[0035] Step S4, Lightweight Model Inference and Optimization: Model compression: The channel pruning algorithm is used to reduce the number of parameters in ResNet-50 and PointNet, and the model size is reduced by 50% after pruning.

[0036] Quantization acceleration: Converts floating-point operations to 8-bit integer operations, increasing inference speed by 3 times.

[0037] Hardware adaptation: The model computation graph is optimized for the NVIDIA Jetson AGX Xavier platform, and the TensorRT engine is used to accelerate inference.

[0038] Step S5, Adaptive Learning in Dynamic Environment: The EWC algorithm is used to retain knowledge from old tasks and avoid catastrophic forgetting.

[0039] Based on the reconstruction error, abnormal data is identified (outliers are detected using an autoencoder). The modal weights are then dynamically updated or the model is fine-tuned accordingly. In practice, the modal weights are updated every 100 frames, and real-time parameter updates are achieved through ROS.

[0040] Implementation results: 1. In actual road tests, this method significantly improves the accuracy and robustness of target detection, especially under adverse weather conditions (such as rain and fog) or sudden changes in lighting conditions, by dynamically adjusting the modal weights and effectively utilizing the advantages of each modal data.

[0041] 2. The model achieves an inference speed of over 30 FPS on NVIDIA Jetson AGX Xavier, meeting real-time requirements.

[0042] 3. Compared with the single-modal method, the mAP (mean accuracy) of target detection is improved by about 15%, and the false detection rate and false negative rate are significantly reduced.

[0043] Example 2: Abnormal Behavior Detection in Intelligent Security Scenarios In the field of intelligent security, it is necessary to monitor abnormal behaviors in video surveillance in real time, such as trespassing into restricted areas or leaving items behind. This embodiment combines camera images, audio signals, and sensor data (such as infrared sensors) to detect abnormal behaviors.

[0044] Step S1: Dynamic acquisition and preprocessing of multi-source data: The system uses a high-definition camera (720P resolution), an infrared sensor, and an alarm system (text message) to synchronously collect video images, audio signals, and infrared sensor data in real time. The camera captures a frame of RGB image with a resolution of 1280x720; the infrared sensor captures two-dimensional data containing 10 hotspots; and the alarm system receives a text message containing the keyword "intrusion".

[0045] Use software interpolation algorithms to align timestamps and ensure data time synchronization.

[0046] The image is denoised and enhanced, the audio signal is denoised and its features are extracted, and the infrared sensor data is thresholded.

[0047] The sampling timestamps of the camera, infrared sensor, and alarm system are aligned to millisecond-level precision using a software interpolation algorithm. Then, data cleaning is performed: median filtering is used to denoise the images, thresholding is applied to the infrared data to remove background noise, and text information is segmented and stop-word removed. Finally, format standardization is applied: images are converted to tensor format (1x3x720x1280), infrared data is converted to tensor format (1x10x2), and text information is converted to tensor format (1x10, using word embedding).

[0048] Step S2, Dynamic Modal Weight Allocation: Calculate the confidence level of each modality: Visual image confidence score: Sharpness score = 0.8 (based on contrast calculation), noise level score = 0.75 (based on gradient analysis). In this embodiment, the weighting coefficients for sharpness and noise level are 0.6 and 0.4, respectively. Therefore, the overall confidence score is calculated to be 0.775.

[0049] Infrared sensor confidence score: Hot spot quantity score = 0.9 (many hot spots detected), temperature stability score = 0.85 (small temperature change). In this embodiment, the weighting coefficients for hot spot quantity and temperature stability are 0.7 and 0.3, respectively. Therefore, the overall confidence score is calculated to be 0.875.

[0050] Text data confidence: Language model score = 0.9 (semantic coherence calculated using BERT model), keyword matching score = 1 (includes the keyword "intrusion"). In this embodiment, the weight coefficients of language model score and keyword matching score are 0.5 and 0.5, respectively. Therefore, the overall confidence score is calculated to be 0.95.

[0051] Dynamic weight calculation: The attention mechanism is used to calculate the weights of each modality. The weight of visual image is 0.2, the weight of infrared sensor is 0.3, and the weight of text data is 0.5.

[0052] Step S3, Cross-modal feature enhancement and fusion: Feature extraction: Image features (512 dimensions) were extracted using MobileNet, infrared data were processed to obtain features (64 dimensions), and text information was embedded to obtain features (300 dimensions).

[0053] Feature alignment: Mapping features of different modalities to a shared semantic space (256 dimensions) using a projection matrix.

[0054] Feature fusion: A graph neural network (GNN) is used to aggregate cross-modal information to obtain joint features (256 dimensions). Step S4, Lightweight Model Inference and Optimization: Model compression: The knowledge distillation algorithm is used to reduce the number of parameters in MobileNet, and the model size is reduced by 60% after distillation.

[0055] Quantization acceleration: Converts floating-point operations to 8-bit integer operations, increasing inference speed by 4 times.

[0056] Hardware adaptation: The model computation graph is optimized for the ARM CPU platform, and the NEON instruction set is used to accelerate the execution of key operators.

[0057] Step S5, Adaptive Learning in Dynamic Environment: Incremental learning is used to retain knowledge from old scenarios and avoid degrading model performance.

[0058] Abnormal data is identified using statistical methods (such as the 3σ principle), modal weights are dynamically updated based on the identification results, and real-time parameter updates are achieved through ROS.

[0059] Implementation results: 1. In actual security monitoring scenarios, this method effectively improves the accuracy and timeliness of abnormal behavior detection, and reduces false alarms and missed alarms.

[0060] 2. The model's inference speed meets the requirements for real-time monitoring, and its resource consumption is low.

[0061] 3. Compared with traditional single-modal methods, the recall and precision of abnormal behavior detection are improved by about 10%, and the overall system performance is stable and reliable.

[0062] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. An enhanced recognition method for multi-source heterogeneous scenarios, characterized in that, Includes the following steps: S1. Dynamic acquisition and preprocessing of multi-source data: Real-time synchronous acquisition of multimodal data and standardized processing to achieve time alignment and format unification of heterogeneous data; S2, Dynamic Modal Weight Allocation: Real-time evaluation of the reliability of each modal data based on scene changes, and dynamic adjustment of fusion weights; S3. Cross-modal feature enhancement and fusion: Mining complementary information among multimodal data to generate joint features; S4, Lightweight Model Inference and Optimization: Achieve efficient model inference on edge devices through model compression, quantization acceleration, and hardware adaptation.

2. The enhanced recognition method for multi-source heterogeneous scenarios according to claim 1, characterized in that, In step S1, the standardization process includes the following operations: Time synchronization: Hardware-level synchronization or software interpolation algorithms are used to align the sampling timestamps of different sensors to millisecond-level accuracy; Data cleaning: removing noise and filling in missing values; Format standardization: Convert point cloud and image data into tensor format in a unified manner.

3. The enhanced recognition method for multi-source heterogeneous scenarios according to claim 1, characterized in that, In step S2, the reliability of each modality data is quantified by calculating its confidence level, specifically including the confidence level calculation of visual images, radar signals, text data, and LiDAR point clouds.

4. The enhanced recognition method in a multi-source heterogeneous scenario according to claim 3, characterized in that, The specific method for calculating the confidence level of the visual image is as follows: Step 1, Sharpness Assessment: Extract image quality features using a lightweight CNN and calculate a sharpness score; ； in, Rate the clarity. and The images are respectively in and Gradient of direction, Number of image patches; Step 2, Noise Level Assessment: Quantify noise intensity using image gradient or frequency domain analysis; ； in, To score the noise level, For image frequency domain representation, This is the frequency domain representation after low-pass filtering. The number of frequency domain points; Step 3, Overall Confidence Level: ； in, For visual image confidence, Rate the clarity. To score the noise level, and These are the weighting coefficients.

5. The enhanced recognition method in a multi-source heterogeneous scenario according to claim 3, characterized in that, The specific calculation method for the radar signal confidence level is as follows: Step 1, Signal-to-Noise Ratio Assessment: Calculate the signal-to-noise ratio of the radar signal, which is the ratio of signal power to noise power, to assess its quality; in, The signal-to-noise ratio is scored. and These represent the power of the signal and the power of the noise, respectively. Step 2, Doppler frequency shift stability assessment: Analyze the variance of the target velocity estimate; ； in, To score the stability of the Doppler frequency shift, The variance of the velocity estimate; Step 3, Overall Confidence Level: ； in, For radar signal confidence level, The signal-to-noise ratio is scored. To score the stability of the Doppler frequency shift, and These are the weighting coefficients.

6. The enhanced recognition method in a multi-source heterogeneous scenario according to claim 3, characterized in that, The specific method for calculating the confidence level of the text data is as follows: Step 1, Language Model Scoring: Use the BERT model to calculate the semantic coherence and grammatical correctness of the text; ； in, Score the language model. For language model prediction The probability of each word The length of the text; Step 2, Keyword Matching: Analyze the frequency of scene-related keywords in the text; ； in, Score the keyword matching degree; Step 3, Overall Confidence Level: ； in, Confidence level for text data Score the language model. Score the keyword matching degree. and These are the weighting coefficients.

7. The enhanced recognition method for multi-source heterogeneous scenarios according to claim 3, characterized in that, The specific method for calculating the confidence level of LiDAR point clouds is as follows: Step 1, Point Cloud Density: Calculate the number of points per unit volume; ； in, Point cloud density, For points, The volume occupied by the point cloud; Step 2, Uniformity of reflection intensity: Analyze the variance of reflection intensity in the point cloud; ； in, For uniformity of reflection intensity, The variance of the reflection intensity; Step 3, Overall Confidence Level: in, Confidence of LiDAR point clouds Point cloud density, For uniformity of reflection intensity, and These are the weighting coefficients.

8. The enhanced recognition method in a multi-source heterogeneous scenario according to claim 3, characterized in that, In step S2, the weights are dynamically calculated using the entropy method or attention mechanism and then normalized, so that the sum of all modal weights is 1. Entropy method: ; in, For the first The weights of each modality For the first Information entropy of a modality For the k-th sample in the th... The confidence percentage of each modality; Attention mechanism: ; in, For the first The weights of each modality Its confidence level.

9. The enhanced recognition method in a multi-source heterogeneous scenario according to claim 1, characterized in that, Step S3 includes the following sub-steps: S31. Feature Extraction: High-level semantic features are extracted using a modality-specific network. S32. Feature Alignment: Mapping features of different modalities to a shared semantic space using a projection matrix; S33. Feature fusion: Employ weighted splicing or graph neural networks to aggregate cross-modal information.

10. The enhanced recognition method in a multi-source heterogeneous scenario according to claim 1, characterized in that, It also includes the following steps: S5. Dynamic Environment Adaptive Learning: It retains knowledge of old tasks by using incremental learning, identifies outlier data based on anomaly detection, and dynamically updates weights or triggers model fine-tuning through a feedback mechanism.

Images