Underground anti-interference unmanned vehicle multi-sensor fusion method and system

Abstract

The invention discloses an underground anti-interference unmanned vehicle multi-sensor fusion method and system. Comprising the following steps: acquiring multi-sensor original data in an unmanned vehicle operation environment, and performing time synchronization on the multi-sensor original data; reconstructing the synchronized data of each multi-sensor, and removing noise components to obtain reconstructed sensor data; deep features of the reconstructed sensor data are extracted; the deep-level features are input into a self-attention module, weights input by different sensors are dynamically adjusted, and fused feature representation is generated; performing graph optimization based on the fused feature representation to obtain a Laplacian matrix; and establishing a joint optimization function through multi-sensor original data, fused feature representation and a Laplacian matrix, solving by adopting an alternating direction multiplier method, and carrying out iterative updating until convergence to obtain a fusion result. And the anti-interference capability, the data processing efficiency and the decision accuracy of the mine unmanned vehicle in a complex environment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source information fusion, and in particular to a multi-sensor fusion method and system for an anti-interference unmanned vehicle in a mine. Background Art

[0002] In recent years, the rapid development of algorithms has promoted the application of neural networks in the field of multi-sensor data fusion. However, these methods are highly dependent on a large amount of labeled data for model training. At the same time, complex conditions such as high dust, low light and weak texture in the mine environment lead to significant noise interference in the sensor data, which further causes missing or incorrect labels. For example, the disclosed patent CN119380315A proposes a multi-sensor data fusion method based on a deep neural network. The deep neural network extracts features to construct a graph structure and uses graph optimization technology to obtain the global optimal fusion result. Although it improves robustness and accuracy, it has high computing requirements and a strong dependence on labeled data. Or, for example, the disclosed patent CN117333406A proposes a multi-sensor data fusion method in mines based on dynamic weighted fusion and factor graph optimization model. Although it can dynamically adjust the sensor weights according to the real-time data quality, it has high algorithm complexity, large computing resource requirements, and insufficient system stability in extreme environments.

[0003] Therefore, it is necessary to study an interference-resistant multi-sensor fusion method and system for unmanned vehicles in mines, which can effectively improve the stability, robustness and anti-interference ability of multi-source sensor data fusion with a small amount of training data, low-complexity algorithms and extreme environments, so as to meet the needs of unmanned vehicles for accurate perception and real-time recognition of complex environments. Summary of the Invention

[0004] The present invention aims to provide an interference-resistant multi-sensor fusion method and system for unmanned vehicles in mines. This approach overcomes the existing limitations of the existing technology, which rely heavily on large amounts of labeled data for model training and fail to fully account for the missing or erroneous data labels caused by extreme mine environments, such as high dust levels, low light levels, and strong electromagnetic interference. The goal is to further improve the stability, robustness, and interference resistance of multi-sensor data fusion in the complex environment of mines, using minimal labeled data and low-complexity algorithms.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a multi-sensor fusion method for an unmanned vehicle in a mine with anti-interference, comprising the following steps:

[0007] S1, obtains the raw data of multiple sensors in the operating environment of the unmanned vehicle and synchronizes the time of the raw data of multiple sensors;

[0008] S2, reconstruct each synchronized multi-sensor data Xi and remove the noise component to obtain the corresponding reconstructed sensor data Extract individual reconstructed sensor data The deep features of Hi;

[0009] S3, the deep feature H i Input into the self-attention module, dynamically adjust the weights of different sensor inputs, and generate the fused feature representation H fused ;

[0010] S4, based on the fused feature representation H fused Perform graph optimization to obtain the Laplace matrix L f ;

[0011] S5, through the multi-sensor raw data, the fused feature representation H fused and the Laplace matrix L f A joint optimization function is established, and the alternating direction multiplier method is used to solve it. It is iterated and updated until convergence to obtain the fusion result.

[0012] Furthermore, in S1, before the multi-sensor raw data enters the acquisition terminal, the multi-sensor raw data is first saved in the hardware register and software array vector; the multi-sensor raw data is coordinated and synchronized by combining hardware and software to synchronize the time of the multi-sensor raw data.

[0013] Furthermore, the specific process in S3 is:

[0014] The deep features H of each sensor i Indicated as H i ∈R d , where d is the feature dimension, all the sensors’ deep features H i Splicing to get the input matrix H∈R N×D , where N is the number of sensors;

[0015] Using three independent linear transformation matrices W Q ,W K ,W V Transform the input matrix H separately to obtain the query matrix Q, key matrix K and value matrix V:

[0016] Q=HW Q ;

[0017] K=HW K ;

[0018] V=HW v ;

[0019] Calculate the dot product similarity between the query matrix Q and the key matrix K to obtain the unnormalized attention score matrix: A′=QK T ;

[0020] Scale the dot product result A′ and apply the Softmax function to obtain the normalized attention score matrix:

[0021] According to the normalized attention score matrix A, the value matrix V is weighted and summed to obtain the final fusion feature representation H fused .

[0022] Furthermore, the specific process in S4 is:

[0023] The fused feature representation H fused Construct an adjacency matrix;

[0024] Construct a directed graph based on the adjacency matrix using the K-nearest neighbor algorithm;

[0025] The distance between two nodes of the directed graph is converted into weight W through the Gaussian kernel function f ;

[0026] Weight W on the edges of the adjacency matrix f Distribution, by formula: L f =DW f Calculate the graph Laplacian matrix L f , where D is the diagonal matrix.

[0027] Furthermore, the joint optimization function in S5 is:

[0028]

[0029] Among them, the first term of the formula is the reconstruction loss term, λ1, λ2, and λ3 are the weight parameters of the control graph regularization term, the consistency constraint term between the pseudo label and the real label, and the label consistency term, respectively. J represents the joint optimization objective function, and x i is the raw data collected by the i-th sensor, H fusedi is the i-th data after sensor data fusion, P i is a pseudo label, y i is the true label, and M is the classification weight matrix.

[0030] Furthermore, the process of solving the joint optimization function using the alternating direction multiplier method is as follows:

[0031] Convert the problem of solving the joint optimization function into multiple sub-problems that can be processed in parallel;

[0032] Iteratively update the pseudo labels and classification weight matrix of each sub-problem;

[0033] At the end of each iteration, the Lagrange multiplier Λ and penalty parameter ρ of each subproblem are updated to obtain the current solution; the update rule is expressed as: Λ (k+1) =Λ (k) +ρ(AX (k+1) -BY (k+1) -c), where X (k+1) and Y (k+1) are the new values of the pseudo-label and classification weight matrix in the current iteration step, respectively. A, B, and c are the defined linear constraints;

[0034] The preset convergence criterion is used to determine whether the current solution reaches the preset threshold. If so, the iteration is terminated; otherwise, the iteration continues.

[0035] Furthermore, in S5, noise and / or missing labels are corrected through label propagation.

[0036] Furthermore, the multi-sensor data includes millimeter-wave radar data, visual data, IMU data, vehicle-mounted navigation data, vehicle-mounted OBU terminal data, and vehicle-mounted sensor data.

[0037] In a second aspect, the present invention provides an anti-interference multi-sensor fusion system for an unmanned vehicle in a mine, which is used to implement the above method, including:

[0038] The first unit is to obtain the raw data of multiple sensors in the operating environment of the unmanned vehicle and synchronize the time of the raw data of multiple sensors;

[0039] The second unit reconstructs each synchronized multi-sensor data Xi and removes the noise component to obtain the corresponding reconstructed sensor data. Extract individual reconstructed sensor data The deep features of Hi;

[0040] Unit 3: Deep feature H i Input into the self-attention module, dynamically adjust the weights of different sensor inputs, and generate the fused feature representation H fused ;

[0041] Unit 4: Based on the fused feature representation H fused Perform graph optimization to obtain the Laplace matrix L f ;

[0042] Unit 5, through the multi-sensor raw data, fused feature representation H fused and the Laplace matrix L f A joint optimization function is established, and the alternating direction multiplier method is used to solve it. It is iterated and updated until convergence to obtain the fusion result.

[0043] In a third aspect, the present invention provides an electronic device, comprising:

[0044] processor;

[0045] a memory for storing processor-executable instructions;

[0046] The processor is configured to call the instructions stored in the memory to execute the method described in claim 1 .

[0047] Compared with the prior art, the present invention has the following beneficial technical effects:

[0048] The present invention provides an interference-resistant multi-sensor fusion method for unmanned vehicles in mines, which performs time synchronization on the raw data of multiple sensors, ensures that data from different sensors can be synchronously collected with high precision, and improves the basic accuracy of data fusion. The deep-level features are extracted so that the model can more accurately capture the essential characteristics of multi-sensor data, enhancing the accuracy of decision-making and planning. The self-attention mechanism is introduced to dynamically adjust the weights of different sensor inputs to generate a more accurate fusion feature representation, thereby improving the effect of multi-sensor information fusion and the robustness and adaptability of the system. The multi-sensor fusion problem is modeled as a graph optimization problem, and the alternating direction multiplier method is used to solve the non-convex optimization problem, which reduces the computational complexity and improves the fusion quality and robustness. The present invention significantly improves the anti-interference ability, data processing efficiency and decision-making accuracy of mine unmanned vehicles in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The figure is a logical flow diagram of a multi-sensor fusion method for an unmanned vehicle with anti-interference in a mine according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0051] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0052] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0053] Figure 1 A flowchart of a multi-sensor fusion method for an unmanned vehicle with anti-interference in a mine according to an embodiment of the present invention is shown as an example. The method includes the following steps:

[0054] S1, obtains the raw data of multiple sensors in the operating environment of the unmanned vehicle and synchronizes the time of the raw data of multiple sensors;

[0055] Before each sensor data enters the acquisition terminal, the data of each sensor is first saved in the hardware register and software array vector. The multi-sensor raw data is coordinated and synchronized by combining hardware and software to synchronize the time of the multi-sensor raw data.

[0056] Among them, the multi-sensor data in the mine unmanned vehicle driving environment includes millimeter wave radar data, visual data, IMU data, vehicle navigation data, vehicle OBU terminal data, and vehicle sensor data;

[0057] Onboard navigation data includes: longitude, latitude, altitude and timestamp;

[0058] Millimeter-wave radar data includes: angle, distance, speed, and timestamp;

[0059] Visual data includes: current frame image information and timestamp;

[0060] IMU data includes: acceleration, gyroscope data, and magnetometer data;

[0061] On-board sensor data includes steering wheel angle, acceleration and braking, which are used to obtain real-time driving status information of the intelligent unmanned vehicle.

[0062] Exemplarily, the time synchronization process for multi-sensor raw data specifically includes:

[0063] First, determine a unified clock source, selecting PTP (Precision Time Protocol) as the unified clock source. Configure the onboard computer and each sensor to use PTP for time synchronization. Set the appropriate trigger mode for each sensor. For sensors that support hardware triggering, use the hardware trigger signal provided by PTP for synchronous data collection. For sensors that do not support hardware triggering, parse the PTP time synchronization signal to correct the received data collection time at the software level.

[0064] In this embodiment, a specialized buffer structure is designed to allow hardware registers to temporarily store data from various sensors while also performing preliminary processing on the received data, including format conversion and noise reduction. Using hardware registers to store each sensor's data and its corresponding timestamp ensures real-time and complete data acquisition. By transferring this data into a software array vector and sorting it by timestamp, data points with significant timestamp errors are interpolated or discarded, ensuring the accuracy of multi-sensor raw data acquisition.

[0065] S2, reconstruct each synchronized multi-sensor data Xi and remove the noise component to obtain the corresponding reconstructed sensor data Extract individual reconstructed sensor data The deep features of Hi;

[0066] For example, an independent variational autoencoder (VAE) is used to reconstruct each synchronized multi-sensor data Xi to remove noise components. The specific process of reconstruction and noise removal is as follows:

[0067] VAE consists of an encoder and a decoder. The encoder maps the input data to a potential representation in the latent space, and the decoder outputs the reconstructed data from the potential representation. Among them, each synchronized multi-sensor data Xi is used as input data.

[0068] Define the network structure: Encoder: Contains multiple fully connected layers or convolutional layers, which are used to map each synchronized multi-sensor data Xi to a potential representation z in the latent space; the encoder outputs two values: the mean μ and variance σ of the latent variable z 2 ;Decoder: also contains multiple fully connected layers or convolutional layers, which are used to output reconstructed data from the potential representation z;

[0069] Loss function: Reconstruction loss: measures the difference between the input data and the reconstructed data, calculated using mean squared error (MSE); KL divergence loss: measures the difference between the distribution of the potential representation and the prior distribution (usually a standard normal distribution). This loss term makes the potential representation smoother and more continuous. Total loss function: the weighted sum of reconstruction loss and KL divergence loss;

[0070] Sampling process: At each forward propagation, the mean μ and variance σ of the latent variable z are first calculated through the encoder 2 , then from the distribution N(μ,σ 2 ) to obtain z. This process is called the reparameterization trick; it can update the encoder weights during backpropagation;

[0071] Noise reduction process: Data reconstruction: The trained VAE model can effectively remove noise components and reconstruct clean data. Specifically, for each synchronized multi-sensor data Xi, the encoder maps it to the potential representation z, and then the decoder obtains the reconstructed sensor data from z.

[0072] In this example, the reconstructed sensor data obtained The corresponding deep autoencoder (DAE) is used for deep feature extraction. DAE learns the deep features Hi of the data through multiple layers of nonlinear transformation.

[0073] S3, the deep feature H i Input into the self-attention module, dynamically adjust the weights of different sensor inputs, and generate the fused feature representation H fused ; It can effectively improve the quality of feature representation and provide strong support for further decision-making and planning. The specific process is:

[0074] The deep features H of each sensor i Indicated as H i ∈R d , where d is the feature dimension, all the sensors’ deep features H i Splicing to get the input matrix H∈R N×d , where N is the number of sensors;

[0075] Using three independent linear transformation matrices W Q ,W K ,W V Transform the input matrix H separately to obtain the query matrix Q, key matrix K and value matrix V:

[0076] Q=HW Q ;

[0077] K=HW K ;

[0078] V=HW V ;

[0079] Calculate the dot product similarity between the query matrix Q and the key matrix K to obtain the unnormalized attention score matrix: A′=QK T ;

[0080] Scale the dot product result A′ and apply the Softmax function to obtain the normalized attention score matrix:

[0081] According to the normalized attention score matrix A, the value matrix V is weighted and summed to obtain the final fusion feature representation H fused ; That is H fused =AV.

[0082] S4, based on the fused feature representation H fused Perform graph optimization to obtain the Laplace matrix L f ; The specific process is:

[0083] The fused feature representation H fused Construct an adjacency matrix;

[0084] Calculate the fused feature representation H fused Two data points X p and X q The Euclidean distance between them is then used to construct the adjacency matrix A using the binarization method. f =A1A2…A n If the data point X q Belongs to data point X p The k-nearest neighbor set N p , then it is considered that there is a connection between the two data points, and the corresponding adjacency matrix element A i (p,q) is set to 1; otherwise, it is set to 0, indicating that there is no direct connection between the two data points. p is the k smallest distances from p to all other data points.

[0085] Construct a K-nearest neighbor directed graph G based on the adjacency matrix and the K-nearest neighbor algorithm f =G1, ..., G n ;

[0086] When constructing a directed graph based on K nearest neighbors, the weight of the edge usually reflects the similarity or distance between nodes. The distance between two nodes in the K nearest neighbor directed graph is converted into a weight W through the Gaussian kernel function. f ;

[0087] Weight W on the edges of the adjacency matrix f Distribution, calculate the data point X through the Gaussian kernel function p and X q The distance between them is realized; Gaussian kernel function σ is a parameter that controls the decay rate of similarity; through the formula: L f =DW f Calculate the graph Laplacian matrix L f , where D is the diagonal matrix

[0088] S5, through the multi-sensor raw data, the fused feature representation H fused and the Laplace matrix L f A joint optimization function is established, and the alternating direction multiplier method is used to solve it. It is iterated and updated until convergence to obtain the fusion result.

[0089] The joint optimization function is:

[0090]

[0091] Among them, the first term of the formula is the reconstruction loss term, λ1, λ2, and λ3 are the weight parameters of the control graph regularization term, the consistency constraint term between the pseudo label and the real label, and the label consistency term, respectively. J represents the joint optimization objective function, and x i is the raw data collected by the i-th sensor, H fusedi is the i-th data after sensor data fusion, P i is a pseudo label, y i is the true label, and M is the classification weight matrix.

[0092] For example, the alternating direction multiplier method is used to solve the joint optimization function. By introducing auxiliary variables and corresponding Lagrange multipliers, the original optimization problem is transformed into multiple sub-problems that can be processed in parallel. The specific steps are as follows:

[0093] Convert the problem solved by the joint optimization function into multiple sub-problems that can be processed in parallel.

[0094] Iteratively update the pseudo labels and classification weight matrix of each sub-problem so that the corresponding local objective function is minimized;

[0095] At the end of each iteration, the Lagrange multiplier Λ and penalty parameter ρ of each subproblem are updated to obtain the current solution to ensure global consistency. Specifically, the update rule can be expressed as: Λ (k+1) =Λ (k) +ρ(AX (k+1) -BY (k+1) -c), where X(k+1) and Y (k+1) are the new values of the pseudo labels and classification weight matrix in the current iteration step, respectively, while A, B, and c define the linear constraints.

[0096] The preset convergence criterion is used to determine whether the current solution reaches the preset threshold. If so, the iteration is terminated; otherwise, the iteration continues.

[0097] In this embodiment, through graph optimization and the alternating direction multiplier method, the overall optimization problem is transformed into multiple, more manageable subproblems, reducing computational complexity and improving fusion quality and robustness. This graph optimization-based approach utilizes currently estimated pseudo-labels and the graph Laplacian for label propagation to correct for potential noise or missing labels, further enhancing the robustness of multi-sensor data fusion in complex environments.

[0098] In a second aspect of the embodiments of the present invention, the present invention further provides an anti-interference multi-sensor fusion system for an unmanned vehicle in a mine, comprising:

[0099] The first unit is to obtain the raw data of multiple sensors in the operating environment of the unmanned vehicle and synchronize the time of the raw data of multiple sensors;

[0100] The second unit reconstructs each synchronized multi-sensor data Xi and removes the noise component to obtain the corresponding reconstructed sensor data. Extract individual reconstructed sensor data The deep features of Hi;

[0101] Unit 3: Deep feature H i Input into the self-attention module, dynamically adjust the weights of different sensor inputs, and generate the fused feature representation H fused ;

[0102] Unit 4: Based on the fused feature representation H fused Perform graph optimization to obtain the Laplace matrix L f ;

[0103] Unit 5, through the multi-sensor raw data, fused feature representation H fused and the Laplace matrix L f A joint optimization function is established, and the alternating direction multiplier method is used to solve it. It is iterated and updated until convergence to obtain the fusion result.

[0104] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0105] processor;

[0106] a memory for storing processor-executable instructions;

[0107] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0108] The above description is only a specific embodiment of the present invention. The above embodiments are only used to illustrate the technical solution of the present invention, but are not used to limit the scope of protection of the present invention. All changes within the scope of the present invention or within the scope equivalent to the scope of the present invention are included in the present invention.

Claims

1. A multi-sensor fusion method for unmanned vehicles with anti-interference in mines, characterized by: The following steps are involved: S1, obtains the raw data of multiple sensors in the operating environment of the unmanned vehicle and synchronizes the time of the raw data of multiple sensors; S2, reconstruct each synchronized multi-sensor data Xi and remove the noise component to obtain the corresponding reconstructed sensor data Extract individual reconstructed sensor data The deep features of Hi; S3, the deep feature H i Input into the self-attention module, dynamically adjust the weights of different sensor inputs, and generate the fused feature representation H fused ; S4, based on the fused feature representation H fused Perform graph optimization to obtain the Laplace matrix L f ; S5, through the multi-sensor raw data, the fused feature representation H fused and the Laplace matrix L f A joint optimization function is established, and the alternating direction multiplier method is used to solve it. It is iterated and updated until convergence to obtain the fusion result.

2. The multi-sensor fusion method for anti-interference unmanned vehicle in mines according to claim 1 is characterized in that: In S1, before the multi-sensor raw data enters the acquisition terminal, the multi-sensor raw data is first stored in the hardware register and the software array vector; the multi-sensor raw data is coordinated and synchronized by combining hardware and software to synchronize the time of the multi-sensor raw data.

3. The multi-sensor fusion method for anti-interference unmanned vehicle in mines according to claim 1 is characterized in that: The specific process in S3 is: The deep features H of each sensor i Indicated as H i ∈R d , where d is the feature dimension, all the sensors’ deep features H i Splicing to get the input matrix H∈R N×d , where N is the number of sensors; Using three independent linear transformation matrices W Q ,W K ,W V Transform the input matrix H separately to obtain the query matrix Q, key matrix K and value matrix V: Q=HW Q ; K=HW K ; V=HW V ; Calculate the dot product similarity between the query matrix Q and the key matrix K to obtain the unnormalized attention score matrix: A' = QK T ; Scale the dot product result A′ and apply the Softmax function to obtain the normalized attention score matrix: According to the normalized attention score matrix A, the value matrix V is weighted and summed to obtain the final fusion feature representation H fused .

4. The multi-sensor fusion method for anti-interference unmanned vehicle in mines according to claim 3 is characterized in that: The specific process in S4 is: The fused feature representation H fused Construct an adjacency matrix; Construct a K-nearest neighbor directed graph based on the adjacency matrix using the K-nearest neighbor algorithm; The distance between two nodes in the K-nearest neighbor directed graph is converted into a weight W through the Gaussian kernel function f ; Weight W on the edges of the adjacency matrix f Distribution, by formula: L f =DW f Calculate the graph Laplacian matrix L f , where D is the diagonal matrix.

5. The multi-sensor fusion method for anti-interference unmanned vehicle in mines according to claim 4 is characterized in that: The joint optimization function in S5 is: Among them, the first term of the formula is the reconstruction loss term, λ1, λ2, and λ3 are the weight parameters of the control graph regularization term, the consistency constraint term between the pseudo label and the real label, and the label consistency term, respectively. J represents the joint optimization objective function, and x i is the raw data collected by the i-th sensor, H fusedi is the i-th data after sensor data fusion, P i is a pseudo label, y i is the true label, and M is the classification weight matrix.

6. The multi-sensor fusion method for anti-interference unmanned vehicle in mines according to claim 5 is characterized in that: The specific process of solving the joint optimization function using the alternating direction multiplier method in S5 is: Convert the problem of solving the joint optimization function into multiple sub-problems that can be processed in parallel; Iteratively update the pseudo labels and classification weight matrix of each sub-problem; At the end of each iteration, the Lagrange multiplier Λ and penalty parameter ρ of each subproblem are updated to obtain the current solution; The update rule is expressed as: (k+1) =Λ (k) +ρ(AX (k+1) -BY (k+1) -c); Among them, X (k+1) and Y (k+1) are the new values of the pseudo-label and classification weight matrix in the current iteration step, respectively. A, B, and c are the defined linear constraints; The preset convergence criterion is used to determine whether the current solution reaches the preset threshold. If so, the iteration is terminated; otherwise, the iteration continues.

7. The multi-sensor fusion method for anti-interference unmanned vehicle in mines according to claim 6 is characterized in that: In S5, noise and / or missing labels are corrected by label propagation.

8. The multi-sensor fusion method for anti-interference unmanned vehicle in mines according to claim 1 is characterized in that: The multi-sensor data includes millimeter-wave radar data, visual data, IMU data, vehicle-mounted navigation data, vehicle-mounted OBU terminal data, and vehicle-mounted sensor data.

9. An anti-interference multi-sensor fusion system for unmanned vehicles in mines, used to implement the method according to any one of claims 1 to 8, characterized in that: include: The first unit is to obtain the raw data of multiple sensors in the operating environment of the unmanned vehicle and synchronize the time of the raw data of multiple sensors; The second unit reconstructs each synchronized multi-sensor data Xi and removes the noise component to obtain the corresponding reconstructed sensor data. Extract individual reconstructed sensor data The deep features of Hi; Unit 3: Deep feature H i Input into the self-attention module, dynamically adjust the weights of different sensor inputs, and generate the fused feature representation H fused ; Unit 4: Based on the fused feature representation H fused Perform graph optimization to obtain the Laplace matrix L f ; Unit 5, through the multi-sensor raw data, fused feature representation H fused and the Laplace matrix L f A joint optimization function is established, and the alternating direction multiplier method is used to solve it. It is iterated and updated until convergence to obtain the fusion result.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 8.

Images