A multi-modal sensor data preprocessing method for special purpose vehicles

Abstract

The present invention relates to a multimodal sensor data preprocessing method for special-purpose vehicles, and belongs to the field of detection and early warning of special-purpose vehicles. The method includes: for one-dimensional data generated by special-purpose vehicles collected by multimodal sensors, a data cleaning method based on contextual reasoning is used for data cleaning; for the operation of special-purpose vehicles, when the amount of normal working condition data collected by multimodal sensors is greater than the amount of abnormal working condition data, and the normal working condition data has a small early warning detection effect on abnormal working conditions, a data balancing method based on K-Means clustering and diffusion network is used to achieve data balancing; for multimodal data indicators collected by multi-sensors of special-purpose vehicles, a data fusion method based on graph neural network and multi-scale adaptive optimization module based on attention mechanism is used to improve the detection capability of the early warning model and improve the utilization efficiency of data.

Description

Technical Field

[0001] The invention belongs to the field of special purpose vehicle detection and early warning, and relates to a multi-modal sensor data preprocessing method for special purpose vehicles. Background Art

[0002] When special vehicles are in operation, related sensors will generate a large amount of data, such as image data from cameras, surrounding environment data obtained by radar, vibration data generated by various parts of the car, speed data, excitation signals generated by charging piles, etc. These signals often contain a lot of noise, and the data preprocessing of special vehicles aims to obtain these data of special vehicles through sensors through some methods and then process them, so that the data has a small impact on the fault warning detection results of special vehicles, or even has a positive effect on the warning results. At present, data preprocessing technology is widely used in many fields such as data mining, financial analysis, medical research, environmental testing, fault detection and deep learning, and has achieved wide success. At the same time, the quality of data directly determines the prediction or generalization ability of the model. In real situations, data often contains a large number of missing values ​​or generates a large amount of noise pollution data due to the environment in which they are located. In particular, special vehicles often exist in special environments, which will have a greater impact. At the same time, the quality of the sensor itself also leads to abnormal data collection. These multi-coupling factors when the car is running are very unfavorable for the algorithm model to judge the final result, so it is necessary to preprocess the real-world data to improve the data quality. Common data preprocessing methods include: data cleaning, data fusion, data reduction, data balancing and data enhancement, etc. The result of applying these methods is to process various dirty data and obtain standard, clean and continuous data to provide for data statistics, data mining, early warning detection and other algorithm models. In traditional data preprocessing, the traditional data collection source is single, and the amount of data stored, managed and analyzed is relatively small. Traditional data preprocessing methods have great problems in speed and reliability for big data analysis, and cannot meet the processing requirements of multi-modal sensor data obtained by special vehicles. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a multimodal sensor data preprocessing method for special-purpose vehicles, so as to solve the problem that traditional special-purpose vehicle data preprocessing methods have become relatively ineffective in the current data environment due to rich data sources and varied interference causes, and at the same time, traditional data preprocessing methods lack a systematic processing method for the big data of multimodal sensors of special-purpose vehicles.

[0004] This method first cleans the collected one-dimensional data, uses the Bayesian network that integrates expert experience to confirm the data distribution, then uses the contextual reasoning model to capture the distribution information in the adjacent data, and uses the confidence interval to make a rational judgment, and cleans the outlier data and dirty data that does not conform to the data distribution; in the operation process of special-purpose vehicles, the amount of normal working condition data collected by multimodal sensors is greater than the amount of abnormal working condition data, and the normal data has a small warning detection effect on abnormal working conditions, so it is necessary to use a data balancing method, use the K-Means algorithm to undersample most of the data, for the collected multi-sensor data information, the data distribution is obtained by clustering to form data clusters representing related working conditions, and most of the data clusters are randomly trimmed according to the data distribution, thereby reducing the number of normal data distributions; for a small amount of abnormal working condition data, a data generation method based on a diffusion network is used to oversample a small amount of abnormal working condition information, and diffusion generates highly similar data to the real data through the Markov chain method derived from the front and back. Simulate data, use multi-channel staggered input to accelerate the data refinement process, and use the diffusion model to complete the generation of hidden variable distribution for real data in an asymmetric way, continuously narrowing the difference between imbalanced data in different distribution spaces and balanced data, and accelerating the data reconstruction speed of the entire model channel through iteration to ensure the effectiveness of the data balancing strategy, and ultimately increase the abnormal working condition sample data, enrich a few abnormal working condition sample clusters, and achieve data balancing effect; special purpose vehicle multi-sensor data is often related, such as when a car is in emergency braking, it is often reflected in the car speed, gear speed and other aspects, and has time correlation. For special purpose vehicle multi-sensor data, a vehicle multimodal data fusion method based on different spatiotemporal scales is proposed. The first-order graph convolution and spatiotemporal attention mechanism are used to establish the correlation between multi-sensor multimodal data, and the data of different sensors are input to obtain multi-scale information of sensor data; a multi-scale adaptive fusion optimization module is introduced to realize the extraction and fusion of key information of multi-scale spatiotemporal sequence data, and the fusion efficiency is optimized through adaptive methods to improve the reliability of data fusion. Data fusion can complement the operating condition information of a single sensor with the operating condition information provided by other types of sensors through graph convolution, improve the detection capability of the early warning model, and improve the utilization efficiency of data.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A multimodal sensor data preprocessing method for special purpose vehicles specifically includes the following steps:

[0007] S1: For the one-dimensional data generated by special-purpose vehicles collected by multimodal sensors, a data cleaning method based on contextual reasoning is adopted, which specifically includes: using a Bayesian network that integrates expert experience to confirm the data distribution, and then using a contextual reasoning model to capture the distribution information in adjacent data, and using confidence intervals to make rational judgments, and cleaning up outlier data and dirty data that does not conform to the data distribution.

[0008] S2: In the operation process of special purpose vehicles, when the amount of normal working condition data collected by multimodal sensors is greater than the amount of abnormal working condition data and the abnormal working condition data is smaller, and the normal working condition data has little warning detection effect on abnormal working conditions and it is necessary to increase the abnormal working condition data that conforms to the data distribution, a data balancing method based on K-Means clustering and diffusion network is adopted, specifically including: using K-Means clustering method for undersampling processing, obtaining sample cluster clustering by calculating the data distribution, and using random trimming method for most sample clusters to reduce redundant and repeated data to achieve data balancing; for a small number of data sample clusters, using diffusion-based data generation method to oversample a small amount of abnormal working condition information, and generating new data samples by imitating the probability distribution of a small number of abnormal working condition information in sensors in special purpose vehicles to achieve data balancing.

[0009] S3: For the multimodal data indicators collected by multiple sensors of special purpose vehicles (which can have a positive impact on certain tasks), a data fusion method based on a graph neural network with an attention mechanism and a multi-scale adaptive optimization module is used to fuse the data of multiple sensors (one-dimensional data) to achieve more accurate judgment than single sensor data. Specifically, it includes: using prior knowledge to combine sensors that have a positive impact on the detection of certain abnormal working conditions, establishing relevant connections between these sensor data through the structural connection of the graph, establishing connections between multiple sensors through the first-order approximate graph convolution that integrates the spatiotemporal attention mechanism, and then extracting and fusing the key information of the multi-scale spatiotemporal sequence data through the multi-scale adaptive optimization module.

[0010] Furthermore, in step S1, a data cleaning method is used to clean not only outlier data, but also dirty data generated by coupled noise interference, or abnormal data generated when a sensor fails, etc. These data come from vibration signals, speed signals, rotation speed signals, etc. of special vehicles, and are all one-dimensional data.

[0011] The data cleaning method specifically comprises the following steps:

[0012] S11: The Bayesian network that integrates expert prior knowledge provides an intuitive method to capture the relationship between features in the data domain; for special-purpose vehicle multi-sensor data, a data model is established for the sensor through expert knowledge; the data distribution information is captured in the network structure through the Bayesian method to obtain the causal relationship of data features; Among them, the Bayesian network provides a compact representation of data by utilizing conditional independence and local structure, and can effectively calculate the joint probability distribution, laying the foundation for contextual reasoning;

[0013] S12: The context information of special purpose vehicles is represented as the data distribution of adjacent data, which has time correlation. For the input one-dimensional data of special purpose vehicles, the data distribution relationship is captured through the Bayesian network, the data probability distribution is calculated, and the context reasoning is used to determine whether the data distribution conforms to the prior probability distribution of reasonable data. By setting the confidence interval, the outlier data and abnormal data are cleaned.

[0014] Further, in step S2, a large amount of data (mainly one-dimensional data) will be generated during the long-term operation of special-purpose vehicles, among which there is a large amount of normal working condition data in the sensors that detect the state of the vehicle. These data will affect the early warning and fault detection efficiency of the relevant algorithms. Therefore, the K-Means clustering method is used for undersampling processing, and the sample clusters are obtained by calculating the data distribution. The random cutting method is implemented for most sample clusters to reduce redundant and repeated data to achieve data balance. For a small number of data sample clusters, a diffusion-based data generation method is used to perform oversampling operations on a small amount of abnormal working condition information, and new data samples are generated by imitating the probability distribution of a small number of abnormal working condition information in the sensors of special-purpose vehicles to achieve data balance.

[0015] The data balancing method specifically comprises the following steps:

[0016] S21: Cluster the special vehicle sensor data through K-Means clustering, so that the majority class samples generate N sample clusters. Considering the requirements of multi-sensors for the number of data samples for different prediction tasks or fault detection tasks, the N sample clusters are randomly cropped to reduce the impact on feature extraction;

[0017] S22: In the diffusion-based data generation method, the diffusion network is divided into a forward chain and a reverse chain. The role of the forward chain is to perturb the data. It gradually adds Gaussian noise to the data according to a pre-designed noise schedule until the distribution of the data tends to the prior distribution, that is, the distribution of abnormal sensor data; then the reverse chain uses the prior distribution generated by the forward chain and the parameterized Gaussian transformation kernel to learn to gradually restore the original data distribution; through training, data that is highly similar to the original data can be generated.

[0018] Furthermore, in step S22, the forward process does not contain learnable parameters. As t increases, the final distribution becomes a Gaussian distribution that is independent in all directions. Gaussian noise is added to the forward process for a total of T steps, thereby generating a series of noisy samples, β t Indicates adding noise;

[0019]

[0020] Among them, q(x t |x t-1 ) represents the joint distribution probability between the data at time t and time t-1, t represents the time step, x t represents the noise data at time step t, I represents the unit matrix, Represents the data symbol Gaussian distribution.

[0021] Since it is defined as a Markov chain, given x0 and x 1:T The joint probability distribution of 1:T |x0);p θ (x t-1 |x t ) represents the forward process:

[0022]

[0023] The inverse process is also a Markov chain process, q(x T:1 |x0) is the approximate posterior;

[0024]

[0025]

[0026] x t:T represents the data generated at a certain moment, p(x 0:T ) represents the data distribution at time T obtained by forward Markov chain derivation, μ θ (x t ,t) represents the mean of the Gaussian distribution at time t, q(x T:1 |x0) represents the probability distribution at time 0 of the reverse process;

[0027] The final training is performed by optimizing the usual variation constraints of the negative log-likelihood, and iteratively updating the Gaussian transformation kernel, ε θ represents a prediction approximator, Represents a data distribution that conforms to a Gaussian distribution and can be reparameterized. Through reparameterization, relevant training can be performed, and abnormal operating data can be put into the trained diffusion network model to generate abnormal operating data with a distribution similar to the original one-dimensional signal:

[0028]

[0029] Among them, Loss represents the loss function, Represents the true ε and predicted ε θ Expected difference, ε represents the noise data distribution, ||·|| 2 represents the KL divergence.

[0030] Furthermore, in step S3, considering that the multimodal data indicators collected by the multi-sensors of special-purpose vehicles can have a positive impact on certain tasks, more accurate judgments can be achieved by fusing the data from multiple sensors (one-dimensional data) than single sensor data. Sensors that have a positive impact on the detection of a certain abnormal working condition are combined using prior knowledge, and the sensor data are connected through the structural connection of the graph to establish relevant connections, and the connection between multiple sensors is established through the first-order approximate graph convolution that integrates the spatiotemporal attention mechanism, and then the key information of the multi-scale spatiotemporal sequence data is extracted and fused through the multi-scale adaptive optimization module.

[0031] The fusion of data from multiple sensors includes the following steps:

[0032] S31: Combine multi-sensor data through prior knowledge and establish connection information between sensor data through graph structure, which is expressed as:

[0033]

[0034] Among them, X is the feature matrix of the data, A is the adjacency matrix, I is the identity matrix, yes The degree matrix of , F is the constructed graph information matrix;

[0035] S32: Input the established graph information into the first-order graph convolutional network that integrates the spatiotemporal attention mechanism, make judgments on the feature information and importance obtained by multiple sensors, and extract features through the propagation layer of GCN. No classification is performed here;

[0036]

[0037]

[0038] Among them, σ is the activation function, X represents the current feature matrix, H l is the feature matrix of the lth layer, W l is the network weight of the lth layer; Formula (8) is the spatiotemporal attention mechanism, S represents the attention matrix, G represents the dimensional relationship of the graph structure, t represents the time step, and b s represents random error;

[0039] S33: Use the multi-scale adaptive optimization module to extract and fuse the key information of multi-scale spatiotemporal series data to enhance data quality;

[0040] Z t =Attention(f(H,A) (9)

[0041]

[0042]

[0043] Among them, Z t is the feature obtained by the attention mechanism of the graph neural network, Y i represents the adaptive module of the i-th sensor, g i is sampled from the independent distribution Gumbel (0, 1) distribution, υ is the adaptive learning parameter; It is a brief representation of data fusion, indicating the final fused data. The fused data is still one-dimensional data. i represents the random weight of the multi-scale module, and n represents the number of sensors.

[0044] The beneficial effects of the present invention are as follows: the method of the present invention mainly targets the one-dimensional data generated by the sensor, and uses data cleaning, data balancing and data fusion methods to comprehensively process these data to ensure the validity and reliability of the data, and can provide high-quality data samples for early warning and fault detection of special-purpose vehicles. Data cleaning cleans the sensor data through a data cleaning model that integrates the contextual reasoning of the Bayesian network; data balancing balances the multi-sensor data under normal working conditions of the special-purpose vehicle through an oversampling method based on K-Means clustering, and performs data balancing on a small amount of data under abnormal working conditions by undersampling through a data generation method based on a diffusion network; data fusion establishes the connection information of multiple sensors through a graph structure, extracts the characteristic information of the sensor through a graph neural network and a spatiotemporal attention mechanism, and uses a multi-scale adaptive optimization module to perform data fusion, and finally obtains high-quality data.

[0045] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0047] Figure 1 This is a flow chart of a multi-modal sensor data preprocessing method for special-purpose vehicles according to the present invention;

[0048] Figure 2 A data cleaning method model based on contextual reasoning;

[0049] Figure 3 It is a data balancing model based on K-Means clustering and diffusion network;

[0050] Figure 4 It is a data fusion model for the attention-based graph and multi-scale adaptive optimization modules;

[0051] Figure 5 This is the structure diagram of the graph convolutional neural network;

[0052] Figure 6 This is the structure diagram of the attention mechanism. DETAILED DESCRIPTION

[0053] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0054] See also Figure 1 to Figure 6The present invention provides a multi-modal sensor data preprocessing method for special-purpose vehicles. It mainly performs data preprocessing on the massive data generated by various sensors of special-purpose vehicles facing complex road conditions and vehicle conditions in multiple scenarios. The main work includes three aspects: data cleaning, data balancing and data fusion. The relevant data preprocessing method is mainly aimed at the one-dimensional signals obtained by sensors at various positions of special vehicles, such as vibration signals at positions such as axles or frames, speed signals of vehicles, rotation speed signals, electromagnetic excitation signals generated by charging piles and other multi-modal data. The present invention hopes to provide high-quality data for subsequent prediction and early warning work and improve the detection and early warning effect through these methods. The processed data is still in the form of one-dimensional data. Among them, the data cleaning method mainly uses the Bayesian network that incorporates expert experience as prior knowledge, combined with the use of an inference model based on context information to clean the outliers and abnormally distributed data in the multi-modal one-dimensional sensor signals. The data balancing method uses a combination of undersampling and oversampling to process the one-dimensional raw data after data cleaning, clusters the sensor data under most normal working conditions using the K-means algorithm, and then undersamples most normal working condition data, and oversamples a few abnormal working condition data by generating data based on the Diffusion generation network; finally, data fusion is performed based on the multimodal data obtained by different sensors, first using graph convolution and spatiotemporal attention mechanisms to establish the connection between multi-sensor data, and then using a multi-scale adaptive optimization module for data fusion. Therefore, through these methods, a set of multi-source heterogeneous big data preprocessing methods for special-purpose vehicles is established, which specifically includes the following steps:

[0055] Step 1: Use special purpose vehicle multi-sensors to collect vehicle operation data (mainly one-dimensional data), upload the sensor data and pass it into the data cleaning model. The multi-sensor data obtains data distribution information through the Bayesian network that integrates expert prior knowledge, and then combines it with the context reasoning model for reasoning. The outlier data and the data that is severely interfered by noise and sensor failure are judged by setting confidence. When it is judged that the data distribution is unreasonable, the data is cleaned to obtain higher quality one-dimensional original data.

[0056] Step 2: After the data is cleaned, considering that there are a large number of normal operation data and a small amount of abnormal working condition data in the sensor data of special-purpose vehicles, it is necessary to balance the collected data. The K-Means algorithm is used to cluster the data and calculate multiple center points. The center points represent the characteristics of different types of data sample clusters. After all the data are classified, the undersampling method of randomly deleting the feature clusters far from the center point of the data is used to balance the data. For the fewer abnormal working condition information clusters, the diffusion-based data generation method is used. By passing the abnormal data into the forward propagation process of diffusion, the forward propagation uses the Markov chain and adds Gaussian noise to change the abnormal data distribution to Gaussian distribution. As the forward propagation finally converges to Gaussian distribution, according to the Markov rule, the reverse chain is to use the Gaussian distribution generated by the forward process to convert it into a parameterized Gaussian kernel. The noise that conforms to the Gaussian distribution is restored through the learned data distribution, thereby generating abnormal working condition data similar to the original data distribution. Data balance is achieved through the overfitting data generation method to obtain high-quality one-dimensional balanced data.

[0057] Step 3: Considering that multi-sensor multi-modal data fusion in special-purpose vehicles can play a positive role in some anomaly detection and fault location, a new method of multi-sensor data fusion is used to fuse multiple sensor data through prior knowledge to achieve better detection results than a single sensor. By establishing connections between the graph structures through which multiple sensors provide data, and establishing connections between sensors through graph neural networks, weighted operations are performed on different sensors through the spatiotemporal attention mechanism, and data fusion weights are assigned. Finally, a multi-scale adaptive optimization module is used to establish relevant scale parameters, obtain scale variables related to the current sensor combination, and design related functions. Adaptive learning is performed by updating the values ​​of the relevant functions, and finally the multi-sensor data of special-purpose vehicles are fused to obtain high-quality one-dimensional fused data.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A multi-modal sensor data preprocessing method for special purpose vehicles, characterized in that: The method specifically comprises the following steps: S1: For the one-dimensional data generated by special-purpose vehicles collected by multimodal sensors, a data cleaning method based on contextual reasoning is adopted, which includes: using the Bayesian network integrating expert experience to confirm the data distribution, then using the contextual reasoning model to capture the distribution information in the adjacent data, and using the confidence interval to make a reasonable judgment, and cleaning the outlier data and dirty data that does not conform to the data distribution; S2: In the operation process of special purpose vehicles, when the amount of normal working condition data collected by multimodal sensors is greater than the amount of abnormal working condition data and the amount of abnormal working condition data is small, and the normal working condition data has little warning detection effect on abnormal working conditions and needs to increase abnormal working condition data that conforms to the data distribution, a data balancing method based on K-Means clustering and diffusion network is used, which specifically includes: using K-Means clustering method for undersampling processing, obtaining sample cluster clustering by calculating data distribution, and using random cutting method for most sample clusters to reduce redundant and repeated data to achieve data balancing; using diffusion-based data generation method for a small number of data sample clusters to oversample a small amount of abnormal working condition information, and generating new data samples by imitating the probability distribution of a small number of abnormal working condition information in sensors in special purpose vehicles to achieve data balancing; S3: For the multimodal data indicators collected by multiple sensors of special purpose vehicles, a data fusion method based on a graph neural network with an attention mechanism and a multi-scale adaptive optimization module is used to fuse the data of multiple sensors. Specifically, the method includes: using prior knowledge to combine sensors that have a positive impact on the detection of a certain abnormal working condition, establishing relevant connections between these sensor data through the structural connection of the graph, establishing connections between multiple sensors through the first-order approximate graph convolution that integrates the spatiotemporal attention mechanism, and then extracting and fusing the key information of the multi-scale spatiotemporal sequence data through the multi-scale adaptive optimization module.

2. The method for preprocessing multimodal sensor data for special-purpose vehicles according to claim 1, characterized in that: In step S1, the data cleaning method specifically includes the following steps: S11: A Bayesian network that integrates expert prior knowledge is used to capture the relationship between features in the data domain. For multi-sensor data of special-purpose vehicles, a data model is established for the sensor through expert knowledge. The data distribution information is captured in the network structure through the Bayesian method to obtain the causal relationship of data features. Among them, the Bayesian network provides a compact representation of data by utilizing conditional independence and local structure, and can effectively calculate the joint probability distribution, laying the foundation for contextual reasoning. S12: The context information of special purpose vehicles is represented as the data distribution of adjacent data, which has time correlation. For the input one-dimensional data of special purpose vehicles, the data distribution relationship is captured through the Bayesian network, the data probability distribution is calculated, and the context reasoning is used to determine whether the data distribution conforms to the prior probability distribution of reasonable data. By setting the confidence interval, the outlier data and abnormal data are cleaned.

3. The method for preprocessing multimodal sensor data for special-purpose vehicles according to claim 1, characterized in that: In step S2, the data balancing method specifically includes the following steps: S21: Cluster the special vehicle sensor data through K-Means clustering, so that the majority class samples generate N sample clusters. Considering the requirements of multi-sensors for the number of data samples for different prediction tasks or fault detection tasks, the N sample clusters are randomly cropped to reduce the impact on feature extraction; S22: In the diffusion-based data generation method, the diffusion network is divided into a forward chain and a reverse chain. The role of the forward chain is to perturb the data. It gradually adds Gaussian noise to the data according to a pre-designed noise schedule until the distribution of the data tends to the prior distribution, that is, the distribution of abnormal sensor data; then the reverse chain uses the prior distribution generated by the forward chain and the parameterized Gaussian transformation kernel to learn to gradually restore the original data distribution; through training, data that is highly similar to the original data can be generated.

4. The method for preprocessing multimodal sensor data for special-purpose vehicles according to claim 3, characterized in that: In step S22, the forward process does not contain learnable parameters. As t increases, the final distribution becomes a Gaussian distribution that is independent in all directions. Gaussian noise is added to the forward process for a total of T steps, thereby generating a series of noisy samples, β t Indicates adding noise; Among them, q(x t |x t-1 ) represents the joint distribution probability between the data at time t and time t-1, t represents the time step, x t represents the noise data at time step t, I represents the unit matrix, Indicates that the data conforms to a Gaussian distribution; Since it is defined as a Markov chain process, the joint probability distribution at any given time is p θ (x t-1 |x t ); p(x 0:T ) represents the probability distribution of the forward process at time T, and the forward process is as follows: The inverse process is also a Markov chain process, q(x T:1 |x0) is the approximate posterior; Among them, x t:T represents the data generated at a certain moment, p(x 0:T ) represents the data distribution at time T obtained by forward Markov chain derivation, μ θ (x t ,t) represents the mean of the Gaussian distribution at time t, q(x T:1 |x0) represents the probability distribution at time 0 of the reverse process; The final training is performed by optimizing the usual variation constraints of the negative log-likelihood, and iteratively updating the Gaussian transformation kernel, ε θ represents a prediction approximator, Represents a data distribution that conforms to a Gaussian distribution and can be reparameterized. Through reparameterization, relevant training can be performed, and abnormal operating data can be put into the trained diffusion network model to generate abnormal operating data with a distribution similar to the original one-dimensional signal: Among them, Loss represents the loss function, Represents the true ε and predicted ε θ Expected difference, ε represents the noise data distribution, ||·|| 2 represents the KL divergence.

5. The method for preprocessing multimodal sensor data for special-purpose vehicles according to claim 1, characterized in that: In step S3, the data of multiple sensors are fused, which specifically includes the following steps: S31: Combine multi-sensor data through prior knowledge and establish connection information between sensor data through graph structure, which is expressed as: Among them, X is the feature matrix of the data, A is the adjacency matrix that can represent the relationship between multiple sensors, I is the identity matrix, yes The degree matrix of , F is the constructed graph information matrix; S32: Input the established graph information into the first-order graph convolutional network that integrates the spatiotemporal attention mechanism, make judgments on the feature information and importance obtained by multiple sensors, and extract features through the propagation layer of GCN. No classification is performed here; Among them, σ is the activation function, X represents the current feature matrix, H l is the feature matrix of the lth layer, W l is the network weight of the lth layer; Formula (8) is the spatiotemporal attention mechanism, S represents the attention matrix, G represents the dimensional relationship of the graph structure, t represents the time step, and b s represents random error; S33: Use the multi-scale adaptive optimization module to extract and fuse the key information of multi-scale spatiotemporal series data to enhance data quality; Z t =Attention(f(H,A)) (9) Among them, Z t is the feature obtained by the attention mechanism of the graph neural network, Y i represents the adaptive module of the i-th sensor, g i is sampled from the independent distribution Gumbel (0, 1) distribution, υ is the adaptive learning parameter; It is a brief representation of data fusion, indicating the final fused data. The fused data is still one-dimensional data. i represents the random weight of the multi-scale module, and n represents the number of sensors.

Images