Vehicle fault prediction model training method and device and vehicle fault prediction method

Abstract

The invention discloses a vehicle fault prediction model training method and device and a vehicle fault prediction method, and the method comprises the steps: obtaining sample data from different sensor sources, carrying out the feature extraction of the sample data through a neural network model, and carrying out the weighted fusion of the extracted features based on an attention mechanism, and obtaining a target feature; constructing a fault knowledge graph, and converting the fault knowledge graph into low-dimensional vector representation; adversarial training is carried out on a multi-modal deep learning model by using the target features and the low-dimensional vector representation to obtain a fault prediction model, the multi-modal deep learning model comprises double branch neural networks, one branch neural network is used for target feature training, and the other branch neural network is used for fault prediction. The other branch neural network is used for processing the low-dimensional vector representation; and the dual-branch neural network realizes fusion of different modal information based on a cross-modal attention mechanism. According to the invention, the multi-source heterogeneous data of the intelligent automobile can be effectively processed, and high-precision fault prediction is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of fault prediction, and in particular to a training method and device for a vehicle fault prediction model and a vehicle fault prediction method. Background Art

[0002] With the development of automobile technology, the level of vehicle electrification and intelligence is constantly improving, and the reliability and safety of vehicles are receiving more and more attention. Existing fault prediction methods mainly include rule-based methods, statistical analysis methods, and machine learning methods. However, these methods often have problems such as low prediction accuracy, poor real-time performance, and weak generalization ability when dealing with complex multi-source heterogeneous data of smart cars. Therefore, there is an urgent need for a fault prediction method that can fully utilize the characteristics of smart car data and has high accuracy, high real-time performance, and strong generalization ability. Summary of the invention

[0003] In response to the above technical problems, the present invention provides a training method and device for a vehicle fault prediction model and a vehicle fault prediction method, which can improve the accuracy of fault prediction.

[0004] A first aspect of the present invention provides a method for training a vehicle fault prediction model, comprising: Acquire sample data from different sensor sources, use a neural network model to extract features of the sample data, and perform weighted fusion of the extracted features based on an attention mechanism to obtain target features; construct a fault knowledge graph, and convert the fault knowledge graph into a low-dimensional vector representation; use the target features and the low-dimensional vector representation to perform adversarial training on a multimodal deep learning model to obtain a fault prediction model, the multimodal deep learning model includes a dual-branch neural network, one branch of the neural network is used for the target feature training, and the other branch of the neural network is used to process the low-dimensional vector representation; the dual-branch neural network realizes the fusion of different modal information based on a cross-modal attention mechanism.

[0005] In an optional embodiment, the neural network model is used to extract features from the sample data, and the extracted features are weightedly fused based on an attention mechanism to obtain target features, including: using a convolutional neural network to extract spatial features from sensor data, using a long short-term memory network to extract temporal features from time series data, and weighted fusion of the spatial features and the temporal features based on an attention mechanism.

[0006] In an optional implementation, the extracting time features of time series data using a long short-term memory network includes: performing frequency domain analysis on the time series data using Fourier transform and extracting frequency domain features.

[0007] In an optional implementation, constructing a fault knowledge graph and converting the fault knowledge graph into a low-dimensional vector representation includes: constructing a fault knowledge graph based on a vehicle fault diagnosis manual and an expert knowledge base, and converting the fault knowledge graph into a low-dimensional vector representation based on a graph embedding algorithm; wherein the fault knowledge graph is a dynamic knowledge graph.

[0008] In an optional embodiment, the use of the target features and the low-dimensional vector representation to perform adversarial training on the multimodal deep learning model includes: adding residual connections and batch normalization layers in each fully connected layer of the multimodal deep learning model to improve the convergence and generalization ability of the multimodal deep learning model.

[0009] In an optional implementation, the multimodal deep learning model adopts a Transformer model structure.

[0010] In an optional embodiment, the method further includes adjusting the probability distribution of the output fault prediction using a confidence calibration algorithm in the output layer of the multimodal deep learning model.

[0011] In an optional implementation, the method further includes using a GCN model to extract features from Internet of Vehicles data when the multimodal deep learning model extracts features from the sample data.

[0012] A second aspect of the present invention provides a training device for a vehicle fault prediction model, comprising: A feature extraction module is used to obtain sample data from different sensor sources, extract features from the sample data using a neural network model, and perform weighted fusion of the extracted features based on an attention mechanism to obtain target features; A graph conversion module, used to construct a fault knowledge graph and convert the fault knowledge graph into a low-dimensional vector representation; A training module is used to use the target features and the low-dimensional vector representation to perform adversarial training on a multimodal deep learning model to obtain a fault prediction model, wherein the multimodal deep learning model includes a two-branch neural network, one branch of the neural network is used for the target feature training, and the other branch of the neural network is used to process the low-dimensional vector representation; the two-branch neural network realizes the fusion of different modal information based on a cross-modal attention mechanism.

[0013] A third aspect of the present invention provides a vehicle fault prediction method, comprising: Obtain vehicle information of a vehicle to be predicted, the vehicle information including at least battery management system data, motor controller data, and on-board diagnostic system data; input the vehicle information into a fault prediction model trained by the training method of the vehicle fault prediction model provided according to the first aspect of the present invention, and the fault prediction model outputs a fault prediction result corresponding to the vehicle to be predicted.

[0014] A fourth aspect of the present invention provides an electronic device, comprising: At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method described in the first aspect or the third aspect of the embodiment of the present invention.

[0015] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the method according to the first aspect or the third aspect of the embodiment of the present invention is executed.

[0016] The present invention obtains a fault prediction model by performing adversarial training on a multimodal deep learning model using target features and low-dimensional vector representations. The fault prediction model is trained separately using a dual-branch neural network and realizes the fusion of different modal information based on a cross-modal attention mechanism. It can effectively process multi-source heterogeneous data of smart cars and achieve high-precision fault prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 The figure is a flow chart of a method for training a vehicle fault prediction model in an embodiment of the present invention.

[0019] Figure 2 The figure is a flow chart of a vehicle fault prediction method in an embodiment of the present invention.

[0020] Figure 3 A module schematic diagram of a vehicle fault prediction model training device in an embodiment of the present invention.

[0021] Figure 4 The present invention is a schematic diagram of a business process for applying a vehicle fault prediction method in an embodiment of the present invention.

[0022] Figure 5FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0024] It should be understood that the terms "first", "second", "third", etc. in the claims, specifications and drawings of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0025] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the claims, the singular forms of "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in the specification of the present invention and the claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0026] The present invention proposes an efficient and reliable vehicle fault prediction method by innovatively combining deep learning, time series analysis and knowledge graph technology. The method can make full use of multi-source heterogeneous data of smart cars to achieve high-precision and high-real-time fault prediction, providing important support for improving vehicle safety and reliability. In addition, the present invention has good scalability and adaptability, and can be flexibly adjusted and optimized according to actual application scenarios.

[0027] See also Figure 1 The present invention provides a training method for a vehicle fault prediction model, comprising the following steps.

[0028] Step 100: Obtain sample data from different sensor sources, use a neural network model to extract features from the sample data, and perform weighted fusion on the extracted features based on an attention mechanism to obtain target features.

[0029] In this step, the sample data is derived from the vehicle's operating data, including but not limited to engine status, fuel system, braking system, suspension system and other data, which can be classified as vehicle-mounted sensor data, on-board diagnostic system (OBD) data, vehicle control unit (ECU) data, Internet of Vehicles data, etc.

[0030] The collected sample data is cleaned, denoised and standardized, and the time series data is segmented and aligned using the sliding window method. For example, median filtering is used to remove outliers and Z-score standardization is performed. A 10-minute sliding window with a step size of 1 minute is used to segment the time series data.

[0031] Since the sample data come from different sources, it is necessary to assign weights to different data to achieve the purpose of data fusion. Among them, the attention mechanism is an important concept in the field of deep learning. When processing information, it can focus on the most relevant part of the current task and ignore other less important information. For example, the Transformer model is a neural network model completely based on the attention mechanism.

[0032] The weighted fusion of the extracted features is based on the similarity or matching of the features to obtain an attention score matrix, i.e., a probability distribution, and then a weighted sum is performed according to the probability distribution to obtain the final target feature. The target feature represents the acquired feature data that can be used to train the fault prediction model as input feature data for other models; wherein the sample data is the labeled sensor data.

[0033] Step 200: construct a fault knowledge graph, and convert the fault knowledge graph into a low-dimensional vector representation.

[0034] For example, a fault knowledge graph is constructed based on the vehicle fault diagnosis manual and the expert knowledge base. The fault knowledge graph is a structured semantic knowledge base used to describe concepts and their relationships in the physical world. The fault knowledge graph effectively processes, processes, and integrates complex document data into simple and clear "entity, relationship, entity" triples, and finally aggregates a large amount of knowledge to achieve rapid response and reasoning of knowledge. The elements of the knowledge graph are usually automatically or semi-automatically obtained from the vehicle fault diagnosis manual and the expert knowledge base through machine learning.

[0035] Then, the fault knowledge graph is converted into a low-dimensional vector representation based on a graph embedding algorithm. The fault knowledge graph is a dynamic knowledge graph. Dynamic knowledge graph technology can automatically update the graph structure according to the newly emerged fault type. The low-dimensional vector representation refers to converting high-dimensional data into a low-dimensional vector representation through a certain mapping or conversion method. This representation method is widely used in machine learning and deep learning, mainly used to improve computing efficiency, capture complex relationships between data, and retain the semantic or structural information of the original data.

[0036] For example, a knowledge graph containing 5,000 entities and 10,000 relationships is constructed based on the vehicle fault diagnosis manual, and the TransE algorithm is used to convert the knowledge graph into a 100-dimensional vector representation.

[0037] Step 300: Use the target features and the low-dimensional vector representation to perform adversarial training on a multimodal deep learning model to obtain a fault prediction model, wherein the multimodal deep learning model includes a dual-branch neural network, one branch of the neural network is used for the target feature training, and the other branch of the neural network is used to process the low-dimensional vector representation; the dual-branch neural network realizes the fusion of different modal information based on a cross-modal attention mechanism.

[0038] The multimodal deep learning model integrates information from different perceptual modalities (such as images, text, voice, etc.) into a deep learning model to achieve richer information expression and more accurate prediction. In one embodiment of the present invention, the multimodal deep learning model adopts a Transformer model structure. In other embodiments, it can also be based on a pre-trained open source large model, such as Lamma2, Qianwen, etc. as a multimodal deep learning model. The multimodal deep learning model receives processed data features and can perform fault probability prediction after model training.

[0039] For example, a dual-branch neural network is constructed, where one branch of the neural network processes the fused sensor features, i.e., the target feature training, and the other branch of the neural network processes the knowledge graph embedding, i.e., the low-dimensional vector representation.

[0040] Cross-modal Attention Mechanism is an important application in multimodal deep learning. It allows the model to find corresponding parts of information related to another modality in one modality, thereby achieving effective fusion between different modalities. The multimodal Transformer model combined with adaptive modality weights (MTAMW) can achieve multimodal information fusion, where adaptive modality weights introduce a multimodal adaptive weight matrix to adaptively learn the importance of different modalities. Others include Modality Specific Adaptive Scaling and Attention Network (MASAN), which solves the problem of interference features in pre-trained models through adaptive scaling and attention mechanisms, and effectively integrates text semantics.

[0041] Adversarial training technology is used in multimodal deep learning model training, and a cross-modal attention mechanism is implemented based on the structure of the Transformer model. The attention mechanism and adversarial training technology are introduced in the model training process of the present invention, which improves the model's ability to identify different fault types and its anti-interference ability.

[0042] The present invention obtains a fault prediction model by using target features and low-dimensional vector representation to perform adversarial training on a multimodal deep learning model. The fault prediction model is trained separately using a dual-branch neural network and realizes the fusion of different modal information based on a cross-modal attention mechanism. It can effectively process multi-source heterogeneous data of smart cars and realize high-precision and high-real-time fault prediction. The following application examples will be used to illustrate this.

[0043] Collect vehicle operation data, clean, filter and aggregate the received raw data, annotate the data and form sample data. In the process of processing the raw data, preprocessing can be performed based on the score standardization method. Score standardization standardizes the mean and standard deviation of the raw data so that the data conforms to the standard normal distribution. Use a 10-minute sliding window with a step size of 1 minute to segment the time series data.

[0044] When data is aggregated, key features related to fault prediction are extracted and then fused. In the above step 100, a convolutional neural network (CNN) is used to extract spatial features of sensor data, and a long short-term memory network (LSTM) is used to extract temporal features of time series data, and the spatial features and the temporal features are weightedly fused based on the attention mechanism.

[0045] For example, a one-dimensional convolutional neural network (1D-CNN) is used to extract features from battery management system (BMS) data, with a convolution kernel size of 3, a stride of 1, and a total of 3 convolution layers; a bidirectional LSTM network is used to extract features from motor controller data, with a hidden layer size of 128. The extracted features are then weighted and fused using a self-attention mechanism, which can help the model identify the features of items that users are most interested in.

[0046] When the multimodal deep learning model extracts features from the sample data, the GCN model is used to extract features from the Internet of Vehicles data, for example, using a GCN network (Graph Convolutional Network) to extract features of interactions between vehicles.

[0047] Furthermore, when extracting time features, Fourier transform can be used to perform frequency domain analysis on the time series data and extract frequency domain features. Fourier transform is a mathematical method for converting time series data into frequency domain. Through Fourier transform, a time series signal can be decomposed into a combination of sine and cosine waves of different frequencies. It is very useful in the field of signal processing and can facilitate the analysis and extraction of frequency domain features of signals.

[0048] Then the above features are input into the model to conduct adversarial training on the multimodal deep learning model. Model training usually trains the parameter values ​​of the model. The model usually includes input layer, convolution layer, excitation layer, pooling layer, and fully connected layer. Data is input through the input layer, and the convolution layer uses convolution kernel to perform convolution to extract features. In order to improve the nonlinear ability of the network and improve the expression ability of the network, each convolution layer is followed by an activation layer; the excitation layer uses the activation function to activate the convolution layer. The pooling layer is usually located between the convolution layers to reduce the number of parameters of the feature map and improve the calculation speed. It can retain some important feature information, improve fault tolerance, and prevent overfitting to a certain extent. The fully connected layer is located after the convolution layer and the pooling layer. Its function is to map the learned feature representation to the sample label space; each neuron in the fully connected layer is connected to all neurons in the previous layer, and is used to reassemble all local features extracted by the convolution layer and the pooling layer into a complete feature map through the weight matrix.

[0049] In one embodiment, a residual connection and a batch normalization layer are added to each fully connected layer of the multimodal deep learning model to improve the convergence and generalization ability of the multimodal deep learning model.

[0050] Among them, residual connections solve the gradient vanishing problem in deep neural networks by introducing "quick connections" or "skip connections". In a residual network, a residual block contains two paths: one is a direct connection (i.e., a shortcut connection), and the other is a path after passing through a series of layers (such as convolutional layers, activation layers, etc.). The outputs of these two paths are added to form the final output of the residual block. Residual connections allow gradients to be passed directly through shortcut connections, thereby alleviating the gradient vanishing problem and making the network deeper and easier to train.

[0051] Batch normalization is a technique used to accelerate deep network training and provide a certain degree of regularization. Batch normalization normalizes the input of each layer to make the input distribution more stable, which helps to accelerate the training process. Batch normalization calculates the mean and variance of small batches of data, then normalizes the output of each neuron, and finally uses learnable parameters (stretch parameters) to normalize the output of each neuron. γ and offset parameters β ) to make adjustments.

[0052] Residual connections and batch normalization are often used in combination. For example, in a residual block, the batch normalization layer is usually placed before the activation function, which normalizes the data before the activation function, which helps the stability and convergence of the network; it not only speeds up training, but also improves the generalization ability of the model.

[0053] Furthermore, the training method of the vehicle fault prediction model also includes adjusting the probability distribution of the output fault prediction using a confidence calibration algorithm in the output layer of the multimodal deep learning model.

[0054] The last fully connected layer of the multimodal deep learning model is used as the output layer, and the confidence calibration technology is introduced in the output layer to improve the reliability of the prediction. For example, the probability distribution of the model output is adjusted by using the temperature scaling method. Other confidence calibration techniques include the Platt Scaling algorithm, which adjusts the output probability by training an additional logistic regression layer on the trained model.

[0055] During the model training phase, supervised learning is performed using a large-scale labeled data set. Transfer learning methods are used to improve the prediction performance in small sample scenarios using pre-trained models. Adversarial training techniques are introduced to improve the robustness of the model. A cyclic learning rate strategy is adopted, with an initial learning rate of 0.01, which decays to 1 / 10 of the original rate every 50 epochs. New fault data and prediction feedback are collected weekly, and online learning is performed using the stochastic gradient descent method with a learning rate of 0.0001. The model is fully evaluated and retrained every month.

[0056] Through the above improvements, the present invention has significantly improved the fault prediction accuracy and real-time performance, especially the ability to identify new faults has been significantly enhanced.

[0057] See also Figure 2 The present invention also provides a vehicle fault prediction method, comprising the following steps: Step 210: Acquire vehicle information of the vehicle to be predicted, wherein the vehicle information at least includes battery management system data, motor controller data, and on-board diagnostic system data; Step 220: Input the vehicle information into a fault prediction model trained according to a vehicle fault prediction model training method, and the fault prediction model outputs a fault prediction result corresponding to the vehicle to be predicted.

[0058] The fault prediction model is trained at least in the following manner.

[0059] Obtain sample data from different sensor sources, use a neural network model to extract features from the sample data, and perform weighted fusion of the extracted features based on an attention mechanism to obtain target features. Construct a fault knowledge graph and convert the fault knowledge graph into a low-dimensional vector representation. Perform adversarial training on a multimodal deep learning model using the target features and the low-dimensional vector representation to obtain a fault prediction model. The multimodal deep learning model includes a dual-branch neural network, one branch of the neural network is used for the target feature training, and the other branch of the neural network is used to process the low-dimensional vector representation; the dual-branch neural network realizes the fusion of different modal information based on a cross-modal attention mechanism.

[0060] For more information, please refer to the above description of the training method for the vehicle fault prediction model.

[0061] See also Figure 3 The present invention also provides a training device for a vehicle fault prediction model, including a feature extraction module 31, a graph conversion module 32, and a training module 33.

[0062] The feature extraction module 31 is used to obtain sample data from different sensor sources, use a neural network model to extract features from the sample data, and perform weighted fusion of the extracted features based on an attention mechanism to obtain target features.

[0063] For example, convolutional neural networks are used to extract spatial features from sensor data, long short-term memory networks are used to extract temporal features from time series data, and GCN models are used to extract features from Internet of Vehicles data; the spatial features and the temporal features are weightedly fused based on the attention mechanism.

[0064] The graph conversion module 32 is used to construct a fault knowledge graph and convert the fault knowledge graph into a low-dimensional vector representation.

[0065] For example, a fault knowledge graph is constructed based on a vehicle fault diagnosis manual and an expert knowledge base, and the fault knowledge graph is converted into a low-dimensional vector representation based on a graph embedding algorithm; wherein the fault knowledge graph is a dynamic knowledge graph.

[0066] The training module 33 is used to use the target features and the low-dimensional vector representation to perform adversarial training on the multimodal deep learning model to obtain a fault prediction model. The multimodal deep learning model includes a two-branch neural network, one branch neural network is used for the target feature training, and the other branch neural network is used to process the low-dimensional vector representation; the two-branch neural network realizes the fusion of different modal information based on the cross-modal attention mechanism.

[0067] The multimodal deep learning model adopts a Transformer model structure. A residual connection and a batch normalization layer are added to each fully connected layer of the multimodal deep learning model to improve the convergence and generalization ability of the multimodal deep learning model. A confidence calibration algorithm is used in the output layer of the multimodal deep learning model to adjust the probability distribution of the output fault prediction.

[0068] For details, please refer to the description of the training method of the vehicle fault prediction model, which will not be repeated here.

[0069] See also Figure 4 , Figure 4 The following is a business process diagram of a vehicle fault prediction method in an embodiment of the present invention. The vehicle is regularly connected to the vehicle's CAN bus, sensors and telemetry equipment to collect vehicle operation data in real time, including but not limited to engine status, fuel system, brake system, suspension system and other data. Data processing module: This module is based on the Apache Flink real-time stream processing engine to clean, filter and aggregate the received raw data and extract key features related to fault prediction.

[0070] Then the valid data is transmitted to the vehicle data aggregation unit, and the vehicle-side data is uploaded and monitored through the vehicle network. After receiving the vehicle-side data, the cloud performs data preprocessing, which is divided into offline processing and real-time processing. Offline, the cloud data is distributed to multiple models, the models are trained and fine-tuned, and the training results and fine-tuning parameters are fed back to the prediction model in real time. The prediction model then predicts the data, stores the prediction results, and predicts faults once a minute for the real-time data. When the probability of fault exceeds 0.8, an early warning is triggered. The early warning information is pushed to the driver and maintenance personnel through the vehicle display and mobile phone APP. The interface UI of the vehicle display includes real-time data display (real-time data display area), fault warning unit (fault warning display area), signal pool unit (signal display area), and logged-in account.

[0071] like Figure 5 As shown, the present invention also provides an electronic device, including: At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the above-mentioned vehicle fault prediction model training method or vehicle fault prediction method.

[0072] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the training method of the vehicle fault prediction model or the vehicle fault prediction method described above is implemented.

[0073] It can be understood that computer-readable storage media may include: any entity or device capable of carrying a computer program, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and software distribution media, etc.

[0074] In certain embodiments of the present invention, the electronic device may include a controller or a processor, and the controller is a single-chip microcomputer chip that integrates a processor, a memory, a communication module, etc. The processor may refer to a processor included in the controller. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0075] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0076] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0077] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a vehicle fault prediction model, characterized in that: include: Obtain sample data from different sensor sources, extract features from the sample data using a neural network model, and perform weighted fusion of the extracted features based on an attention mechanism to obtain target features; Constructing a fault knowledge graph and converting the fault knowledge graph into a low-dimensional vector representation; The multimodal deep learning model is adversarially trained using the target features and the low-dimensional vector representation to obtain a fault prediction model. The multimodal deep learning model includes a two-branch neural network, one branch of the neural network is used for the target feature training, and the other branch of the neural network is used to process the low-dimensional vector representation; the two-branch neural network realizes the fusion of different modal information based on the cross-modal attention mechanism.

2. The vehicle fault prediction model training method according to claim 1, characterized in that: The method of extracting features from the sample data using a neural network model and performing weighted fusion on the extracted features based on an attention mechanism to obtain target features includes: A convolutional neural network is used to extract spatial features from sensor data, and a long short-term memory network is used to extract temporal features from time series data. The spatial features and the temporal features are weightedly fused based on an attention mechanism.

3. The vehicle fault prediction model training method according to claim 2, characterized in that: The method of extracting time features from time series data using a long short-term memory network includes: Fourier transform is used to perform frequency domain analysis on the time series data and extract frequency domain features.

4. The training method of the vehicle fault prediction model according to claim 1, characterized in that: The constructing of the fault knowledge graph and converting the fault knowledge graph into a low-dimensional vector representation includes: A fault knowledge graph is constructed based on a vehicle fault diagnosis manual and an expert knowledge base, and the fault knowledge graph is converted into a low-dimensional vector representation based on a graph embedding algorithm; wherein the fault knowledge graph is a dynamic knowledge graph.

5. The vehicle fault prediction model training method according to claim 1, characterized in that: The using the target feature and the low-dimensional vector representation to perform adversarial training on the multimodal deep learning model includes: Residual connections and batch normalization layers are added to each fully connected layer of the multimodal deep learning model to improve the convergence and generalization capabilities of the multimodal deep learning model.

6. The vehicle fault prediction model training method according to claim 5, characterized in that: The multimodal deep learning model adopts the Transformer model structure.

7. The method for training a vehicle fault prediction model according to any one of claims 1 to 6, characterized in that: The method also includes adjusting the probability distribution of output fault prediction using a confidence calibration algorithm in the output layer of the multimodal deep learning model.

8. The method for training a vehicle fault prediction model according to any one of claims 1 to 6, characterized in that: The method also includes using a GCN model to extract features from Internet of Vehicles data when the multimodal deep learning model extracts features from the sample data.

9. A training device for a vehicle fault prediction model, characterized in that: include: A feature extraction module is used to obtain sample data from different sensor sources, extract features from the sample data using a neural network model, and perform weighted fusion of the extracted features based on an attention mechanism to obtain target features; A graph conversion module, used to construct a fault knowledge graph and convert the fault knowledge graph into a low-dimensional vector representation; A training module is used to use the target features and the low-dimensional vector representation to perform adversarial training on a multimodal deep learning model to obtain a fault prediction model, wherein the multimodal deep learning model includes a two-branch neural network, one branch of the neural network is used for the target feature training, and the other branch of the neural network is used to process the low-dimensional vector representation; the two-branch neural network realizes the fusion of different modal information based on a cross-modal attention mechanism.

10. A vehicle fault prediction method, characterized in that: include: Acquire vehicle information of a vehicle to be predicted, wherein the vehicle information at least includes battery management system data, motor controller data, and on-board diagnostic system data; The vehicle information is input into a fault prediction model trained by the vehicle fault prediction model training method according to any one of claims 1 to 8, and the fault prediction model outputs a fault prediction result corresponding to the vehicle to be predicted.

11. An electronic device, characterized in that: include: at least one processor; And at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method as claimed in any one of claims 1 to 8 or claim 10.

12. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 8 or claim 10 is executed.

Images