Unmanned vehicle interaction fault analysis method and device, terminal equipment and storage medium

Abstract

The invention discloses an unmanned vehicle interaction fault analysis method and device, terminal equipment and a storage medium. The method comprises the following steps: acquiring multi-source data of a to-be-detected unmanned vehicle; wherein the multi-source data comprises state data, environment sensing data, system data and fault recording data; generating fusion data of the to-be-detected unmanned vehicle according to the state data, the environment perception data, the system data and the fault record data; inputting the fusion data of the to-be-detected unmanned vehicle into an unmanned vehicle interaction fault analysis model, so that the unmanned vehicle interaction fault analysis model outputs fault data of the to-be-detected unmanned vehicle; wherein the fault data comprises a fault type, a fault priority and a fault probability. According to the invention, the accuracy of interaction fault analysis of the unmanned vehicle can be improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, and in particular to an unmanned vehicle interaction fault analysis method, device, terminal equipment and storage medium. Background Art

[0002] With the rapid development of artificial intelligence and sensor technology, driverless cars are gradually being used in actual production. However, in the actual operation process, driverless cars still face complex environmental changes and uncertain user interactions, which leads to various interaction failures. These interaction failures not only affect the operation efficiency of driverless cars, but also may bring safety hazards. Therefore, it is urgent to analyze the interaction failures of driverless cars.

[0003] Traditional interactive fault analysis of unmanned vehicles mainly relies on the vehicle's sensor data, such as the operating status data obtained by the sensor. The operating status data is used to determine whether the vehicle's operating status is abnormal, thereby determining whether the unmanned vehicle has a fault. However, this single fault analysis method lacks comprehensiveness and has a problem of insufficient accuracy in fault analysis. Summary of the invention

[0004] The embodiments of the present invention provide an unmanned vehicle interaction fault analysis method, apparatus, terminal equipment and storage medium, which can improve the accuracy of unmanned vehicle interaction fault analysis.

[0005] An embodiment of the present invention provides an unmanned vehicle interaction fault analysis method, comprising:

[0006] Acquire multi-source data of the unmanned vehicle to be detected; wherein the multi-source data includes: status data, environmental perception data, system data and fault record data;

[0007] Generate fusion data of the unmanned vehicle to be detected based on status data, environmental perception data, system data and fault record data;

[0008] The fusion data of the unmanned vehicle to be detected is input into the unmanned vehicle interactive fault analysis model, so that the unmanned vehicle interactive fault analysis model outputs the fault data of the unmanned vehicle to be detected; wherein the fault data includes: fault type, fault priority and fault probability.

[0009] Further, the state data includes vehicle speed, vehicle acceleration, vehicle position and state timestamp;

[0010] The environmental perception data includes the distance between the vehicle and the obstacle in front, the position of the obstacle in front of the vehicle, the number of obstacles in front of the vehicle and the perception timestamp;

[0011] The system data includes vehicle CPU usage, vehicle memory usage and system timestamp;

[0012] The fault record data includes: a vehicle historical fault time series, a vehicle historical fault type, a vehicle historical fault handling measure and a vehicle historical fault handling result; wherein the vehicle historical fault time series includes a number of historical fault timestamps.

[0013] Furthermore, the generating of fusion data of the unmanned vehicle to be detected according to the state data, the environment perception data, the system data and the fault record data includes:

[0014] Extract the state timestamp, perception timestamp, system timestamp and vehicle historical fault time series of the state data, environment perception data, system data and fault record data respectively, and generate the fusion time series features of multi-source data according to the state timestamp, perception timestamp, system timestamp and vehicle historical fault time series;

[0015] Extracting a first spatial feature of the distance between the vehicle and the obstacle in front, and a second spatial feature of the position of the obstacle in front of the vehicle, and generating a fused spatial feature based on the first spatial feature and the second spatial feature;

[0016] Extracting a first quantitative feature of vehicle speed, a second quantitative feature of vehicle acceleration, a third quantitative feature of the number of obstacles in front of the vehicle, a fourth quantitative feature of vehicle CPU usage, and a fifth quantitative feature of vehicle memory usage, and generating a fused quantitative feature based on the first quantitative feature, the second quantitative feature, the third quantitative feature, the fourth quantitative feature, and the fifth quantitative feature;

[0017] Extracting a first attribute feature of the vehicle location, a second attribute feature of the vehicle historical fault type, a third attribute feature of the vehicle historical fault handling measures, and a fourth attribute feature of the vehicle historical fault handling results, and generating a fused attribute feature based on the first attribute feature, the second attribute feature, the third attribute feature, and the fourth attribute feature;

[0018] A multidimensional feature model is constructed based on the fusion of temporal features, spatial features, quantitative features and attribute features, and fusion data of the unmanned vehicle to be detected is generated based on the multidimensional feature model.

[0019] Furthermore, the construction of the unmanned vehicle interaction fault analysis model includes:

[0020] Acquire a number of historical multi-source data samples; wherein each historical multi-source data sample includes: a historical state data sample, a historical environment perception data sample, a historical system data sample, and a historical fault record data sample; each historical multi-source data sample is annotated with corresponding historical fault data;

[0021] For each historical multi-source data sample, a historical fusion data sample is generated according to each data sample in the historical multi-source data sample; wherein the historical fusion data sample is annotated with corresponding historical fault data;

[0022] An initial unmanned vehicle interaction fault analysis model is constructed, and historical fusion data samples are used as the input of the initial unmanned vehicle interaction fault analysis model, and historical fault data are used as the output of the initial unmanned vehicle interaction fault analysis model. The initial unmanned vehicle interaction fault analysis model is iteratively trained until the initial unmanned vehicle interaction fault analysis model converges, thereby generating an unmanned vehicle interaction fault analysis model.

[0023] Furthermore, after obtaining multi-source data of the unmanned vehicle to be detected, the method further includes:

[0024] Convert state data, environment perception data, system data and fault record data into the same data format to obtain multi-source data after format conversion;

[0025] Performing data cleaning on the multi-source data after format conversion to obtain the multi-source data after data cleaning; wherein the data cleaning includes: missing value completion, outlier detection and duplicate data removal;

[0026] According to the state timestamp of the state data, the perception timestamp of the environment perception data, the system timestamp of the system data and the vehicle historical fault time series of the fault record data in the multi-source data after data cleaning, the state data, the environment perception data, the system data and the fault record data after data cleaning are synchronized in time to obtain the preprocessed multi-source data;

[0027] The generating of fusion data of the unmanned vehicle to be detected according to the state data, the environment perception data, the system data and the fault record data includes:

[0028] Generate fused data of the unmanned vehicle to be detected based on the processed multi-source data.

[0029] Furthermore, the multi-source data also includes: user usage data;

[0030] The generating of fusion data of the unmanned vehicle to be detected according to the state data, the environment perception data, the system data and the fault record data includes:

[0031] Generate fusion data of the unmanned vehicle to be tested based on status data, environmental perception data, system data, fault record data and user usage data.

[0032] Furthermore, it also includes:

[0033] Generate fault warning information according to the fault data of the unmanned vehicle to be detected; wherein the fault warning information includes the fault type, fault probability and fault handling solution; the fault handling solution is determined according to the fault type, fault probability and preset fault handling solution;

[0034] The fault warning information is sent to the unmanned vehicle operation and maintenance personnel according to the fault priority.

[0035] Based on the above method embodiment, the present invention provides a corresponding device embodiment;

[0036] An embodiment of the present invention provides an unmanned vehicle interactive fault analysis device, including: a multi-source data acquisition module, a data fusion module and a fault analysis module;

[0037] The multi-source data acquisition module is used to acquire multi-source data of the unmanned vehicle to be detected; wherein the multi-source data includes: status data, environmental perception data, system data and fault record data;

[0038] The data fusion module is used to generate fusion data of the unmanned vehicle to be detected based on the status data, environmental perception data, system data and fault record data;

[0039] The fault analysis module is used to input the fusion data of the unmanned vehicle to be detected into the unmanned vehicle interactive fault analysis model, so that the unmanned vehicle interactive fault analysis model outputs the fault data of the unmanned vehicle to be detected; wherein the fault data includes: fault type, fault priority and fault probability.

[0040] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the unmanned vehicle interaction fault analysis method described in the above-mentioned embodiment of the invention.

[0041] Another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the unmanned vehicle interaction fault analysis method described in the above-mentioned invention embodiment.

[0042] The following beneficial effects are achieved by implementing the present invention:

[0043] The present invention provides an unmanned vehicle interactive fault analysis method, device, terminal equipment and storage medium. The method obtains multi-source data of an unmanned vehicle to be detected including status data, environmental perception data, system data and fault record data, and then fuses the obtained multi-source data of the unmanned vehicle to be detected to obtain fused data of the unmanned vehicle to be detected; inputs the fused data of the unmanned vehicle to be detected into an unmanned vehicle interactive fault analysis model, so that the unmanned vehicle interactive fault analysis model outputs the fault type, fault priority and fault probability of the unmanned vehicle to be detected; and by obtaining multi-source data and fusing the multi-source data, the problem of inaccurate analysis results of unmanned vehicle fault analysis using single operating data in the prior art is solved, and the accuracy of unmanned vehicle interactive fault analysis is improved from the data source. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of an unmanned vehicle interactive fault analysis method provided by one embodiment of the present invention.

[0045] Figure 2 It is a structural schematic diagram of an unmanned vehicle interactive fault analysis device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] like Figure 1 As shown, a method for analyzing an unmanned vehicle interaction fault is provided in one embodiment of the present invention, comprising:

[0048] Step S1: Acquire multi-source data of the unmanned vehicle to be detected; wherein the multi-source data includes: state data, environmental perception data, system data and fault record data;

[0049] Step S2: Generate fusion data of the unmanned vehicle to be detected according to the state data, environmental perception data, system data and fault record data;

[0050] Step S3: Input the fusion data of the unmanned vehicle to be detected into the unmanned vehicle interactive fault analysis model, so that the unmanned vehicle interactive fault analysis model outputs the fault data of the unmanned vehicle to be detected; wherein the fault data includes: fault type, fault priority and fault probability.

[0051] For step S1, multi-source data of the unmanned vehicle to be detected is obtained, and the multi-source data includes at least status data, environmental perception data, system data and fault record data; among them, the status data and environmental perception data can be obtained based on the vehicle sensors of the unmanned vehicle, the vehicle cameras of the unmanned vehicle, the unmanned vehicle radars, the unmanned vehicle lidars, etc.; the system data can be obtained based on the system log data of the unmanned vehicle; fault record data.

[0052] In a preferred embodiment, the multi-source data also includes: user usage data.

[0053] Specifically, user usage data includes user ratings and review texts of driverless car services, and the user usage data is mainly obtained through mobile applications or website interfaces associated with driverless cars.

[0054] In a preferred embodiment, the status data includes vehicle speed, vehicle acceleration, vehicle position and status timestamp; the environmental perception data includes the distance between the vehicle and the obstacle in front, the position of the obstacle in front of the vehicle, the number of obstacles in front of the vehicle and the perception timestamp; the system data includes vehicle CPU usage, vehicle memory usage and system timestamp; the fault record data includes: vehicle historical fault time series, vehicle historical fault type, vehicle historical fault handling measures and vehicle historical fault handling results; wherein, the vehicle historical fault time series includes several historical fault timestamps.

[0055] Specifically, for example, the state data of the unmanned vehicle to be processed includes: the vehicle speed is 60km / h, the vehicle acceleration is 0.5g, the vehicle position coordinates are (x1, y1), and the state timestamp is 10:00:00; wherein the state timestamp corresponds to the collection time when the state data is collected; the environmental perception data includes: the distance between the vehicle and the obstacle in front is 5m, the position coordinates of the obstacle in front of the vehicle are (x2, y2), the number of obstacles in front of the vehicle is 1, and the perception timestamp is 10:00:00; wherein the distance between the vehicle and the obstacle in front is based on the position of the obstacle in front of the vehicle, the number of obstacles in front of the vehicle, and the vehicle position. Determine; system data includes: vehicle CPU usage is 90%, vehicle memory usage is 70%, system timestamp is 10:00:00; fault record data includes: vehicle historical fault time series is [(01:00:00), (02:00:00)], vehicle historical fault type is [brake delay, brake failure], vehicle historical fault handling measures [adjust brake response time, repair brake system], vehicle historical fault handling results are [fault repair, fault repair]; wherein, vehicle fault type, vehicle historical fault handling measures and vehicle historical fault handling results correspond to vehicle historical fault time series. Preferably, user usage data is: user rating star rating of 3 stars and user comment text 'there is sudden braking during driving, poor experience'; wherein, the user rating star rating is 0 stars at the lowest and 5 stars at the highest.

[0056] For step S2, fusion data of the unmanned vehicle to be detected is generated according to the status data, environmental perception data, system data and fault record data.

[0057] In a preferred embodiment, after obtaining multi-source data of the unmanned vehicle to be detected, it also includes: converting the status data, environmental perception data, system data and fault record data into the same data format to obtain multi-source data after format conversion; performing data cleaning on the multi-source data after format conversion to obtain multi-source data after data cleaning; wherein, the data cleaning includes: missing value completion, outlier detection and duplicate data deduplication; according to the status timestamp of the status data, the perception timestamp of the environmental perception data, the system timestamp of the system data and the vehicle historical fault time series of the fault record data in the multi-source data after data cleaning, the status data, environmental perception data, system data and fault record data after data cleaning are time synchronized to obtain pre-processed multi-source data; the generating of fused data of the unmanned vehicle to be detected based on the status data, environmental perception data, system data and fault record data includes: generating fused data of the unmanned vehicle to be detected based on the processed multi-source data.

[0058] Specifically, the state data, environmental perception data, system data, fault record data and user usage data are converted into the same data format, such as JSON format or CSV format, to obtain multi-source data after format conversion. The multi-source data after format conversion is cleaned, including missing value filling, outlier detection and duplicate data removal. Among them, missing value filling can be done by mean filling or deletion, and outlier detection can use the 3σ principle to eliminate outliers. The multi-source data after data cleaning is synchronized in time, that is, according to the state timestamp of the state data, the perception timestamp of the environmental perception data, the system timestamp of the system data, the vehicle historical fault time series of the fault record data and the interaction timestamp of the user usage data in the multi-source data after data cleaning, the state data, environmental perception data, system data, fault record data and user usage data after data cleaning are synchronized in time to ensure the time sequence consistency between the data and obtain the pre-processed multi-source data. The fusion data of the unmanned vehicle to be detected is generated according to the processed multi-source data.

[0059] In a preferred embodiment, the generating of fused data of the unmanned vehicle to be detected based on the state data, environmental perception data, system data and fault record data includes: extracting the state timestamp, perception timestamp, system timestamp and vehicle historical fault time series of the state data, environmental perception data, system data and fault record data respectively, and generating fused time series features of multi-source data based on the state timestamp, perception timestamp, system timestamp and vehicle historical fault time series; extracting the first spatial feature of the distance between the vehicle and the obstacle in front, the second spatial feature of the position of the obstacle in front of the vehicle, and generating fused spatial features based on the first spatial feature and the second spatial feature; extracting the first quantitative feature of the vehicle speed, the second quantitative feature of the vehicle acceleration, the obstacle in front of the vehicle The third quantitative feature of the quantity, the fourth quantitative feature of the vehicle CPU usage and the fifth quantitative feature of the vehicle memory usage are extracted, and a fused quantitative feature is generated based on the first quantitative feature, the second quantitative feature, the third quantitative feature, the fourth quantitative feature and the fifth quantitative feature; the first attribute feature of the vehicle location, the second attribute feature of the vehicle historical fault type, the third attribute feature of the vehicle historical fault handling measures and the fourth attribute feature of the vehicle historical fault handling results are extracted, and a fused attribute feature is generated based on the first attribute feature, the second attribute feature, the third attribute feature and the fourth attribute feature; a multidimensional feature model is constructed based on the fused temporal features, the fused spatial features, the fused quantitative features and the fused attribute features, and the fused data of the unmanned vehicle to be detected is generated based on the multidimensional feature model.

[0060] Specifically, classification features are extracted and fused for different data in multi-source data. First, extract time series features and determine fused time series features, including: extracting state timestamps, perception timestamps, system timestamps, vehicle historical fault time series and interaction timestamps of state data, environmental perception data, system data, fault record data and user usage data respectively, and generating fused time series features of multi-source data according to state timestamps, perception timestamps, system timestamps, vehicle historical fault time series and interaction timestamps. Secondly, extract spatial features and determine fused spatial features, including: extracting a first spatial feature of the distance between the vehicle and the obstacle in front, a second spatial feature of the position of the obstacle in front of the vehicle, and generating fused spatial features according to the first spatial feature and the second spatial feature. Further, extract quantity features and determine fused quantity features, including: extracting a first quantity feature of vehicle speed, a second quantity feature of vehicle acceleration, a third quantity feature of the number of obstacles in front of the vehicle, a fourth quantity feature of vehicle CPU usage and a fifth quantity feature of vehicle memory usage, and generating fused quantity features according to the first quantity feature, the second quantity feature, the third quantity feature, the fourth quantity feature and the fifth quantity feature. Then extract attribute features and determine fused attribute features. The method includes: extracting a first attribute feature of the vehicle location, a second attribute feature of the vehicle's historical fault type, a third attribute feature of the vehicle's historical fault handling measures, and a fourth attribute feature of the vehicle's historical fault handling results, and generating a fused attribute feature based on the first attribute feature, the second attribute feature, the third attribute feature, and the fourth attribute feature. Preferably, extract the first text feature of the user rating star and the second text feature of the user comment text in the user usage data, determine the user sentiment score based on the first text feature and the second text feature, and use the user sentiment score as the fused text feature; exemplarily, determine the weight of the first text feature based on the user rating star, determine the weight of the second text feature based on the keywords in the user comment text, and determine the fused text feature based on the weight of the first text feature and the weight of the second text feature; preferably, natural language processing (NLP) technology can also be used to perform sentiment analysis on the first text feature and the second text feature, and extract keywords to determine the user sentiment score. Finally, a multidimensional feature model is constructed based on the fusion of temporal features, spatial features, quantitative features, attribute features and text features, that is, the fusion of temporal features, spatial features, quantitative features, attribute features and text features are integrated into a unified multidimensional data model, and the fusion data of the unmanned vehicle to be detected is generated based on the multidimensional feature model. For example, the fusion data of the unmanned vehicle to be detected is as follows:

[0061] [timestamp, vehicle speed, acceleration, vehicle position, distance between vehicle and obstacle in front, number of obstacles in front of vehicle, coordinates of obstacle position in front of vehicle, user sentiment score, vehicle CPU usage, vehicle memory usage, historical fault mark];

[0062] [10:00:00,60km / h,0.5g,(x1,y1),5m,(x2,y2),1,0.7,90%,70%,has a history of brake failure].

[0063] Preferably, before the fused data of the unmanned vehicle to be detected is input into the unmanned vehicle interactive fault analysis model, the fused data is standardized so that its mean is 0 and its variance is 1. PCA is used to reduce the dimension of the high-dimensional feature vector in the fused data, extract the main features, and reduce the computational complexity. Exemplarily, the feature dimension of the fused data is 100 dimensions, which is reduced to 20 dimensions by PCA, while retaining 95% of the information. Among them, PCA is a principal component analysis algorithm, which is used to reduce the dimension of the data, extract the main principal components and reduce redundant information.

[0064] In a preferred embodiment, the construction of the unmanned vehicle interaction fault analysis model includes: obtaining a number of historical multi-source data samples; wherein each historical multi-source data sample includes: a historical status data sample, a historical environmental perception data sample, a historical system data sample and a historical fault record data sample; each historical multi-source data sample is annotated with corresponding historical fault data; for each historical multi-source data sample, a historical fusion data sample is generated according to each data sample in the historical multi-source data sample; wherein the historical fusion data sample is annotated with corresponding historical fault data; constructing an initial unmanned vehicle interaction fault analysis model, using the historical fusion data samples as the input of the initial unmanned vehicle interaction fault analysis model, using the historical fault data as the output of the initial unmanned vehicle interaction fault analysis model, and iteratively training the initial unmanned vehicle interaction fault analysis model until the initial unmanned vehicle interaction fault analysis model converges to generate an unmanned vehicle interaction fault analysis model.

[0065] Specifically, a number of historical multi-source data samples are obtained; each of which includes: a historical state data sample, a historical environment perception data sample, a historical system data sample, and a historical fault record data sample; each of which is annotated with corresponding historical fault data. For each historical multi-source data sample, a historical fusion data sample is generated according to each data sample in the historical multi-source data sample; the historical fusion data sample is annotated with corresponding historical fault data.

[0066] An initial unmanned vehicle interaction fault analysis model is constructed. The initial unmanned vehicle interaction fault analysis model is a deep learning model that combines convolutional neural network (CNN) and long short-term memory network (LSTM). Among them, the CNN network is a network used to extract the spatial relationship of features and extract deep-level features; the LSTM network is a network used to process time series features and capture dynamic changes in time. The initial unmanned vehicle interaction fault analysis model includes an input layer, a convolution layer, a pooling layer, an LSTM layer, a fully connected layer, and an output layer. Among them, the input layer is used to accept the feature vector sequence after dimensionality reduction; the convolution layer is used to extract local features and capture the correlation features between features, such as time series correlation features, spatial correlation features, quantity correlation features, attribute correlation features, and text correlation features; the pooling layer is used to reduce feature dimensions and prevent overfitting; the LSTM layer is used to process time series data and capture the time dynamic changes of features; the fully connected layer is used to map high-level features to fault category probabilities; and the output layer is used to output predicted fault data.

[0067] The initial unmanned vehicle interactive fault analysis model is iteratively trained with historical fusion data samples as the input of the initial unmanned vehicle interactive fault analysis model and historical fault data as the output of the initial unmanned vehicle interactive fault analysis model. During the iterative training process, the historical fusion data samples pass through the input layer, convolution layer, pooling layer, LSTM layer, fully connected layer and output layer in sequence, and output the historical fault data to the output layer. In addition, during the iterative training process, the difference between the predicted historical fault data output by the output layer and the annotated real historical fault data is calculated by the cross entropy loss function to determine the prediction loss of each iteration process, and the gradient is calculated based on the loss to update the parameters of the initial unmanned vehicle interactive fault analysis model. Furthermore, an optimization algorithm, such as the Adam algorithm, is used to iteratively update the model parameters until the loss function of the initial unmanned vehicle interactive fault analysis model decreases by less than the preset loss threshold in several consecutive iterations, or the prediction accuracy reaches the expected performance index, such as when the accuracy reaches 95%, it is considered that the initial unmanned vehicle interactive fault analysis model converges, and the training of the initial unmanned vehicle interactive fault analysis model is stopped at this time to generate an unmanned vehicle interactive fault analysis model.

[0068] Preferably, new sample data is collected regularly, and the unmanned vehicle interaction fault analysis model is incrementally trained based on the newly collected sample data, so that the unmanned vehicle interaction fault analysis model can achieve self-iterative optimization and improve the real-time prediction of the model.

[0069] For step S3, the fusion data of the unmanned vehicle to be detected after the above processing is input into the unmanned vehicle interactive fault analysis model, so that the unmanned vehicle interactive fault analysis model outputs the fault data of the unmanned vehicle to be detected, wherein the fault data includes the fault type, fault priority and fault probability.

[0070] Preferably, the calculation of the fault probability is jointly determined based on the probability of occurrence of the faulty equipment of each fault type in the fault data; the fault priority is determined based on the probability of occurrence of the faulty equipment and the impact weight, and the calculation formula is: fault priority = probability of occurrence of the faulty equipment × impact weight. Exemplary: Fault A (brake system failure): probability: 80% impact weight: 10 priority score: 8.0; Fault B (sensor abnormality): probability: 60% impact weight: 8 priority score: 4.8. Priority sorting result: Fault A>Fault B.

[0071] In a preferred embodiment, it also includes: generating fault warning information based on the fault data of the unmanned vehicle to be detected; wherein the fault warning information includes the fault type, fault probability and fault handling plan; the fault handling plan is determined according to the fault type and the preset fault handling plan; and the fault warning information is sent to the unmanned vehicle operation and maintenance personnel according to the fault priority.

[0072] Specifically, fault warning information is generated based on the fault data of the unmanned vehicle to be detected, and the fault warning information includes the fault type, fault probability and fault handling solution; the fault handling solution is determined based on the fault type, fault probability and preset fault handling solution. Exemplarily, several corresponding fault handling solutions are selected from the preset fault handling solutions based on each fault type in the fault data and the fault probability corresponding to each fault type. Among them, the preset fault handling solution includes several fault types and corresponding fault handling solutions under the fault probability. The fault warning information is sent to the unmanned vehicle operation and maintenance personnel according to the fault priority for reference by the unmanned vehicle operation and maintenance personnel and for maintenance and repair of the unmanned vehicle.

[0073] Based on the above method embodiment, the present invention provides a corresponding device embodiment.

[0074] like Figure 2 As shown, an embodiment of the present invention provides an unmanned vehicle interactive fault analysis device, including: a multi-source data acquisition module, a data fusion module and a fault analysis module;

[0075] The multi-source data acquisition module is used to acquire multi-source data of the unmanned vehicle to be detected; wherein the multi-source data includes: status data, environmental perception data, system data and fault record data;

[0076] The data fusion module is used to generate fusion data of the unmanned vehicle to be detected based on the status data, environmental perception data, system data and fault record data;

[0077] The fault analysis module is used to input the fusion data of the unmanned vehicle to be detected into the unmanned vehicle interactive fault analysis model, so that the unmanned vehicle interactive fault analysis model outputs the fault data of the unmanned vehicle to be detected; wherein the fault data includes: fault type, fault priority and fault probability.

[0078] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0079] Those skilled in the art can clearly understand that for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0080] Based on the above method item embodiments, the present invention provides corresponding terminal device item embodiments.

[0081] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, an unmanned vehicle interaction fault analysis method as described in any one of the present invention is implemented.

[0082] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0083] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0084] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0085] Based on the above method item embodiments, the present invention provides a corresponding storage medium item embodiment.

[0086] An embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute an unmanned vehicle interaction fault analysis method as described in any one of the present inventions.

[0087] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium.

[0088] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for analyzing interactive failures of unmanned vehicles, characterized in that: include: Acquire multi-source data of the unmanned vehicle to be detected; wherein the multi-source data includes: status data, environmental perception data, system data and fault record data; Generate fusion data of the unmanned vehicle to be detected based on status data, environmental perception data, system data and fault record data; The fusion data of the unmanned vehicle to be detected is input into the unmanned vehicle interactive fault analysis model, so that the unmanned vehicle interactive fault analysis model outputs the fault data of the unmanned vehicle to be detected; wherein the fault data includes: fault type, fault priority and fault probability.

2. The unmanned vehicle interaction fault analysis method according to claim 1, characterized in that: The state data includes vehicle speed, vehicle acceleration, vehicle position and state timestamp; The environmental perception data includes the distance between the vehicle and the obstacle in front, the position of the obstacle in front of the vehicle, the number of obstacles in front of the vehicle and the perception timestamp; The system data includes vehicle CPU usage, vehicle memory usage and system timestamp; The fault record data includes: a vehicle historical fault time series, a vehicle historical fault type, a vehicle historical fault handling measure and a vehicle historical fault handling result; wherein the vehicle historical fault time series includes a number of historical fault timestamps.

3. The unmanned vehicle interaction fault analysis method according to claim 2, characterized in that: The generating of fusion data of the unmanned vehicle to be detected according to the state data, the environment perception data, the system data and the fault record data includes: Extract the state timestamp, perception timestamp, system timestamp and vehicle historical fault time series of the state data, environment perception data, system data and fault record data respectively, and generate the fusion time series features of multi-source data according to the state timestamp, perception timestamp, system timestamp and vehicle historical fault time series; Extracting a first spatial feature of the distance between the vehicle and the obstacle in front, and a second spatial feature of the position of the obstacle in front of the vehicle, and generating a fused spatial feature based on the first spatial feature and the second spatial feature; Extracting a first quantitative feature of vehicle speed, a second quantitative feature of vehicle acceleration, a third quantitative feature of the number of obstacles in front of the vehicle, a fourth quantitative feature of vehicle CPU usage, and a fifth quantitative feature of vehicle memory usage, and generating a fused quantitative feature based on the first quantitative feature, the second quantitative feature, the third quantitative feature, the fourth quantitative feature, and the fifth quantitative feature; Extracting a first attribute feature of the vehicle location, a second attribute feature of the vehicle historical fault type, a third attribute feature of the vehicle historical fault handling measures, and a fourth attribute feature of the vehicle historical fault handling results, and generating a fused attribute feature based on the first attribute feature, the second attribute feature, the third attribute feature, and the fourth attribute feature; A multidimensional feature model is constructed based on the fusion of temporal features, spatial features, quantitative features and attribute features, and fusion data of the unmanned vehicle to be detected is generated based on the multidimensional feature model.

4. The unmanned vehicle interaction fault analysis method according to claim 1, characterized in that: The construction of the unmanned vehicle interaction fault analysis model includes: Acquire a number of historical multi-source data samples; wherein each historical multi-source data sample includes: a historical state data sample, a historical environment perception data sample, a historical system data sample, and a historical fault record data sample; each historical multi-source data sample is annotated with corresponding historical fault data; For each historical multi-source data sample, a historical fusion data sample is generated according to each data sample in the historical multi-source data sample; wherein the historical fusion data sample is annotated with corresponding historical fault data; An initial unmanned vehicle interaction fault analysis model is constructed, and historical fusion data samples are used as the input of the initial unmanned vehicle interaction fault analysis model, and historical fault data are used as the output of the initial unmanned vehicle interaction fault analysis model. The initial unmanned vehicle interaction fault analysis model is iteratively trained until the initial unmanned vehicle interaction fault analysis model converges, thereby generating an unmanned vehicle interaction fault analysis model.

5. The unmanned vehicle interaction fault analysis method according to claim 2, characterized in that: After obtaining multi-source data of the unmanned vehicle to be detected, it also includes: Convert state data, environmental perception data, system data and fault record data into the same data format to obtain multi-source data after format conversion; Performing data cleaning on the multi-source data after format conversion to obtain the multi-source data after data cleaning; wherein the data cleaning includes: missing value completion, outlier detection and duplicate data removal; According to the state timestamp of the state data, the perception timestamp of the environment perception data, the system timestamp of the system data and the vehicle historical fault time series of the fault record data in the multi-source data after data cleaning, the state data, the environment perception data, the system data and the fault record data after data cleaning are synchronized in time to obtain the preprocessed multi-source data; The generating of fusion data of the unmanned vehicle to be detected according to the state data, the environment perception data, the system data and the fault record data includes: Generate fused data of the unmanned vehicle to be detected based on the processed multi-source data.

6. The unmanned vehicle interaction fault analysis method according to claim 1, characterized in that: The multi-source data also includes: user usage data; The generating of fusion data of the unmanned vehicle to be detected according to the state data, the environment perception data, the system data and the fault record data includes: Generate fusion data of the unmanned vehicle to be tested based on status data, environmental perception data, system data, fault record data and user usage data.

7. The unmanned vehicle interaction fault analysis method according to claim 1, characterized in that: Also includes: Generate fault warning information according to the fault data of the unmanned vehicle to be detected; wherein the fault warning information includes the fault type, fault probability and fault handling solution; the fault handling solution is determined according to the fault type, fault probability and preset fault handling solution; The fault warning information is sent to the unmanned vehicle operation and maintenance personnel according to the fault priority.

8. An unmanned vehicle interactive fault analysis device, characterized in that: include: Multi-source data acquisition module, data fusion module and fault analysis module; The multi-source data acquisition module is used to acquire multi-source data of the unmanned vehicle to be detected; wherein the multi-source data includes: status data, environmental perception data, system data and fault record data; The data fusion module is used to generate fusion data of the unmanned vehicle to be detected based on the status data, environmental perception data, system data and fault record data; The fault analysis module is used to input the fusion data of the unmanned vehicle to be detected into the unmanned vehicle interactive fault analysis model, so that the unmanned vehicle interactive fault analysis model outputs the fault data of the unmanned vehicle to be detected; wherein the fault data includes: fault type, fault priority and fault probability.

9. A terminal device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements an unmanned vehicle interactive fault analysis method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute an unmanned vehicle interactive fault analysis method as described in any one of claims 1 to 7.

Images