A vehicle repair diagnosis method, system, device, and medium

Abstract

The application relates to vehicle diagnosis technology and discloses a vehicle maintenance diagnosis method, system, device and medium. The method comprises the following steps: in response to received fault information, processing the fault information through a double-channel extraction module to obtain first output information; wherein the double-channel extraction module is used to realize bidirectional mapping between fault codes and maintenance structured data; processing the first output information, the fault information and vehicle basic information as diagnosis input information of a multi-task learning module, so that the multi-task learning module extracts task common features based on the diagnosis input information and generates a maintenance guide based on the task common features; wherein the multi-task learning module comprises a plurality of task processing modules which simultaneously process tasks based on the task common features. The application can quickly locate faults through the double-channel extraction module, and then automatically generate a maintenance guide in combination with the multi-task learning module, thereby improving maintenance efficiency and reducing the maintenance threshold.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle diagnosis, and in particular to a vehicle maintenance diagnosis method, system, device and medium. BACKGROUND

[0002] With the continuous improvement of the level of automobile electrification and intelligence, the modern automobile maintenance industry puts forward more strict requirements on the professional technical ability of practitioners. The traditional automobile maintenance mode mainly relies on paper maintenance manual and circuit diagram, and its inherent defects mainly manifest in the following aspects: 1. Insufficient information retrieval efficiency: technicians need to manually consult a large amount of paper materials in the fault troubleshooting process. This retrieval method not only consumes time and effort, but also is prone to finding errors. 2. Technical update is not timely: in the face of the iteration of new car models and technological innovation, the update cycle of paper maintenance manual is relatively long, which makes it difficult for maintenance personnel to master the latest maintenance technology and solutions in time. 3. Diagnosis method is too experiential: in the absence of a structured knowledge support system, maintenance diagnosis mainly relies on personal experience, which may lead to the omission of key maintenance steps or the failure of fault judgment. 4. Poor cross-model adaptability: due to the significant differences in the formats and technical terms of maintenance manuals adopted by different automobile manufacturers, maintenance personnel need to spend a lot of effort to learn and adapt to multiple knowledge systems, which objectively increases the cost and difficulty of professional training. SUMMARY

[0003] In view of this, the embodiments of the present application provide a vehicle maintenance diagnosis method, system, device and medium.

[0004] In a first aspect, the embodiments of the present application provide a vehicle maintenance diagnosis method, comprising:

[0005] In response to the received fault information, processing the fault information through a double-channel extraction module to obtain first output information; wherein the double-channel extraction module is used to realize the bidirectional mapping between fault codes and maintenance structured data;

[0006] After processing the first output information, the fault information and the vehicle basic information, taking them as diagnosis input information of a multi-task learning module, so that the multi-task learning module extracts task common features based on the diagnosis input information, and generates a maintenance guide based on the task common features; wherein the multi-task learning module includes a plurality of task processing modules that simultaneously process tasks based on the task common features.

[0007] In an optional implementation, the fault information is fault code or fault language description information; the double-channel extraction module includes a fault code-context channel and a context-fault code channel;

[0008] If the fault information is the fault code, the fault information is processed by the double-channel extraction module to obtain first output information, including:

[0009] The fault code is processed by the fault code-context channel to obtain corresponding structured context information; wherein the structured context information includes fault phenomenon and fault reason;

[0010] If the fault information is the fault language description information, the fault information is processed by the double-channel extraction module to obtain first output information, including:

[0011] The fault language description information is processed by the context-fault code channel to obtain corresponding fault code and fault code interpretation.

[0012] In an optional embodiment, the fault information and vehicle basic information are processed, including:

[0013] The fault language description information in the fault information is processed by a text feature extraction submodule to obtain semantic features;

[0014] Based on the vehicle basic information, a target vehicle circuit diagram and a structure diagram are determined, and image visual information of the target vehicle circuit diagram and the structure diagram is extracted by an image feature extraction submodule to obtain image features.

[0015] In an optional embodiment, the multi-task learning module includes a shared feature layer output layer, a fault phenomenon extraction layer, a solution generation layer, and a circuit inspection and component test layer;

[0016] The multi-task learning module is caused to extract task common features based on the diagnostic input information, and to generate a repair guide based on the task common features, including:

[0017] The diagnostic input information is input into the shared feature layer output layer to generate task common features;

[0018] The task common features are input into the fault phenomenon extraction layer, the solution generation layer, and the circuit inspection and component test layer according to feature types respectively to generate fault phenomenon results, repair step information, and detection step information.

[0019] In an optional embodiment, the training process of the multi-task learning module includes:

[0020] The extracted vehicle repair text features and vehicle repair image features are input into a shared feature extraction layer to obtain task common features;

[0021] The task-shared features are respectively input into an output layer of each task according to feature types to obtain a prediction result of each task;

[0022] According to a loss between the prediction result and a labeled result, parameters of the shared feature extraction layer and the output layer of each task are adjusted until the loss between the prediction result and the labeled result reaches a stop condition, and a multi-task learning model is obtained.

[0023] In an optional implementation, the fault code-context channel uses a sequence labeling model as a base model to obtain by training.

[0024] The context-fault code channel uses a similarity model as a base model to obtain by training.

[0025] In a second aspect, an embodiment of the present application provides a vehicle maintenance diagnosis system, comprising: a double-channel extraction module and a multi-task learning module;

[0026] The double-channel extraction module is configured to process received fault information to obtain first output information, and is configured to realize bidirectional mapping between fault codes and maintenance structured data.

[0027] The multi-task learning module is configured to process the first output information, the fault information and vehicle basic information as diagnosis input information of the multi-task learning module, so that the multi-task learning module extracts task-shared features based on the diagnosis input information and generates a maintenance guide based on the task-shared features, and the multi-task learning module comprises a plurality of task processing modules that simultaneously process tasks based on the task-shared features.

[0028] In an optional implementation, the system further comprises a multi-modal feature extraction module, and the multi-modal feature extraction module comprises a text feature extraction submodule and an image feature extraction submodule.

[0029] The text feature extraction submodule is configured to process fault language description information in the fault information to obtain semantic features.

[0030] The image feature extraction submodule is configured to extract image visual information of a vehicle circuit diagram and a vehicle component structure diagram to obtain image features.

[0031] In a third aspect, an embodiment of the present application provides a terminal device, comprising a processor and a memory, the memory stores a computer program, and the processor is configured to execute the computer program to implement the vehicle maintenance diagnosis method in the foregoing embodiments.

[0032] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium storing a computer program, which, when executed on a processor, implements the vehicle maintenance diagnosis method according to the foregoing embodiments.

[0033] The embodiments of the present application have the following beneficial effects: the present application processes fault information through the double-channel extraction module to obtain fault codes or maintenance structured data, and processes the information output by the double-channel extraction module, fault information and vehicle basic information through the multi-task learning module to automatically generate a maintenance guide. With the help of the double-channel extraction module, maintenance personnel can quickly locate faults and reduce troubleshooting time. The combination of the double-channel extraction module and the multi-task learning module can automatically generate a maintenance guide, reduce the maintenance threshold, and improve the maintenance efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0035] Figure 1 Fig. 1 shows a structural schematic diagram of a vehicle maintenance diagnosis system according to an embodiment of the present application;

[0036] Figure 2 Fig. 2 shows a first flowchart of a vehicle maintenance diagnosis method according to an embodiment of the present application;

[0037] Figure 3 Fig. 3 shows a second flowchart of a vehicle maintenance diagnosis method according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions of the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments.

[0039] The components of the embodiments of the present application generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0040] Hereinafter, the terms "include", "have", and their conjugates, used in the various embodiments of the present application, merely indicate the presence of the features, numbers, steps, operations, elements, components, or combinations thereof, and do not exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, elements, components, or combinations thereof. In addition, the terms "first", "second", "third", and the like are used only to distinguish the description, and cannot be understood as indicating or implying a relative importance.

[0041] Unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the present application belong. The terms (such as those defined in a generally used dictionary) will be interpreted to have the same meaning as the contextual meaning in the relevant technical field and will not be interpreted to have an idealized or overly formal meaning unless clearly defined in the various embodiments of the present application.

[0042] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0043] Before the vehicle repair diagnosis method of the present application is described, the vehicle repair diagnosis system is first described, as shown in Figure 1 The system includes a multi-modal feature extraction module 100, a double-channel extraction module 200, and a multi-task learning module 300.

[0044] The multi-modal feature extraction module 100 includes a text feature extraction submodule and an image feature extraction submodule. The purpose of the text feature extraction submodule is to extract high-level semantic information from maintenance-related text data, which can be used for tasks such as fault phenomenon classification and maintenance step generation. The purpose of the image feature extraction submodule is to extract key visual information from circuit diagrams or component structure diagrams, such as component position, connection relationship, and topology. The text feature extraction submodule can use a pre-trained language model (such as BERT-base-uncased or RoBERTa) as a base model, and then fine-tune it on a specific dataset in the automotive maintenance field, which includes a large number of official maintenance manuals, third-party maintenance databases, user feedback data (maintenance records, fault phenomenon descriptions), etc. A custom classification head (such as a fully connected layer) is added for specific tasks (such as fault phenomenon classification, maintenance step generation, etc.). The image feature extraction submodule can use a pre-trained image model (such as ResNet-50 or EfficientNet) as a base model, and then fine-tune it on a dataset containing automotive circuit diagrams and component structure diagrams. In addition, these data need to be labeled with key components and their connection relationships, and a target detection head (such as YOLOv5) is added to identify the component relationships in the circuit diagram or structure diagram.

[0045] The double-channel extraction module 200 includes a fault code-context channel and a context-fault code channel. The fault code-context channel is used to generate corresponding structured context information according to the user input fault code, which includes but is not limited to fault phenomenon, fault reason, etc. For example, if the input fault code is P0171, the output structured context information can be fault phenomenon "fuel system too lean" and fault reason "oxygen sensor fault". The context-fault code channel is used to match the corresponding fault code according to the user's natural language description of the vehicle fault problem. For example, if the input natural language description is "engine shaking, fuel consumption increasing", the output corresponding fault code is "P0171", and the description of the corresponding fault code can also be output, such as "abnormal oxygen sensor signal causes misfire". The fault code-context channel can use a sequence labeling model (such as BiLSTM-CRF or BERT sequence labeling model) as the base model, and be trained with labeled vehicle repair related data sources. The context-fault code channel can use a similarity model (such as Siamese Network or BERT-based similarity model), and be trained with labeled vehicle repair related data sources. The main goal is to compare the input natural language description with the semantic representation of known fault codes and find the most similar fault code. The bidirectional mapping relationship of the double-channel extraction module 200 can realize efficient conversion between fault codes and structured context, providing key input information for the downstream multi-task learning module 300, ultimately supporting the generation of dynamic repair guide and accurate fault troubleshooting, thereby improving the automation level of vehicle repair diagnosis, and also narrowing the scope of troubleshooting to help maintenance personnel quickly locate the problem.

[0046] The multi-task learning module 300 includes a multi-task learning model, which specifically includes a shared feature layer output layer, a fault phenomenon extraction layer, a solution generation layer, and a circuit inspection and component testing layer. The multi-task learning module 300 integrates multiple related tasks (such as fault phenomenon extraction, solution generation, circuit inspection and component testing) to realize end-to-end fault diagnosis and repair guide generation. This model can improve diagnosis efficiency and accuracy and reduce the requirement for maintenance personnel's experience. The shared feature layer output layer can be a Transformer Encoder, which can process input multi-modal data (including but not limited to natural language description information, vehicle circuit diagram, structure diagram, and double-channel extraction module 200 output information) to obtain task-shared features. Then, the obtained task-shared features are input into the fault phenomenon extraction layer, the solution generation layer, and the circuit inspection and component testing layer according to the task-shared feature type, and then the corresponding fault phenomenon result, repair step information, and detection step information are obtained.

[0047] Exemplarily, the multi-task learning module 300 can process three tasks, i.e., fault phenomenon extraction, solution generation, and circuit inspection and component testing. Each task has its own specific output layer for converting the task-shared features extracted by the shared feature layer output layer into a specific task result. The design of the three output layers differs according to the different task objectives: the fault phenomenon extraction layer usually uses a fully connected layer (Fully Connected Layer) plus a softmax or sigmoid function for classification. Solution generation usually uses a decoder (Decoder) or a Seq2Seq model to generate sequence output. The circuit inspection and component testing layer can combine a Transformer or attention mechanism to fuse text and image features, and then output the results through a classification head or a generation head. Among them, the solution generation layer can combine the output results of the fault phenomenon extraction layer to output the results, and the data between the circuit inspection and component testing layers can combine the output results of the solution generation layer to output the results. Since there can be certain relevance between different tasks (such as fault phenomenon extraction, solution generation, circuit inspection and component testing), for example, the fault phenomenon "fuel system too lean" is usually associated with "oxygen sensor failure". Through the shared feature extraction layer, the model can learn the common features across tasks (such as the association of "fuel system too lean" and "oxygen sensor"), thereby better capturing the potential links between different tasks. This design of the present embodiment can enable the model to benefit from high-resource tasks when processing low-resource tasks, thereby improving the overall generalization ability.

[0048] In the process of executing specific tasks, the shared feature layer outputs the data input by the output layer as the text features and image features output by the multi-modal feature extraction module 100. The shared feature layer outputs a corresponding feature vector by processing the text features and image features. For example, the text feature "P0171 indicates that the fuel system is too lean" can be encoded as a 768-dimensional vector, and the image feature can be extracted as a 2048-dimensional feature vector. When processing tasks, the vector corresponding to the text feature can be input into the fault phenomenon extraction layer and the solution generation layer, and the vector corresponding to the text feature and the vector corresponding to the image feature can be input into the circuit inspection and component testing layer. The fault phenomenon extraction can output structured fault phenomenon labels (such as "fuel system is too lean"). The solution generation can output specific repair steps (such as "replace the oxygen sensor"). The circuit inspection and component testing can output circuit inspection steps (such as "measure the oxygen sensor voltage") and component testing methods (such as "multimeter test resistance"). In this embodiment, not only can the fault problem and repair steps be obtained, but also specific circuit inspection steps and component testing methods can be generated through the circuit inspection and component testing layer, thereby ensuring that the overall closed-loop process can be completed when the vehicle is repaired, and the task closely cooperates with the fault phenomenon extraction and solution tasks, which can more accurately locate the fault point and reduce unnecessary detection steps. This not only improves the detection efficiency, but also reduces the risk of misjudgment.

[0049] It should be noted that in the multi-task learning model and the double-channel extraction module 200, the output of the fault phenomenon label is involved, but their functional positioning, implementation and correlation in this embodiment are different. The main function of the double-channel extraction module 200 is to semantically analyze from the fault code to the fault phenomenon, and it quickly converts the input fault code into a structured fault phenomenon label by learning the mapping relationship between a large number of fault codes and fault phenomena. The fault phenomenon extraction layer of the multi-task learning model mainly extracts more detailed fault phenomenon labels from natural language descriptions or maintenance manual paragraphs. It further analyzes the input text data to refine the description of the fault phenomenon. In other words, the fault code-context channel of the double-channel extraction module 200 serves as an upstream task to provide preliminary fault phenomenon labels for the fault phenomenon extraction layer of the multi-task learning model. The fault phenomenon extraction layer of the multi-task learning model further refines the description of the fault phenomenon on the basis of the fault code-context channel, and provides more detailed labels. The fault code-context channel focuses on the rapid mapping from fault code to fault phenomenon, and is suitable for processing standardized fault code input; the fault phenomenon extraction layer further excavates the details of the fault phenomenon, and is suitable for processing complex natural language descriptions or maintenance manual paragraphs, and the combination of the two realizes the complete analysis process from fault code to detailed fault phenomenon. In addition, they are mutually coordinated in performance, the output of the fault code-context channel can be used as the input of the fault phenomenon extraction layer, reducing the dependence of the fault phenomenon extraction layer on the original data and improving the efficiency of the overall system. The output of the fault phenomenon extraction layer can also further optimize the performance of the fault code-context channel, and improve the mapping relationship between the fault code and the fault phenomenon through the feedback mechanism.

[0050] Exemplarily, the fault code-context channel and the context-fault code channel in the dual-channel extraction module 200 and the multi-task learning model of the multi-task learning module 300 all need to be pre-trained, and when training, the data sources used include a text data source and an image data source; wherein the text data source includes but is not limited to an official maintenance manual (PDF / DOCX format), which can contain detailed repair information of vehicle models, systems, and components, a third-party repair database (such as AllData, CarMD), which provides fault code definitions, repair cases, etc. User feedback data (repair records, fault phenomenon descriptions), these data can come from actual repair scene cases. These data are the original materials for text feature (semantic feature) extraction, and structured information is extracted from them through natural language processing (NLP) technology to provide a basis for subsequent model training and task execution. The image data source includes but is not limited to circuit diagrams (CAD / PNG format), which label component connection relationships and signal flow directions; component structure diagrams (3D models / two-dimensional drawings), which are used for component testing and replacement guidance. These image data are the original materials for image feature extraction, and structured information is extracted from them through computer vision technology to provide a basis for subsequent model training and task execution.

[0051] When training the model through the above data sources, the data sources need to be preprocessed, and the preprocessing process includes but is not limited to data cleaning, data labeling, data enhancement, etc. Among them, data cleaning can include text cleaning and image cleaning, text cleaning includes but is not limited to OCR conversion (using Tesseract or PaddleOCR to convert PDF manuals into editable text, ensuring character recognition accuracy (such as OCR accuracy ≥ 95%)), noise removal (through regular expressions to delete irrelevant information such as headers and footers, advertising content, etc.), format unification (such as standardizing date formats (such as “2023-10-01”), units (such as “mm” and “inches”)), image cleaning includes but is not limited to denoising (using OpenCV's Gaussian filter or median filter to remove noise in circuit diagrams), binarization (converting circuit diagrams to black and white images for subsequent feature extraction).

[0052] The content of data labeling includes but is not limited to text data labeling, such as labeling fault codes, fault phenomena, and the relationship between maintenance steps (for example, fault code (such as P0171)-fault phenomenon (fuel system is too dilute)-maintenance step (replace oxygen sensor)), circuit diagram labeling, and key components (such as sensors, relays) and their connection relationship (such as "oxygen sensor-ECU") in the circuit diagram can be labeled. The above labeling can be manually labeled or labeled in the form of automatic labeling. If automatic labeling is used, a rule engine (such as "P0171-fuel system related components") can be used for preliminary labeling, and then weakly supervised learning (such as a BERT classification model) can be used to label the unlabeled data.

[0053] Data augmentation includes text augmentation and image augmentation. Text augmentation can include but is not limited to synonym replacement (such as "engine"- "engine") and sentence reorganization (such as "remove oxygen sensor"- "oxygen sensor removal"). Image augmentation can include rotation and flipping of circuit diagrams.

[0054] After preprocessing the data, the text feature extraction submodule and the image feature extraction submodule of the multi-modal feature extraction module 100 can extract the features of the text data and the image data. The features extracted from the text data include but are not limited to fault code related information, maintenance steps, component and system relationship, and description of fault phenomena. These information can be used for training of the multi-task learning model, and can also be used for training of the fault code-context channel. The features extracted from the image data include but are not limited to component position and connection relationship (oxygen sensor, ECU, connection method signal line), component type and identification (extracting the type and identification information of the component from the component structure diagram), signal flow direction and topology structure (extracting the signal flow direction and overall topology structure from the circuit diagram, such as the signal flow direction from the signal starting point to the signal node is oxygen sensor-signal amplifier-ECU), etc. These information can be used together with the information extracted from the text features for training of the multi-task learning model.

[0055] In some embodiments, the system of the present embodiment further includes a knowledge graph module. The knowledge graph can be constructed based on the data source used for model training. Its main function is to associate entities such as fault codes, phenomena, steps and component information, and then support intelligent question answering and reasoning. For example, the information extracted by the text feature extraction submodule can be injected into the knowledge graph to construct the association relationship between fault codes, fault phenomena, maintenance steps and component information. At the same time, the image features extracted by the image feature extraction submodule can also be injected into the knowledge graph to construct the association relationship between components, signal flow direction and fault phenomena. The information obtained by the dual-channel extraction module 200 can also be injected into the knowledge graph.

[0056] In this embodiment, the knowledge graph can be used to organize scattered fault codes, fault phenomena, repair steps, component information, etc. in a structured manner, forming a unified knowledge base for subsequent query, reasoning and expansion. In addition, based on the knowledge graph, question and answer can be realized. Users can query fault information through natural language, and the system returns structured answers based on the knowledge graph, for example, the user inputs "what is the reason for engine shaking", the system can quickly retrieve and return related fault codes (such as P0171) and corresponding repair guide. In addition, using graph neural network (GNN) to reason the knowledge graph can discover potential implicit relationships, for example, "P0171 may cause water pump failure" can be inferred from existing knowledge. This implicit relationship can be used to expand the training set of the dual-channel model, improving the prediction accuracy of remote diagnosis. Further, the entities and relationships in the knowledge graph can be used as input for multi-task learning models, helping to share knowledge between tasks, for example, the results of fault phenomenon extraction can be directly mapped to the input of circuit inspection and component testing, thereby reducing repeated calculations. In addition, by supplementing the model training data with the reasoning results of the knowledge graph, a closed loop of "model training-knowledge sedimentation-model optimization" is formed. This closed loop mechanism can continuously improve the accuracy and robustness of the system.

[0057] In some embodiments, during model training, a BERT-based similarity model can be used to calculate the similarity between the model output and the manually annotated data, for example, the text output by the model (such as fault phenomenon description or repair steps) can be encoded and the cosine similarity between the two can be calculated. By setting a similarity threshold (such as 0.8), if the similarity of a certain sample is lower than the threshold, it is considered that the sample may have misjudgment or quality problems and needs further analysis. When analyzing errors, the specific error type needs to be recorded first. For example, the model may have incorrectly matched a certain fault code to the wrong fault phenomenon (such as misjudging P0171 as "coolant temperature too high" instead of "fuel system too dilute"). Then analyze the cause of the error, which may be due to insufficient data, model overfitting or feature extraction problems, etc. After analyzing the cause of the error, the error can be reduced by adjusting the parameters, for example, by reducing the learning rate to slow down overfitting, or increasing the batch size to improve training stability. In addition, the embodiment can also generate corresponding repair guides and fault codes based on existing fault phenomena or fault definitions. For example, based on the phenomenon of "fuel system too dilute", a repair guide containing the step of "replace oxygen sensor" is generated. Then manually review these reverse generated data to confirm their correctness, and for incorrect parts, make corrections or deletions. These audited data will be re-added to the training set for further optimization of the model.

[0058] The vehicle repair diagnosis method is described based on the vehicle repair diagnosis system described above.

[0059] Figure 2 A flowchart of a vehicle repair diagnosis method of an embodiment of the present application is shown. The vehicle repair diagnosis method exemplarily comprises the following steps:

[0060] In step S100, the fault information is processed by the dual-channel extraction module 200 to obtain first output information in response to the received fault information.

[0061] The dual-channel extraction module 200 is used to realize bidirectional mapping between fault codes and repair structured data.

[0062] Exemplarily, the fault information is fault codes or fault language description information; the dual-channel extraction module 200 comprises a fault code-context channel and a context-fault code channel; if the fault information is fault codes, the fault information is processed by the dual-channel extraction module 200 to obtain the first output information, comprising: the fault codes are processed by the fault code-context channel to obtain corresponding structured context information; wherein the structured context information comprises fault phenomena and fault causes; if the fault information is fault language description information, the fault information is processed by the dual-channel extraction module 200 to obtain the first output information, comprising: the fault language description information is processed by the context-fault code channel to obtain corresponding fault codes and fault code interpretations.

[0063] It can be understood that when the input is fault codes, the fault code-context channel is called to output corresponding structured context information. When the input is fault language description information, the context-fault code channel is called, and the fault language description information is processed using the model of the context-fault code channel to output corresponding fault codes and their interpretations.

[0064] The bidirectional mapping adopted in this embodiment enables the system to accurately analyze and generate corresponding context information or fault codes regardless of whether the input is fault codes or natural language description, thereby improving the flexibility and applicability of the system. In addition, the dual-channel extraction module can also reduce the time for manual searching and judgment, thereby improving the repair efficiency. For example, when the input is fault codes (such as P0171), the system outputs fault phenomena and causes (such as “fuel system is too lean, oxygen sensor is faulty”), which helps technicians quickly locate the problem. When the input is fault language description (such as “engine shaking, increased fuel consumption”), the system recommends possible fault codes (such as P0171), thereby narrowing down the scope of troubleshooting.

[0065] In step S200, the first output information, the fault information and vehicle basic information are processed as diagnosis input information of the multi-task learning module 300, so that the multi-task learning module 300 extracts task common features based on the diagnosis input information, and generates a repair guide based on the task common features.

[0066] The multi-task learning module 300 includes a plurality of task processing modules that simultaneously process tasks based on shared task features. The repair guide includes, but is not limited to, fault phenomenon description, repair steps, circuit inspection and component test location and parameters, etc.

[0067] The shared task features are features extracted from multi-modal data. The multi-modal data includes, but is not limited to, semantic information extracted by the text feature extraction submodule, including semantic associations of fault codes, fault phenomena, and fault causes; visual information extracted by the image feature extraction submodule, including component locations, signal flow directions, and connection relationships; structured data, i.e., associations between fault codes and components, and information input through the double-channel extraction module. For example, taking fault code P0171 as an example, the shared task features can include the following information: semantic information (“fuel system too lean”-BERT encoding captures semantics related to oxygen sensors and air-fuel ratios), visual information (position coordinates (x, y) of the oxygen sensor in the circuit diagram, topology of the connection line (such as connection with the ECU), etc.), structured data (association strength of P0171 and the oxygen sensor (embedded through the knowledge graph)), context information (historical repair records of the fault code (such as “P0171 is often found in vehicles with aging oxygen sensors” and the like), etc. In some embodiments, as shown in FIG. 2B, the fault information and vehicle basic information are processed, including: Figure 3

[0068] Step S210, processing the fault language description information in the fault information through the text feature extraction submodule to obtain semantic features.

[0069] Step S220, determining the target vehicle circuit diagram and structure diagram based on the vehicle basic information, and extracting the circuit diagram visual information of the target vehicle circuit diagram through the image feature extraction submodule to obtain image features.

[0070] The text feature extraction submodule obtains semantic features (i.e., text features), and the image feature extraction submodule extracts image features, which have been described above and will not be repeated here.

[0071] In some embodiments, the multi-task learning module 300 includes a shared feature layer output layer, a fault phenomenon extraction layer, a solution generation layer, and a circuit inspection and component test layer.

[0072] ​The multi-task learning module 300 extracts common features of tasks based on diagnostic input information and generates maintenance guidelines based on these common features. This includes: inputting diagnostic input information into the output layer of the common feature layer to generate common features of tasks; and inputting the common features of tasks into the fault phenomenon extraction layer, the solution generation layer, and the circuit inspection and component testing layer according to feature type to generate fault phenomenon results, maintenance step information, and testing step information.

[0073] The feature types include two categories: text feature types and image feature types. Text feature types include features extracted by the text feature extraction submodule and features formed from data corresponding to the fault code-context channel, which are shared features in the task. Image feature types correspond to features extracted by the image feature extraction submodule. As explained in the system description above, text feature type data can be input into the fault phenomenon extraction layer and the solution generation layer, while text and image feature types can be input into the circuit inspection and component testing layer to obtain information such as fault phenomenon results, maintenance steps, and testing steps. This part has already been explained in the system description and will not be repeated here.

[0074] In some implementations, the training process of the multi-task learning module 300 includes: inputting the extracted vehicle repair text features and vehicle repair image features into a shared feature extraction layer to obtain task-shared features; inputting the task-shared features into the output layer of each task according to feature type to obtain the prediction result for each task; adjusting the parameters of the shared feature extraction layer and the output layer of each task based on the loss between the prediction result and the labeled result until the loss between the prediction result and the labeled result reaches the cutoff condition, thus obtaining the multi-task learning model. This training process has already been described in the above system and will not be repeated here.

[0075] This application embodiment effectively solves the problems of low efficiency, insufficient accuracy, and reliance on experience in traditional vehicle maintenance by using technologies such as dual-channel extraction module 200, multi-task learning model, and knowledge graph.

[0076] This application also provides a terminal device, exemplary of which includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to enable the terminal device to perform the functions of the various modules in the above-described vehicle repair and diagnosis method or the above-described vehicle repair and diagnosis device.

[0077] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0078] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving execution instructions.

[0079] This application also provides a computer-readable storage medium for storing the computer program used in the aforementioned terminal device. For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0081] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0082] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0083] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A vehicle repair and diagnostic method, characterized in that, include: In response to the received fault information, the fault information is processed by a dual-channel extraction module to obtain first output information; wherein, the dual-channel extraction module is used to realize bidirectional mapping between fault codes and maintenance structured data; The first output information, the fault information, and the vehicle basic information are processed and used as diagnostic input information for the multi-task learning module, so that the multi-task learning module can extract common features of tasks based on the diagnostic input information and generate maintenance guidelines based on the common features of tasks; wherein, the multi-task learning module includes multiple task processing modules that simultaneously process tasks based on the common features of tasks. The fault information is a fault code or a fault language description; the dual-channel extraction module includes a fault code-context channel and a context-fault code channel. If the fault information is the fault code, then processing the fault information through the dual-channel extraction module to obtain the first output information includes: processing the fault code through the fault code-context channel to obtain the corresponding structured context information; wherein, the structured context information includes the fault phenomenon and the fault cause; If the fault information is the fault language description information, then the step of processing the fault information through the dual-channel extraction module to obtain the first output information includes: processing the fault language description information through the context-fault code channel to obtain the corresponding fault code and fault code explanation; The multi-task learning module includes a shared feature layer output layer, a fault phenomenon extraction layer, a solution generation layer, and a circuit inspection and component testing layer.

2. The vehicle repair and diagnostic method according to claim 1, characterized in that, The fault information and basic vehicle information are processed, including: The text feature extraction submodule processes the fault language description information in the fault information to obtain semantic features; Based on the basic vehicle information, the circuit diagram and structural diagram of the target vehicle are determined, and the image visual information of the circuit diagram and structural diagram of the target vehicle is extracted by the image feature extraction submodule to obtain image features.

3. The vehicle repair and diagnostic method according to claim 2, characterized in that, The step of enabling the multi-task learning module to extract common features of tasks based on the diagnostic input information and generate maintenance guidelines based on the common features of tasks includes: The diagnostic input information is input into the output layer of the shared feature layer to generate task-shared features; The shared features of the task are input into the fault phenomenon extraction layer, the solution generation layer, and the circuit inspection and component testing layer according to the feature type, so as to generate fault phenomenon results, maintenance step information, and testing step information.

4. The vehicle repair and diagnostic method according to claim 3, characterized in that, The training process of the multi-task learning module includes: The extracted semantic and image features are input into a shared feature extraction layer to obtain task-shared features; The common features of the tasks are input into the output layer of each task according to the feature type to obtain the prediction results for each task; Based on the loss between the prediction results and the annotation results, the parameters of the shared feature extraction layer and the output layer of each task are adjusted until the loss between the prediction results and the annotation results reaches the cutoff condition, thus obtaining the multi-task learning model.

5. The vehicle repair and diagnostic method according to claim 1, characterized in that, The fault code-context channel is obtained by training a sequence labeling model as the base model; The context-fault code channel is obtained by training a similarity model as the base model.

6. A vehicle maintenance and diagnostic system, characterized in that, include: Dual-channel extraction module and multi-task learning module; The dual-channel extraction module is used to process the received fault information in response to obtain first output information; wherein, the dual-channel extraction module is used to realize bidirectional mapping between fault codes and maintenance structured data; A multi-task learning module is used to process the first output information, the fault information, and the vehicle basic information and use them as diagnostic input information for the multi-task learning module, so that the multi-task learning module can extract task-shared features based on the diagnostic input information and generate a maintenance guide based on the task-shared features; wherein, the multi-task learning module includes multiple task processing modules that simultaneously process tasks based on the task-shared features. The fault information is a fault code or a fault language description; the dual-channel extraction module includes a fault code-context channel and a context-fault code channel. If the fault information is the fault code, then processing the fault information through the dual-channel extraction module to obtain the first output information includes: processing the fault code through the fault code-context channel to obtain the corresponding structured context information; wherein, the structured context information includes the fault phenomenon and the fault cause; If the fault information is the fault language description information, then the fault information is processed by the dual-channel extraction module to obtain the first output information, including: processing the fault language description information through the context-fault code channel to obtain the corresponding fault code and fault code explanation; The multi-task learning module includes a shared feature layer output layer, a fault phenomenon extraction layer, a solution generation layer, and a circuit inspection and component testing layer.

7. The vehicle repair and diagnostic system according to claim 6, characterized in that, The system also includes a multimodal feature extraction module; the multimodal feature extraction module includes a text feature extraction submodule and an image feature extraction submodule; The text feature extraction submodule is used to process the fault language description information in the fault information to obtain semantic features; The image feature extraction submodule is used to extract image visual information from vehicle circuit diagrams and vehicle component structure diagrams to obtain image features.

8. A terminal device, characterized in that, The terminal device includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the vehicle repair and diagnostic method according to any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed on a processor, implements the vehicle repair and diagnostic method according to any one of claims 1-5.

Images