Vehicle maintenance diagnosis method, system, equipment and medium

Abstract

The invention relates to the vehicle diagnosis technology, and discloses a vehicle maintenance diagnosis method, system and device and a medium, and the method comprises the steps: responding to received fault information, and processing the fault information through a dual-channel extraction module to obtain first output information; wherein the dual-channel extraction module is used for realizing bidirectional mapping between a fault code and maintenance structured data; the first output information, the fault information and the vehicle basic information are processed to serve as diagnosis input information of a multi-task learning module, so that the multi-task learning module extracts task sharing characteristics based on the diagnosis input information, and a maintenance guide is generated based on the task sharing characteristics; wherein the multi-task learning module comprises a plurality of task processing modules for simultaneously performing task processing based on task sharing characteristics. According to the application, the fault is quickly positioned through the dual-channel extraction module, and then the maintenance guide can be automatically generated in combination with the multi-task learning module, so that the maintenance efficiency is improved, and the maintenance threshold is reduced.

Description

Technical Field

[0001] The present application relates to the field of vehicle diagnosis technology, and in particular to a vehicle maintenance diagnosis method, system, equipment and medium. Background Art

[0002] With the continuous advancement of automotive electronics and intelligent technology, the modern automotive repair industry places increasingly stringent demands on the professional and technical capabilities of its personnel. The traditional automotive repair model primarily relies on paper-based repair manuals and circuit diagrams, which have inherent drawbacks in the following areas: 1. Inefficient information retrieval: Troubleshooting requires technicians to manually review large amounts of paper-based documentation, a time-consuming and labor-intensive process that is prone to errors. 2. Inadequate technical updates: Faced with the rapid evolution of vehicle models and technological innovations, the long update cycle for paper-based repair manuals makes it difficult for repair personnel to stay current on the latest repair techniques and solutions. 3. Overly empirical diagnostic methods: In the absence of a structured knowledge support system, repair diagnosis relies primarily on personal experience and judgment, which can lead to omissions of key repair steps or misdiagnosis of faults. 4. Poor adaptability across vehicle models: Due to significant differences in repair manual formats and technical terminology across different automakers, repair personnel must expend considerable effort learning and adapting to multiple knowledge systems, which increases the cost and difficulty of professional training. Summary of the Invention

[0003] In view of this, embodiments of the present application provide a vehicle maintenance diagnosis method, system, device, and medium.

[0004] In a first aspect, an embodiment of the present application provides a vehicle maintenance diagnosis method, comprising: In response to the received fault information, the fault information is processed by a dual-channel extraction module to obtain first output information; wherein the dual-channel extraction module is used to implement a bidirectional mapping between the fault code and the maintenance structured data; The first output information, the fault information and the basic vehicle information are processed and used as diagnostic input information of a multi-task learning module, so that the multi-task learning module extracts common task features based on the diagnostic input information and generates a maintenance guide based on the common task features; wherein the multi-task learning module includes multiple task processing modules that simultaneously perform task processing based on the common task features.

[0005] In an optional embodiment, the fault information is a fault code or fault language description information; the dual-channel extraction module includes a fault code-context channel and a context-fault code channel; If the fault information is the fault code, processing the fault information through the dual-channel extraction module to obtain first output information includes: Processing the fault code through the fault code-context channel to obtain corresponding structured context information; wherein the structured context information includes the fault phenomenon and the fault cause; If the fault information is the fault language description information, the dual-channel extraction module processes the fault information to obtain first output information, including: The fault language description information is processed through the context-fault code channel to obtain a corresponding fault code and a fault code explanation.

[0006] In an optional implementation manner, processing the fault information and basic vehicle information includes: Processing the fault language description information in the fault information through a text feature extraction submodule to obtain semantic features; The target vehicle circuit diagram and structure diagram are determined based on the basic vehicle information, and the image visual information of the target vehicle circuit diagram and the structure diagram is extracted through an image feature extraction submodule to obtain image features.

[0007] In an optional embodiment, the multi-task learning module includes a shared feature layer output layer, a fault phenomenon extraction layer, a solution generation layer, and a circuit verification and component testing layer; The step of causing the multi-task learning module to extract common task features based on the diagnostic input information and generating a maintenance guide based on the common task features includes: Inputting the diagnostic input information into the shared feature layer output layer to generate task common features; The task common features are input into the fault phenomenon extraction layer, the solution generation layer and the circuit inspection and component testing layer respectively according to feature types to generate fault phenomenon results, maintenance step information and detection step information.

[0008] In an optional embodiment, the training process of the multi-task learning module includes: Inputting the extracted vehicle maintenance text features and vehicle maintenance image features into a shared feature extraction layer to obtain task common features; Input the task common features into the output layer of each task according to the feature type to obtain the prediction results of each task; According to the loss between the prediction result and the labeling result, the parameters of the shared feature extraction layer and the output layer of each task are adjusted until the loss between the prediction result and the labeling result reaches a cutoff condition, thereby obtaining a multi-task learning model.

[0009] In an optional embodiment, the fault code-context channel is obtained by training using a sequence labeling model as a basic model; The context-fault code channel is obtained through training using a similarity model as a basic model.

[0010] In a second aspect, an embodiment of the present application provides a vehicle maintenance diagnostic system, comprising: a dual-channel extraction module and a multi-task learning module; The dual-channel extraction module is configured to process the received fault information to obtain first output information in response to the received fault information; wherein the dual-channel extraction module is configured to implement a bidirectional mapping between the fault code and the maintenance structured data; A multi-task learning module is used to process the first output information, the fault information and the basic vehicle information as diagnostic input information of the multi-task learning module, so that the multi-task learning module extracts common task features based on the diagnostic input information and generates a maintenance guide based on the common task features; wherein the multi-task learning module includes multiple task processing modules that simultaneously perform task processing based on the common task features.

[0011] In an optional embodiment, the system further includes a multimodal feature extraction module; the multimodal feature extraction module includes a text feature extraction submodule and an image feature extraction submodule; The text feature extraction submodule is used to process the fault language description information in the fault information to obtain semantic features; The image feature extraction submodule is used to extract image visual information of the vehicle circuit diagram and the vehicle component structure diagram to obtain image features.

[0012] In a third aspect, an embodiment of the present application provides a terminal device, which includes a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the vehicle maintenance diagnosis method described in the aforementioned embodiment.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed on a processor, the vehicle maintenance diagnosis method described in the aforementioned embodiment is implemented.

[0014] The embodiments of the present application have the following beneficial effects: the present application processes fault information through a dual-channel extraction module to obtain fault codes or maintenance structured data, and processes the information output from the dual-channel extraction module, fault information, and basic vehicle information through a multi-task learning module to automatically generate a maintenance guide. With the help of the dual-channel extraction module, maintenance personnel can quickly locate faults and reduce troubleshooting time. By combining the dual-channel extraction module and the multi-task learning module, a maintenance guide can be automatically generated, which lowers the maintenance threshold and improves maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A schematic structural diagram of a vehicle maintenance diagnostic system according to an embodiment of the present application is shown; Figure 2 A first flow chart of the vehicle maintenance diagnosis method according to an embodiment of the present application is shown; Figure 3 A second flow chart of the vehicle maintenance diagnosis method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0018] The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but rather merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0019] Hereinafter, the terms "including", "having" and their cognates used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the aforementioned items, and should not be understood as excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the aforementioned items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the aforementioned items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and should not be understood as indicating or implying relative importance.

[0020] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.

[0021] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0022] Before describing the vehicle maintenance diagnosis method of the present application, the vehicle maintenance diagnosis system is first described. Figure 1 As shown, the system includes a multimodal feature extraction module 100, a dual-channel extraction module 200 and a multi-task learning module 300.

[0023] The multimodal feature extraction module 100 includes a text feature extraction submodule and an image feature extraction submodule. The text feature extraction submodule aims to extract high-level semantic information from maintenance-related text data, which can be used for tasks such as fault phenomenon classification and repair procedure generation. The image feature extraction submodule aims to extract key visual information from circuit diagrams or component structure diagrams, such as component location, connection relationships, and topology. The text feature extraction submodule can use a pretrained language model (such as BERT-base-uncased or RoBERTa) as a base model and then fine-tune it on a dataset specific to the automotive maintenance field. This dataset includes a large number of official repair manuals, third-party repair databases, and user feedback data (maintenance records, fault phenomenon descriptions). Custom classification heads (such as fully connected layers) are then added for specific tasks (such as fault phenomenon classification and repair procedure generation). The image feature extraction submodule can use a pre-trained image model (such as ResNet-50 or EfficientNet) as the base model, and then fine-tune it on a dataset containing automotive circuit diagrams and component structure diagrams. In addition, these data need to be labeled with key components and their connection relationships, and an object detection head (such as YOLOv5) is added to identify the component relationships in the circuit diagram or structure diagram.

[0024] The dual-channel extraction module 200 includes a fault code-context channel and a context-fault code channel. The fault code-context channel is used to generate corresponding structured context information based on the fault code entered by the user. This structured context information includes, but is not limited to, the fault phenomenon and fault cause. For example, if the fault code entered is P0171, the output structured context information may include the fault phenomenon "fuel system too lean" and the fault cause "oxygen sensor failure." The context-fault code channel is used to match the fault code to the vehicle fault problem described by the user in natural language. For example, if the natural language description entered is "engine shaking, increased fuel consumption," the output corresponding fault code is "P0171." In addition, a description of the corresponding fault code can be output, such as "abnormal oxygen sensor signal causing air-fuel ratio imbalance." The fault code-context channel can be trained using a sequence labeling model (such as a BiLSTM-CRF or BERT sequence labeling model) in combination with labeled vehicle maintenance data sources. The context-fault code channel can be trained using a similarity model (such as a SiameseNetwork or BERT-based similarity model) combined with labeled vehicle maintenance-related data sources. The main goal is to compare the input natural language description with the semantic representation of known fault codes to find the most similar fault code. The bidirectional mapping relationship of the dual-channel extraction module 200 enables efficient conversion between fault codes and structured context, providing key input information for the downstream multi-task learning module 300. Ultimately, it supports the generation of dynamic maintenance guides and accurate troubleshooting, thereby improving the automation level of vehicle maintenance diagnosis, narrowing the scope of investigation, and helping maintenance personnel quickly locate problems.

[0025] The multi-task learning module 300 comprises a multi-task learning model, specifically comprising a shared feature layer output layer, a fault phenomenon extraction layer, a solution generation layer, and a circuit verification and component testing layer. By integrating multiple related tasks (such as fault phenomenon extraction, solution generation, circuit verification and component testing), the multi-task learning module 300 achieves end-to-end fault diagnosis and repair guide generation. This model can improve diagnostic efficiency and accuracy and reduce the experience requirements of maintenance personnel. The shared feature layer output layer can be a Transformer Encoder, which processes the input multimodal data (including but not limited to natural language descriptions, vehicle circuit diagrams and structure diagrams, and information output by the dual-channel extraction module 200) to obtain task-shared features. The obtained task-shared features are then input into the fault phenomenon extraction layer, solution generation layer, and circuit verification and component testing layer according to the task-shared feature type, thereby obtaining the corresponding fault phenomenon results, repair step information, and detection step information.

[0026] For example, the multi-task learning module 300 can handle three tasks: fault symptom extraction, solution generation, and circuit verification and component testing. Each task has its own dedicated output layer, which converts the shared features extracted by the shared feature layer into specific task results. The design of the three output layers varies depending on the task objectives: The fault symptom extraction layer typically uses a fully connected layer with a softmax or sigmoid function for classification. Solution generation typically uses a decoder or Seq2Seq model to generate sequence outputs. The circuit verification and component testing layers may combine text and image features using a Transformer or attention mechanism, and then output the results through a classification head or a generation head. The solution generation layer can combine the output of the fault symptom extraction layer to output results, while the data between the circuit verification and component testing layers can combine the output of the solution generation layer to output results. Because different tasks (such as fault symptom extraction, solution generation, circuit verification, and component testing) may have certain correlations, for example, the fault phenomenon of "lean fuel system" is often associated with "oxygen sensor failure." By sharing feature extraction layers, the model can learn common features across tasks (such as the association between "fuel system too lean" and "oxygen sensor"), thereby better capturing the potential connections between different tasks. This design in this embodiment allows the model to benefit from high-resource tasks while also handling low-resource tasks, improving overall generalization capabilities.

[0027] During the execution of a specific multi-task learning task, the shared feature layer output layer receives input data from the text and image features output by the multimodal feature extraction module 100. The shared feature layer output layer processes the text and image features to obtain corresponding feature vectors. For example, the text feature "P0171 indicates the fuel system is too lean" can be encoded as a 768-dimensional vector, and the image features can be extracted as a 2048-dimensional feature vector. During task processing, the vectors corresponding to the text features are input into the fault phenomenon extraction layer and the solution generation layer, while the vectors corresponding to the text and image features are input into the circuit inspection and component testing layer. Fault phenomenon extraction can output a structured fault phenomenon label (e.g., "the fuel system is too lean"). Solution generation can output specific repair steps (e.g., "replace the oxygen sensor"). Circuit inspection and component testing can output circuit inspection steps (e.g., "measure the oxygen sensor voltage") and component testing methods (e.g., "test resistance with a multimeter"). In this embodiment, not only can the fault problem and maintenance steps be obtained, but also specific circuit inspection steps and component testing methods can be generated through the circuit inspection and component testing layer, thereby ensuring that the overall closed-loop process can be completed when repairing the vehicle. This task works closely with the tasks of fault phenomenon extraction and solution, and can locate the fault point more accurately and reduce unnecessary detection steps. This not only improves detection efficiency, but also reduces the risk of misjudgment.

[0028] It should be noted that both the multi-task learning model and the dual-channel extraction module 200 involve the output of fault phenomenon labels, but their functional positioning, implementation methods, and associations in this embodiment are different. The dual-channel extraction module 200 mainly functions to perform semantic parsing from fault codes to fault phenomena. By learning the mapping relationships between a large number of fault codes and fault phenomena, it quickly converts the input fault codes into structured fault phenomenon labels. The fault phenomenon extraction layer of the multi-task learning model mainly extracts more detailed fault phenomenon labels from natural language descriptions or maintenance manual paragraphs. It further analyzes the input text data to refine the description of the fault phenomenon. In other words, the fault code-context channel of the dual-channel extraction module 200 serves as an upstream task, providing preliminary fault phenomenon labels for the fault phenomenon extraction layer of the multi-task learning model. Based on the fault code-context channel, the fault phenomenon extraction layer of the multi-task learning model further refines the description of the fault phenomenon and provides more detailed labels. The fault code-context channel focuses on rapid mapping from fault codes to fault symptoms, making it suitable for processing standardized fault code inputs. The fault symptom extraction layer further explores the details of the fault symptoms, making it suitable for processing complex natural language descriptions or maintenance manual paragraphs. The combination of the two enables a complete parsing process from fault codes to detailed fault symptoms. Furthermore, their performance is synergistic. The output of the fault code-context channel can serve as input to the fault symptom extraction layer, reducing its reliance on raw data and improving overall system efficiency. The output of the fault symptom extraction layer can also further optimize the performance of the fault code-context channel, improving the mapping relationship between fault codes and fault symptoms through a feedback mechanism.

[0029] Exemplarily, the fault code-context channel and context-fault code channel in the dual-channel extraction module 200, as well as the multi-task learning model in the multi-task learning module 300, need to be pre-trained. The data sources used for training include text and image data sources. Text data sources include, but are not limited to, official repair manuals (PDF / DOCX format), which may contain detailed repair information for vehicle models, systems, and components; third-party repair databases (such as AllData and CarMD), which provide fault code definitions and repair cases; and user feedback data (repair records, fault symptom descriptions), which can be derived from actual repair scenarios. This data serves as the raw material for text feature (semantic feature) extraction. Natural language processing (NLP) techniques are used to extract structured information from this data, providing a foundation for subsequent model training and task execution. Image data sources include, but are not limited to, circuit diagrams (CAD / PNG format), which annotate component connections and signal flow; and component structure diagrams (3D models / 2D drawings), which are used for component testing and replacement guidance. This image data serves as the raw material for image feature extraction. Computer vision techniques are used to extract structured information from this data, providing a foundation for subsequent model training and task execution.

[0030] When training a model using the above data sources, the data sources need to be preprocessed. The preprocessing process includes but is not limited to data cleaning, data labeling, and data enhancement. Data cleaning can include text cleaning and image cleaning. Text cleaning includes but is not limited to OCR conversion (using Tesseract or PaddleOCR to convert PDF manuals into editable text, ensuring character recognition accuracy (e.g., OCR accuracy ≥ 95%)), noise removal (using regular expressions to remove irrelevant information such as headers, footers, and advertising content), format unification (such as standardizing date formats (e.g., "2023-10-01") and units (e.g., "mm" and "inches")), image cleaning includes but is not limited to denoising (using OpenCV's Gaussian filter or median filter to remove noise from circuit diagrams), and binarization (converting circuit diagrams to black and white images to facilitate subsequent feature extraction).

[0031] Data annotation includes, but is not limited to, text data annotation, such as the relationship between fault codes, fault symptoms, and repair steps (e.g., fault code (e.g., P0171) - fault symptom (fuel system too lean) - repair step (replace oxygen sensor)). Circuit diagram annotation can also include key components (e.g., sensors, relays) and their connections (e.g., "oxygen sensor - ECU"). This annotation can be done manually or automatically. Automatic annotation can be performed using a rule engine (e.g., "P0171 - fuel system related components") for preliminary annotation. Then, weakly supervised learning (e.g., BERT classification model) can be used to annotate unlabeled data.

[0032] Data augmentation includes text augmentation and image augmentation. Text augmentation can include, but is not limited to, synonym replacement (e.g., "engine" to "engine") and sentence reorganization (e.g., "remove oxygen sensor" to "oxygen sensor removal"). Image augmentation can involve rotating or flipping circuit diagrams.

[0033] After preprocessing the data, the text feature extraction submodule and image feature extraction submodule of the multimodal feature extraction module 100 can extract features from the text and image data. Features extracted from the text data include, but are not limited to, fault code information, repair steps, component-system relationships, and fault symptom descriptions. This information can be used to train a multi-task learning model and a fault code-context channel. Features extracted from the image data include, but are not limited to, component location and connection relationships (oxygen sensor, ECU, connection signal lines), component type and identification (extracting component type and identification information from component structure diagrams), and signal flow and topology (extracting signal flow and overall topology from circuit diagrams, e.g., the signal flow from the signal starting point to the signal node is oxygen sensor-signal amplifier-ECU). This information can be used together with the information extracted from the text features to train the multi-task learning model.

[0034] In some embodiments, the system of this embodiment also includes a knowledge graph module. The knowledge graph can be constructed based on the data source used for model training. Its main function is to associate entities such as fault codes, phenomena, steps and component information, thereby supporting intelligent question answering and reasoning. For example, the information extracted by the text feature extraction submodule can be injected into the knowledge graph to construct the association between fault codes, fault phenomena, maintenance steps and component information. At the same time, the image features extracted by the image feature extraction submodule can also be injected into the knowledge graph to construct the association between components, signal flow and fault phenomena. The information obtained by the dual-channel extraction module 200 can also be injected into the knowledge graph.

[0035] In this embodiment, the knowledge graph is used to organize scattered fault codes, fault symptoms, repair procedures, and component information in a structured manner, forming a unified knowledge base for subsequent querying, reasoning, and expansion. Furthermore, the knowledge graph enables question-and-answering. Users can query fault information using natural language, and the system returns structured answers based on the knowledge graph. For example, if a user enters "What causes engine vibration?", the system can quickly retrieve and return relevant fault codes (such as P0171) and corresponding repair instructions. Furthermore, reasoning on the knowledge graph using a graph neural network (GNN) can discover potential implicit relationships. For example, by inferring "P0171 may cause a water pump failure" based on existing knowledge, this implicit relationship can be used to expand the training set of the dual-channel model and improve the predictive accuracy of remote diagnosis. Furthermore, the entities and relationships in the knowledge graph can serve as inputs for multi-task learning models, facilitating knowledge sharing between tasks. For example, the results of fault symptom extraction can be directly mapped to the inputs for circuit inspection and component testing, thereby reducing duplicate computation. In addition, the inference results of the knowledge graph are used to supplement the model training data, forming a closed loop of "model training-knowledge precipitation-model optimization". This closed-loop mechanism can continuously improve the accuracy and robustness of the system.

[0036] In some embodiments, during model training, a BERT-based similarity model can be used to calculate the similarity between the model output and manually annotated content. For example, the model output text (such as a fault description or repair steps) can be encoded with the manually annotated content and the cosine similarity between the two can be calculated. A similarity threshold (e.g., 0.8) is set. If the similarity of a sample falls below this threshold, the sample is considered to have a possible misclassification or quality issue and requires further analysis. When analyzing errors, the specific error type must first be recorded. For example, the model may mistakenly match a fault code to the wrong fault phenomenon (e.g., misclassifying P0171 as "coolant temperature too high" instead of "fuel system too lean"). The cause of the error can then be analyzed, potentially due to insufficient data, model overfitting, or feature extraction issues. After analyzing the cause of the error, parameters can be adjusted to reduce the error, such as by reducing the learning rate to mitigate overfitting or increasing the batch size to improve training stability. Furthermore, this embodiment can also reverse-generate corresponding repair instructions and fault codes based on existing fault phenomena or fault definitions. For example, based on the "fuel system is too lean" phenomenon, a maintenance guide containing the steps for "replacing the oxygen sensor" is generated. This reverse-generated data is then manually reviewed to confirm its accuracy. Incorrect parts are corrected or deleted. This reviewed data is then added back to the training set for further model optimization.

[0037] The vehicle maintenance diagnosis method will be described based on the above-mentioned vehicle maintenance diagnosis system.

[0038] Figure 2 A flow chart of a vehicle maintenance diagnosis method according to an embodiment of the present application is shown. Exemplarily, the vehicle maintenance diagnosis method includes the following steps: Step S100 : In response to the received fault information, the dual-channel extraction module 200 processes the fault information to obtain first output information.

[0039] The dual-channel extraction module 200 is used to implement a bidirectional mapping between fault codes and maintenance structured data.

[0040] Exemplarily, the fault information is a fault code or a fault language description; the dual-channel extraction module 200 includes a fault code-context channel and a context-fault code channel. If the fault information is a fault code, the dual-channel extraction module 200 processes the fault information to obtain first output information, including processing the fault code through the fault code-context channel to obtain corresponding structured context information; wherein the structured context information includes the fault phenomenon and the fault cause. If the fault information is a fault language description, the dual-channel extraction module 200 processes the fault information to obtain first output information, including processing the fault language description through the context-fault code channel to obtain the corresponding fault code and fault code explanation.

[0041] As you can understand, when the input is a fault code, the fault code-context channel is invoked to output the corresponding structured context information. When the input is a fault language description, the context-fault code channel is invoked to process the fault language description using the context-fault code channel model to output the corresponding fault code and its explanation.

[0042] This embodiment utilizes bidirectional mapping, enabling the system to accurately parse and generate corresponding contextual information or fault codes, regardless of whether the input is a fault code or a natural language description. This improves the system's flexibility and applicability. Furthermore, the bidirectional channel extraction module reduces manual search and judgment time, improving maintenance efficiency. For example, when a fault code (such as P0171) is input, the system outputs the fault symptoms and causes (such as "fuel system too lean, oxygen sensor fault"), helping technicians quickly locate the problem. When a fault description (such as "engine shaking, increased fuel consumption") is input, the system recommends possible fault codes (such as P0171), thereby narrowing the scope of investigation.

[0043] In step S200 , the first output information, fault information, and basic vehicle information are processed and used as diagnostic input information for the multi-task learning module 300 , so that the multi-task learning module 300 extracts common features of tasks based on the diagnostic input information and generates a maintenance guide based on the common features of tasks.

[0044] The multi-task learning module 300 includes multiple task processing modules that simultaneously process tasks based on common task features. The maintenance guide includes, but is not limited to, a description of the fault phenomenon, maintenance steps, locations and parameters for circuit inspection and component testing, etc.

[0045] Among them, the task-shared features are features extracted from multimodal data. Among them, multimodal data includes but is not limited to semantic information extracted by the text feature extraction submodule, including semantic associations of fault codes, fault phenomena, and fault causes; visual information extracted by the image feature extraction submodule, including spatial features such as component location, signal flow, and connection relationships; structured data, that is, the association between fault codes and components, etc., and information input through the dual-channel extraction module. For example, taking the fault code P0171 as an example, the task-shared features may include the following information: semantic information ("fuel system is too lean" - BERT encoding captures semantics related to oxygen sensors and air-fuel ratios), visual information (position coordinates (x, y) of the oxygen sensor in the circuit diagram, topological structure of the connection line (such as connection with the ECU), etc.), structured data (the strength of the association between P0171 and the oxygen sensor (through knowledge graph embedding)), contextual information (historical maintenance records of fault codes (such as "P0171 is common in vehicles with aging oxygen sensors", etc.). In some embodiments, such as Figure 3 As shown, the fault information and basic vehicle information are processed, including: Step S210 : Processing the fault language description information in the fault information through the text feature extraction submodule to obtain semantic features.

[0046] Step S220 , determining the target vehicle circuit diagram and structure diagram based on the basic vehicle information, and extracting circuit diagram visual information of the target vehicle circuit diagram through the image feature extraction submodule to obtain image features.

[0047] The acquisition of semantic features (ie, text features) through the text feature extraction submodule and the extraction of image features through the image feature extraction submodule have been described above and will not be repeated here.

[0048] In some embodiments, the multi-task learning module 300 includes a shared feature layer output layer, a fault phenomenon extraction layer, a solution generation layer, and a circuit verification and component testing layer.

[0049] The multi-task learning module 300 extracts common features of tasks based on the diagnostic input information and generates a maintenance guide based on the common features of tasks, including: inputting the diagnostic input information into the shared feature layer output layer to generate common features of tasks; inputting the common features of tasks into the fault phenomenon extraction layer, the solution generation layer and the circuit inspection and component testing layer according to the feature type to generate fault phenomenon results, maintenance step information and detection step information.

[0050] Among them, the feature types include text feature types and image feature types. The text feature types include the features extracted by the text feature extraction submodule in the task common features and the features formed by the data corresponding to the fault code-context channel; the image feature types correspond to the features extracted by the image feature extraction submodule. As explained in the above system description, the data of the text feature type can be input into the fault phenomenon extraction layer and the solution generation layer, and the features of the text feature type and the features of the image feature type can be input into the circuit inspection and component testing layer to obtain information such as fault phenomenon results, maintenance step information and detection step information. The above system has already explained this part and will not be repeated here.

[0051] In some embodiments, the training process of the multi-task learning module 300 includes: inputting the extracted vehicle maintenance text features and vehicle maintenance image features into a shared feature extraction layer to obtain task-shared features; inputting the task-shared features into the output layer of each task according to feature type to obtain a prediction result for each task; and adjusting the parameters of the shared feature extraction layer and the output layer of each task based on the loss between the prediction result and the annotation result until the loss between the prediction result and the annotation result reaches a cutoff condition, thereby obtaining a multi-task learning model. This training process has been described in the above system and will not be repeated here.

[0052] The embodiment of the present application effectively solves the problems of low efficiency, lack of accuracy and experience dependence in traditional vehicle maintenance through technologies such as the dual-channel extraction module 200, a multi-task learning model and a knowledge graph.

[0053] The present application also provides a terminal device. Exemplarily, the terminal device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the terminal device to execute the functions of the various modules in the above-mentioned vehicle maintenance diagnostic method or the above-mentioned vehicle maintenance diagnostic device.

[0054] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, etc., and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0055] The memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM). The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving an execution instruction.

[0056] This application also provides a computer-readable storage medium for storing the computer program used in the terminal device. For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0057] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or flowchart, and the combination of boxes in the structure diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0058] In addition, the functional modules or units in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0059] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a smart phone, personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0060] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A vehicle maintenance diagnosis method, characterized in that: include: In response to the received fault information, the fault information is processed by a dual-channel extraction module to obtain first output information; wherein the dual-channel extraction module is used to implement a bidirectional mapping between the fault code and the maintenance structured data; The first output information, the fault information and the basic vehicle information are processed and used as diagnostic input information of a multi-task learning module, so that the multi-task learning module extracts common task features based on the diagnostic input information and generates a maintenance guide based on the common task features; wherein the multi-task learning module includes multiple task processing modules that simultaneously perform task processing based on the common task features.

2. The vehicle maintenance diagnosis method according to claim 1, characterized in that: The fault information is a fault code or fault language description information; the dual-channel extraction module includes a fault code-context channel and a context-fault code channel; If the fault information is the fault code, processing the fault information through the dual-channel extraction module to obtain first output information includes: Processing the fault code through the fault code-context channel to obtain corresponding structured context information; wherein the structured context information includes the fault phenomenon and the fault cause; If the fault information is the fault language description information, the dual-channel extraction module processes the fault information to obtain first output information, including: The fault language description information is processed through the context-fault code channel to obtain a corresponding fault code and a fault code explanation.

3. The vehicle maintenance diagnosis method according to claim 1, characterized in that: Processing the fault information and basic vehicle information includes: Processing the fault language description information in the fault information through a text feature extraction submodule to obtain semantic features; The target vehicle circuit diagram and structure diagram are determined based on the basic vehicle information, and the image visual information of the target vehicle circuit diagram and the structure diagram is extracted through an image feature extraction submodule to obtain image features.

4. The vehicle maintenance diagnosis method according to claim 3, characterized in that: The multi-task learning module includes a shared feature layer output layer, a fault phenomenon extraction layer, a solution generation layer, and a circuit verification and component testing layer; The step of causing the multi-task learning module to extract common task features based on the diagnostic input information and generating a maintenance guide based on the common task features includes: Inputting the diagnostic input information into the shared feature layer output layer to generate task common features; The task common features are input into the fault phenomenon extraction layer, the solution generation layer and the circuit inspection and component testing layer respectively according to feature types to generate fault phenomenon results, maintenance step information and detection step information.

5. The vehicle maintenance diagnosis method according to claim 4, characterized in that: The training process of the multi-task learning module includes: Inputting the extracted semantic features and image features into a shared feature extraction layer to obtain task-shared features; Input the task common features into the output layer of each task according to the feature type to obtain the prediction results of each task; According to the loss between the prediction result and the labeling result, the parameters of the shared feature extraction layer and the output layer of each task are adjusted until the loss between the prediction result and the labeling result reaches a cutoff condition, thereby obtaining a multi-task learning model.

6. The vehicle maintenance diagnosis method according to claim 2, characterized in that: The fault code-context channel is obtained by training using a sequence labeling model as a basic model; The context-fault code channel is obtained through training using a similarity model as a basic model.

7. A vehicle maintenance diagnostic system, characterized in that: include: Dual-channel extraction module and multi-task learning module; The dual-channel extraction module is configured to process the received fault information to obtain first output information in response to the received fault information; wherein the dual-channel extraction module is configured to implement a bidirectional mapping between the fault code and the maintenance structured data; A multi-task learning module is used to process the first output information, the fault information and the basic vehicle information as diagnostic input information of the multi-task learning module, so that the multi-task learning module extracts common task features based on the diagnostic input information and generates a maintenance guide based on the common task features; wherein the multi-task learning module includes multiple task processing modules that simultaneously perform task processing based on the common task features.

8. The vehicle maintenance diagnostic system according to claim 7, characterized in that: The system further comprises a multimodal feature extraction module; the multimodal feature extraction module comprises a text feature extraction submodule and an image feature extraction submodule; The text feature extraction submodule is used to process the fault language description information in the fault information to obtain semantic features; The image feature extraction submodule is used to extract image visual information of the vehicle circuit diagram and the vehicle component structure diagram to obtain image features.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory, the memory stores a computer program, and the processor is configured to execute the computer program to implement the vehicle maintenance diagnosis method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that The device stores a computer program, which, when executed on a processor, implements the vehicle maintenance diagnosis method according to any one of claims 1 to 6.

Images