Fault diagnosis system and method for unmanned transportation logistics vehicle based on large language model

Abstract

The invention provides an unmanned transportation logistics vehicle fault diagnosis system and method based on a large language model, and the system comprises an unmanned transportation logistics vehicle fault diagnosis model which is used for inputting the multi-mode fault information of an unmanned transportation logistics vehicle, the man-machine interaction module outputs fault reasons, fault positioning information, fault removal suggestions and fault removal feedback experience; the unmanned transportation logistics vehicle fault diagnosis system comprises an unmanned transportation logistics vehicle fault diagnosis model, a continuously-updated fault diagnosis knowledge base used for storing historical fault logs, historical fault reasons and historical fault removal experience, and a fault knowledge graph used for training and updating the unmanned transportation logistics vehicle fault diagnosis model and self-updating the continuously-updated fault diagnosis knowledge base. According to the fault diagnosis method and device, based on the fault diagnosis model and the continuously-updated fault diagnosis knowledge base, an auxiliary fault diagnosis function is provided for workers through man-machine interaction, the fault diagnosis efficiency is improved, the diagnosis cost is reduced, new fault types are continuously learned, and a virtuous cycle optimization mechanism is formed.

Description

Technical Field

[0001] This application relates to the field of fault diagnosis technology for unmanned driving, specifically to a fault diagnosis system and method for unmanned transport logistics vehicles based on a large language model. Background Technology

[0002] In recent years, against the backdrop of industrial intelligent transformation, the intelligentization process of the logistics industry has been accelerating. Unmanned logistics vehicles, as an innovative mode of logistics transportation, have been widely applied and promoted. These unmanned logistics vehicles, with their automation and intelligence, greatly improve material transportation efficiency and reduce labor costs, playing an increasingly important role in the material transportation process. They can be uniformly scheduled through application software and accurately complete the handling and distribution of materials within the factory area according to preset task instructions. Moreover, these logistics vehicles have an automatic charging function when the battery is low, effectively ensuring the continuity of transportation tasks.

[0003] However, as the frequency of use and operating time of unmanned transport vehicles increase, the probability of malfunctions during operation also rises. These unmanned transport vehicles operate for extended periods in complex workshop assembly plant environments, needing to handle different application routes (warehouses and production lines, interior trim lines, final delivery lines, etc.) and adapt to frequent task scheduling and high-intensity workloads. Simultaneously, the long-term operation of the electrical system, battery charge-discharge cycles, continuous operation of the software system, and inherent logical vulnerabilities in the software system itself can all lead to various malfunctions.

[0004] Existing fault diagnosis systems for unmanned logistics vehicles already possess certain basic functions. Some systems can use sensors equipped on the vehicle to monitor its operating parameters in real time, such as battery level, speed, and mileage, and perform simple fault diagnosis based on pre-set rules. For example, when the battery level falls below a set threshold, the system will issue a low battery warning and automatically find a charging route based on pre-designed charging thresholds; when the vehicle speed is abnormal, the driving trajectory deviates, or the dispatch task fails to issue instructions, the system can provide preliminary fault indications.

[0005] However, specific faults still require experienced software developers to troubleshoot and analyze the interaction messages between the logistics vehicle and the server based on the fault logs, and to regularly maintain and optimize the software code.

[0006] While existing fault diagnosis methods have achieved some success, numerous problems remain in practical applications. Current fault diagnosis systems can only effectively handle single-type fault scenarios, such as map activation failure prompting a map restart, or vehicles not receiving subsequent instructions, with feedback indicating unstable MQTT connections or missed vehicle location information scans. However, in scenarios with multiple potential faults coexisting, such as when the vehicle's battery system malfunctions while the software scheduling system also malfunctions, existing systems struggle to accurately determine the primary and secondary faults and their sequence, hindering troubleshooting and repair. Therefore, experienced technicians are still required to handle these situations. Furthermore, existing diagnostic systems generally lack adaptability and learning capabilities. They rely heavily on fixed diagnostic models and rules, making it difficult to respond quickly and accurately to emerging fault patterns, especially those caused by vehicle technology upgrades, software updates, or special operating conditions. Simultaneously, existing diagnostic systems have significant shortcomings in processing fault log data, failing to fully extract the deeper information and logic contained within it, resulting in a large amount of valuable data being wasted.

[0007] Patent application number CN201110026142.1 discloses a multi-functional integrated electronic vehicle fault diagnosis system, comprising three components: an in-vehicle network, a VCI system, and PC diagnostic software. The system has two operating modes: offline mode and online mode. In offline mode, it can read fault codes, clear fault codes, read freeze frames, read data streams, and read module information. In online mode, it can perform fault diagnosis for multiple protocols.

[0008] However, this patent only addresses fault diagnosis for the onboard hardware of traditional electric vehicles. Since autonomous vehicle systems include not only simple, traditional body controllers but also the autonomous driving brain (localization, perception, decision-making, and control modules), traditional automotive fault diagnosis systems are not applicable to autonomous vehicle systems.

[0009] Furthermore, existing fault diagnosis methods heavily rely on specialized technical personnel. Accurate fault diagnosis and effective troubleshooting require professionals with extensive knowledge of developing software systems for dispatching unmanned transportation logistics vehicles, as well as a wealth of practical experience. This reliance on specialized personnel not only significantly increases operating costs for enterprises, but also hinders the widespread dissemination and transfer of professional experience within companies, severely limiting the overall improvement and widespread application of fault diagnosis technology.

[0010] In conclusion, with the increasing number of unmanned logistics vehicles and the growing complexity of their application scenarios, improving the intelligence level of unmanned logistics vehicle fault diagnosis systems has become a top priority. On the one hand, it is necessary to enhance the system's comprehensive diagnostic capabilities in handling complex fault scenarios, achieving rapid and accurate fault location and efficient troubleshooting. On the other hand, it is essential to improve the system's adaptability and learning capabilities, enabling it to automatically learn and respond to constantly changing fault types and fully utilize key information from fault log data. Furthermore, it is necessary to reduce reliance on professional personnel, organically integrating human diagnostic experience into the diagnostic system to achieve rapid sharing and application of experience, thereby promoting the intelligent and efficient operation of unmanned logistics vehicles in material transportation. Summary of the Invention

[0011] To address the shortcomings of existing technologies, the purpose of this application is to provide a fault diagnosis system and method for unmanned transport logistics vehicles based on a large language model.

[0012] To achieve the above objectives, according to one aspect of this application, a fault diagnosis system for unmanned transport logistics vehicles based on a large language model is provided, comprising: An unmanned transport logistics vehicle fault diagnosis model is used to input multimodal fault information of the unmanned transport logistics vehicle, integrate a human-machine interaction module, and output fault causes, fault location information, fault troubleshooting suggestions and fault troubleshooting feedback experience through the human-machine interaction module. The multimodal fault information includes on-board sensor data, natural language fault descriptions and unmanned transport logistics vehicle fault logs. A continuously updatable fault diagnosis knowledge base is used to store historical fault logs, historical fault causes, and historical fault troubleshooting experience. The continuously updatable fault diagnosis knowledge base also includes a fault knowledge graph. The fault knowledge graph and the historical fault logs are used to train and update the fault diagnosis model of the unmanned transport logistics vehicle. The continuously updatable fault diagnosis knowledge base is also used to self-update based on the fault troubleshooting feedback experience.

[0013] Optionally, the human-computer interaction module is used by staff and developers to input the natural language fault description and the fault log of the unmanned transport logistics vehicle, and to display the fault cause, fault location information, fault troubleshooting suggestions and fault troubleshooting feedback experience in the form of a visual interface.

[0014] Optionally, the unmanned transport logistics vehicle fault log includes fault events, fault locations, vehicle operating status parameters, system error messages, and error text descriptions.

[0015] Optionally, the fault knowledge graph includes fault entity information, the relationship between environmental parameters and fault entities, and fault event chains that characterize the fault development process.

[0016] According to a second aspect of this application, a fault diagnosis method for unmanned transport logistics vehicles based on a large language model is provided, comprising: The system acquires real-time multimodal fault information of unmanned transport logistics vehicles. The multimodal fault information includes on-board sensor data, natural language fault descriptions, and unmanned transport logistics vehicle fault logs. The unmanned transport logistics vehicle fault logs include fault events, fault locations, vehicle operating status parameters, system error messages, and error text descriptions. Construct a continuously updated fault diagnosis knowledge base, which includes a fault knowledge graph and historical fault logs, and the historical fault logs summarize the causes of historical faults and the experience of troubleshooting historical faults. A pre-trained large language model is jointly trained using the fault knowledge graph and the historical fault logs to determine the fault diagnosis model of the unmanned transport logistics vehicle. The fault diagnosis model of the unmanned transport logistics vehicle integrates a human-computer interaction module. The human-computer interaction module inputs the real-time multimodal fault information of the unmanned transport vehicle into the fault diagnosis model of the unmanned transport vehicle, and determines the fault cause, fault location information, fault troubleshooting suggestions and fault troubleshooting feedback experience output by the fault diagnosis model of the unmanned transport vehicle. Based on the troubleshooting feedback experience, update the continuously updatable fault diagnosis knowledge base and determine a new continuously updatable fault diagnosis knowledge base.

[0017] Optionally, the construction of a continuously updated fault diagnosis knowledge base includes: Retrieve historical fault logs; Keyword extraction processing is performed on the historical fault logs to establish a fault knowledge graph. The fault knowledge graph includes fault entity information, the relationship between environmental parameters and fault entities, and fault event chains that characterize the fault development process.

[0018] Optionally, the keyword extraction process includes entity recognition, relation extraction, and event extraction.

[0019] Optionally, the keyword extraction process includes entity recognition, relation extraction, and event extraction. The step of extracting keywords from the historical fault logs and establishing a fault knowledge graph includes: Perform the entity identification operation on the historical fault log to determine the fault entity information, which includes the fault entity, fault type, and fault entity operating parameters. Perform the relationship extraction operation on the historical fault logs to determine the association between the environmental parameters and the fault entities; The event extraction operation is performed on the historical fault log to determine the fault event chain that characterizes the fault development process. The fault event chain includes time, location, route, event subject, event object, event cause, and fault cause.

[0020] Optionally, updating the continuously updatable fault diagnosis knowledge base based on the fault troubleshooting feedback experience, and determining a new continuously updatable fault diagnosis knowledge base, includes: Keyword extraction and multi-dimensional feature analysis are performed on the troubleshooting feedback experience to determine the fault characteristics; The fault features are matched with knowledge graph nodes in the fault knowledge graph to determine the updated fault knowledge graph. The fault troubleshooting feedback experience or its structured representation is incorporated into the unmanned transport vehicle's fault log. Based on the updated fault knowledge graph and the fault logs of the unmanned transport logistics vehicle, a new, continuously updatable fault diagnosis knowledge base is determined.

[0021] Optionally, the step of performing similarity matching between the fault features and knowledge graph nodes in the fault knowledge graph to determine the updated fault knowledge graph includes: If the entity matching degree between the fault feature and the fault knowledge node in the fault knowledge graph is lower than a preset threshold, a new knowledge graph node is generated, and the updated fault knowledge graph is determined. If the entity matching degree between the fault feature and the knowledge graph node in the fault knowledge graph is not lower than the preset threshold, the knowledge graph node with the highest entity matching degree with the fault feature is updated according to the fault feature, and the updated fault knowledge graph is determined.

[0022] Optionally, the step of jointly training a pre-trained large language model using the fault knowledge graph and the historical fault logs to determine the fault diagnosis model for the unmanned transport logistics vehicle includes: Determine the pre-trained large language model; Load the pre-trained large language model; Freeze the underlying weights of the pre-trained large language model; The cross-entropy loss function is used as the objective function for the joint training of the pre-trained large language model. Determine the joint training parameters of the pre-trained large language model, the joint training parameters including hyperparameters; Define a new task-relevant layer for the pre-trained large language model; Based on the joint training parameters of the pre-trained large language model, the pre-processed fault knowledge graph and historical fault logs are used to jointly train the pre-trained large language model, and the gradient of the objective function with respect to the model parameters is determined by the backpropagation algorithm. The model parameters of the pre-trained large language model are updated according to the preset learning rate until the preset number of training rounds is completed, and the trained large language model is determined. The trained large language model is validated and tested to determine its performance. Based on the performance of the trained large language model after verification and testing, the hyperparameters of the trained large language model are optimized, the trained large language model is adjusted to the optimal training parameters, and the trained large language model corresponding to the optimal training parameters is saved as the fault diagnosis model of the unmanned transport logistics vehicle.

[0023] Compared with the prior art, the embodiments of this application have at least one of the following beneficial effects: Through the above technical solutions, the unmanned transport logistics vehicle fault diagnosis model integrates a human-machine interaction module, constructing a human-machine interaction channel for staff, developers, and the unmanned transport logistics vehicle fault diagnosis model. Based on the multimodal fault information of the unmanned transport logistics vehicle, the fault diagnosis model outputs fault causes, fault location information, fault troubleshooting suggestions, and fault troubleshooting feedback experience through the human-machine interaction module, thereby providing auxiliary fault diagnosis for manual fault diagnosis, reducing the difficulty of manual fault troubleshooting, improving fault diagnosis efficiency, and reducing diagnosis costs. The continuously updated fault diagnosis knowledge base can self-update based on fault troubleshooting feedback experience, and further use the updated continuously updated fault diagnosis knowledge base to train and update the unmanned transport logistics vehicle fault diagnosis model, realizing continuous online training of the model and knowledge base updates, forming a virtuous cycle optimization mechanism, which can learn and adapt to new fault types and fault modes in a timely manner, achieving long-term effective fault diagnosis.

[0024] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0025] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram illustrating the working principle of an unmanned transport logistics vehicle fault diagnosis system based on a large language model, according to an exemplary embodiment.

[0026] Figure 2This is a schematic diagram illustrating a fault diagnosis process using an unmanned transport logistics vehicle fault diagnosis system based on a large language model, according to an exemplary embodiment.

[0027] Figure 3 This is a flowchart illustrating a fault diagnosis method for an unmanned transport logistics vehicle based on a large language model, according to an exemplary embodiment.

[0028] Figure 4 This is a schematic diagram illustrating a process for training a pre-trained large language model according to an exemplary embodiment. Detailed Implementation

[0029] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0030] The terms "comprising" and "having," and any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or devices.

[0031] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0032] Existing fault diagnosis systems for unmanned transport and logistics vehicles can only effectively handle single-type fault scenarios. In scenarios with multiple potential faults coexisting, they struggle to automatically determine the primary and secondary faults and their order of occurrence, hindering troubleshooting and repair work and requiring manual intervention. Furthermore, existing fault diagnosis systems lack adaptability and learning capabilities, relying on fixed diagnostic models and rules, making it difficult to quickly respond to emerging fault modes. Their limited ability to process fault log data also leads to data waste. To address these issues, this application provides a fault log diagnosis system for unmanned transport and logistics vehicles based on a large language model, aiming to resolve these problems.

[0033] Figure 1 This is a schematic diagram illustrating the working principle of a fault log diagnosis system for unmanned transport logistics vehicles based on a large language model, according to an exemplary embodiment.

[0034] This application provides a fault log diagnosis system for unmanned transport logistics vehicles based on a large language model, including: an unmanned transport logistics vehicle fault diagnosis model and a continuously updated fault diagnosis knowledge base.

[0035] The unmanned transport and logistics vehicle fault diagnosis model is used to input multimodal fault information of the unmanned transport and logistics vehicle. It integrates a human-machine interaction module and outputs fault causes, fault location information, fault troubleshooting suggestions, and fault troubleshooting feedback experience through the human-machine interaction module.

[0036] The multimodal fault information of unmanned transport and logistics vehicles includes onboard sensor data, natural language fault descriptions, and unmanned transport and logistics vehicle fault logs.

[0037] Vehicle sensor data can include electrical signals such as battery level, motor current, and motor voltage.

[0038] The fault log of the unmanned transport logistics vehicle includes fault events, fault locations, vehicle operating status parameters, system error messages, and error text descriptions.

[0039] The unmanned transport vehicle fault log is a LOG log generated when the unmanned transport vehicle is running incorrectly.

[0040] The human-computer interaction module establishes a human-computer interaction mechanism, providing a human-computer interaction interface and functions, such as a web interface, for human interaction between staff, developers, and the fault diagnosis model of unmanned transport logistics vehicles.

[0041] The continuously updated fault diagnosis knowledge base is used to store historical fault logs, historical fault causes, and historical fault troubleshooting experience. The continuously updated fault diagnosis knowledge base also includes a fault knowledge graph. The fault knowledge graph and historical fault logs are used to train and update the fault diagnosis model of the unmanned transport vehicle. The continuously updated fault diagnosis knowledge base is also used to update based on fault troubleshooting feedback experience.

[0042] The fault knowledge graph is built based on fault entities, environmental parameters, and maintenance records. The continuously updated fault diagnosis knowledge base contains historical fault logs of unmanned transport logistics vehicles in different scenarios, routes, and operational stages. These historical fault logs summarize the causes of past faults and troubleshooting experiences.

[0043] The continuously updated fault diagnosis knowledge base has the characteristic of continuous updating, and can continuously absorb fault logs and historical fault troubleshooting experience generated by unmanned transport logistics vehicles during operation due to logical problems or other uncontrollable factors.

[0044] Reference Figure 1As shown, the workflow of a fault diagnosis system for unmanned transport logistics vehicles based on a large language model includes: The system uses unmanned logistics vehicles to record fault log text, and builds a continuously updated fault diagnosis knowledge base based on this historical fault log text. This continuously updated fault diagnosis knowledge base is then used to train and update the unmanned logistics vehicle fault diagnosis model. Staff or developers can interact with the unmanned logistics vehicle fault diagnosis model through the web interface of the human-computer interaction module, inputting natural language fault descriptions or uploading fault log text from the unmanned logistics vehicle, and obtaining fault feedback results.

[0045] The training and updating of the fault diagnosis model for unmanned transport logistics vehicles involves processes such as training data processing, obtaining the base model, model fine-tuning, and parameter optimization. The text analysis of the fault logs of unmanned transport logistics vehicles by the fault diagnosis model involves processes such as data cleaning, data augmentation, decision analysis, and response processing.

[0046] In the embodiments described above, the unmanned transport logistics vehicle fault diagnosis model integrates a human-machine interaction module, constructing a human-machine interaction path for staff, developers, and the unmanned transport logistics vehicle fault diagnosis model. Based on the multimodal fault information of the unmanned transport logistics vehicle, the fault diagnosis model outputs fault causes, fault location information, fault troubleshooting suggestions, and fault troubleshooting feedback experience through the human-machine interaction module, thereby providing auxiliary fault diagnosis for manual fault diagnosis, reducing the difficulty of manual fault troubleshooting, improving fault diagnosis efficiency, and reducing diagnosis costs. The continuously updated fault diagnosis knowledge base can be self-updated based on fault troubleshooting feedback experience, and further uses the updated continuously updated fault diagnosis knowledge base to train and update the unmanned transport logistics vehicle fault diagnosis model, realizing continuous online training of the model and knowledge base updates, forming a virtuous cycle optimization mechanism, which can learn and adapt to new fault types and fault modes in a timely manner, achieving long-term effective fault diagnosis.

[0047] In some specific embodiments of this application, the human-computer interaction module is used by staff and developers to input natural language fault descriptions and fault logs of unmanned transport logistics vehicles, and displays the fault causes, fault location information, fault troubleshooting suggestions, and fault troubleshooting feedback experience in the form of a visual interface.

[0048] For example, when a staff member encounters a malfunction in an unmanned logistics vehicle, the staff member sends the vehicle's onboard sensor data to the unmanned logistics vehicle fault diagnosis model, inputs a natural language fault description and uploads the unmanned logistics vehicle fault log into the unmanned logistics vehicle fault diagnosis model in the large language model-based unmanned logistics vehicle fault diagnosis system. The unmanned logistics vehicle fault diagnosis model can analyze the onboard sensor data, natural language fault description and unmanned logistics vehicle fault log, predict the cause of the fault and output fault troubleshooting suggestions through the human-computer interaction module, so as to facilitate the staff member's operation to troubleshoot the fault.

[0049] For example, when developers encounter a difficult-to-solve malfunction of an unmanned logistics vehicle, they can send the vehicle's onboard sensor data to the unmanned logistics vehicle fault diagnosis model, input a natural language fault description, and upload the vehicle's fault log into the unmanned logistics vehicle fault diagnosis model of the large language model-based unmanned logistics vehicle fault log diagnosis system. The unmanned logistics vehicle fault diagnosis model can analyze the onboard sensor data, natural language fault description, and unmanned logistics vehicle fault log, accurately predict the cause of the fault, and output the fault location information through the human-computer interaction module. This facilitates the developers in optimizing the logic layer of the large language model-based unmanned logistics vehicle fault diagnosis system.

[0050] After troubleshooting, the unmanned transport logistics vehicle fault diagnosis model collects and outputs fault troubleshooting feedback experience.

[0051] The embodiments described above in this application, by setting up a human-computer interaction module, facilitate staff to quickly troubleshoot unmanned transport logistics vehicle malfunctions, improve staff troubleshooting efficiency, and also facilitate developers to optimize the logic layer of the unmanned transport logistics vehicle fault diagnosis system based on a large language model, thereby improving the fault diagnosis capability of the unmanned transport logistics vehicle fault diagnosis system based on a large language model.

[0052] In some specific embodiments of this application, the continuously updated fault diagnosis knowledge base includes a fault knowledge graph and historical fault logs. The historical fault logs summarize the causes of historical faults and the experience of troubleshooting historical faults, which are used to train and update the fault diagnosis model of the unmanned transport logistics vehicle.

[0053] Fault knowledge graphs and historical fault logs are used to jointly train the fault diagnosis model for unmanned transport logistics vehicles.

[0054] The updated fault knowledge graph and unmanned transport vehicle fault logs can fine-tune and optimize the jointly trained unmanned transport vehicle fault diagnosis model, i.e., iterative update, to achieve continuous online updating of the model.

[0055] A fault knowledge graph can be built by extracting keywords from the fault diagnosis knowledge base of unmanned transport logistics vehicles. Keyword extraction specifically includes entity recognition, relation extraction, and event extraction. The fault knowledge graph includes fault entity information, the relationship between environmental parameters and fault entities, and fault event chains representing the fault development process.

[0056] Fault entity information includes, but is not limited to, fault entity, fault type, and fault entity operating parameters, such as fault entity: battery, controller; Fault types include, but are not limited to: communication interruption, motor failure, failure to run upon receiving instructions, scheduling deadlock, vehicle collision, inaccurate positioning due to missed vehicle location scanning, and human error. The operating parameters of the faulty entity include, but are not limited to: vehicle battery level, vehicle route, vehicle loss of control, received dispatch instructions, load information, current vehicle location, and next station location.

[0057] The relationship between environmental parameters and faulty entities includes, but is not limited to: failure to operate upon receiving instructions - communication interruption or other reasons - sensor malfunction or network fluctuation.

[0058] The fault event chain includes, but is not limited to: such as the collision fault of unmanned transport logistics vehicles and the fault event chain of unmanned transport logistics vehicles. The fault event chain of unmanned transport logistics vehicle collision fault is: time-[specific time], location-[specific location], route-[vehicle route], event subject-[vehicle number], event object-[damaged parts], event cause-[collision cause], fault cause-[vehicle did not receive message].

[0059] The embodiments described above in this application train and update the fault diagnosis model of the unmanned transport logistics vehicle through a continuously updated fault diagnosis knowledge base, thereby achieving online continuous updating of the model and continuously improving the fault diagnosis capability of the unmanned transport logistics vehicle fault diagnosis model.

[0060] This application provides a fault diagnosis system for unmanned transport and logistics vehicles based on a large language model. The fault diagnosis model for unmanned transport and logistics vehicles is trained using open-source pre-trained large models as the model base, including but not limited to Lora, Chatglm3, LLaMA2, Lora, Chatglm3, and LLaMA2. The above model bases have strong understanding, reasoning, and continuous dialogue capabilities, but their ability to handle tasks in the professional field of unmanned transport and logistics vehicles is not outstanding. Therefore, they need to be trained and updated before they can be used as the fault diagnosis model for unmanned transport and logistics vehicles.

[0061] The fault diagnosis model for unmanned transport logistics vehicles can be embedded in the operation and maintenance software of staff, or in the development tools of developers.

[0062] When an unmanned transport vehicle malfunctions, staff can input the multimodal fault information of the unmanned transport vehicle into the unmanned transport vehicle fault diagnosis model through the human-computer interaction module to obtain the cause of the fault and troubleshooting suggestions. In addition, staff can input newly observed information into the unmanned transport vehicle fault diagnosis model to obtain further fault auxiliary diagnosis information.

[0063] Developers can obtain the fault analysis results of the unmanned transportation logistics vehicle fault diagnosis model through the human-computer interaction module, such as fault cause and fault location information, and obtain the optimization direction of the system.

[0064] The fault diagnosis system for unmanned transport logistics vehicles based on a large language model provided in this application has intelligent interactive capabilities, which can continuously update fault diagnosis information and improve the accuracy, efficiency and intelligence level of fault diagnosis.

[0065] Figure 2 This is a schematic diagram illustrating a fault diagnosis process using an unmanned transport logistics vehicle fault diagnosis system based on a large language model, according to an exemplary embodiment.

[0066] Reference Figure 2 As shown, during the startup phase, the user starts the unmanned transport logistics vehicle, records the cause of the fault based on the faults that occur during transportation, and saves the log text generated by the vehicle during transportation as acquired experience knowledge, namely the unmanned transport logistics vehicle fault log, and establishes an unmanned transport logistics vehicle fault knowledge graph.

[0067] The base model is obtained based on the server configuration. Models such as LLAMA or LoRa can be used as the base model for fine-tuning. During fine-tuning, the model's hyperparameters are continuously optimized to achieve the best accuracy in identifying different faults. Staff and developers interact with the server through a PC website to ultimately troubleshoot the problem. After troubleshooting, feedback and experience are gathered, and the knowledge base and the fault knowledge graph of the unmanned transport logistics vehicle are updated.

[0068] Figure 3 This is a flowchart illustrating a fault diagnosis method for an unmanned transport logistics vehicle based on a large language model, according to an exemplary embodiment.

[0069] like Figure 3 As shown, this application provides a fault diagnosis method for unmanned transport logistics vehicles based on a large language model, including S11 to S15.

[0070] S11, acquire real-time multimodal fault information of unmanned transport logistics vehicles.

[0071] The multimodal fault information includes onboard sensor data, natural language fault descriptions, and fault logs from unmanned transport logistics vehicles.

[0072] The fault log of the unmanned transport logistics vehicle includes the fault event, fault location, vehicle operating status parameters, system error messages, and error descriptions. The system error messages are the system error codes.

[0073] The causes of multimodal fault information may include, but are not limited to, human error during vehicle operation, production workshop environment, unstable connection between vehicle controller and server, and software system logic problems.

[0074] S12, Build a continuously updated fault diagnosis knowledge base.

[0075] Specifically, the continuously updated fault diagnosis knowledge base includes a fault knowledge graph and historical fault logs, which summarize the causes of historical faults and troubleshooting experiences.

[0076] The fault knowledge graph is constructed based on fault entities, environmental parameters, and maintenance records. It includes fault entity information, the relationship between environmental parameters and fault entities, and fault event chains that characterize the fault development process.

[0077] The continuously updated fault diagnosis knowledge base includes historical fault logs from different scenarios, routes, and operational stages.

[0078] S13. By jointly training a pre-trained large language model using a fault knowledge graph and historical fault logs, a fault diagnosis model for unmanned transport logistics vehicles is determined.

[0079] Among them, the fault diagnosis model for unmanned transport logistics vehicles integrates a human-machine interaction module.

[0080] Pre-trained large language models can be derived from open-source pre-trained large models, including but not limited to Lora, Chatglm3, and LLaMA2.

[0081] The large language model is pre-trained using a fault knowledge graph and historical fault logs. The main approach is to use a hybrid training mode that combines structured data from the fault knowledge graph with unstructured data from the historical fault logs. By combining graph relation reasoning and log sequence analysis, the diagnostic accuracy is improved. This enables the development of a diagnostic model with fault reasoning capabilities through multimodal data fusion training, which is the fault diagnosis model for unmanned transport logistics vehicles.

[0082] S14. Input real-time multimodal fault information of unmanned transportation and logistics vehicles into the fault diagnosis model of unmanned transportation and logistics vehicles through the human-computer interaction module, and determine the fault cause, fault location information, fault troubleshooting suggestions and fault troubleshooting feedback experience output by the fault diagnosis model of unmanned transportation and logistics vehicles.

[0083] The human-computer interaction module serves as a medium for staff and developers to interact with the unmanned transport and logistics vehicle fault diagnosis model. Staff and developers input multimodal fault information into the unmanned transport and logistics vehicle fault diagnosis model through the human-computer interaction module. Furthermore, the output results of the unmanned transport and logistics vehicle fault diagnosis model are visualized in the human-computer interaction module. For example, the human-computer interaction module displays visualized diagnostic results including fault causes and location coordinates, and generates fault troubleshooting suggestions.

[0084] S15. Based on troubleshooting feedback experience, update the continuously updatable fault diagnosis knowledge base and determine a new continuously updatable fault diagnosis knowledge base.

[0085] Specifically, updating the continuously updatable fault diagnosis knowledge base includes updating the graph nodes in the fault knowledge graph and the historical case library of historical fault logs.

[0086] In the embodiments described above, the large language model for fault diagnosis of unmanned transport logistics vehicles provides human-computer interaction functions. It inputs multimodal fault information of unmanned transport logistics vehicles and outputs fault causes, fault location information, and fault troubleshooting suggestions, providing auxiliary fault diagnosis for manual fault diagnosis, reducing the difficulty of manual fault troubleshooting, improving fault diagnosis efficiency, and reducing diagnosis costs. The large language model for fault diagnosis of unmanned transport logistics vehicles also collects and outputs fault troubleshooting feedback experience. Based on the fault troubleshooting feedback experience, it updates the fault diagnosis knowledge base of unmanned transport logistics vehicles, realizing continuous updating of the fault diagnosis knowledge base and timely learning and adapting to new fault types and patterns.

[0087] In some specific embodiments of this application, S12, a continuously updated fault diagnosis knowledge base is constructed, including S121 to S122.

[0088] S121, retrieve historical fault logs.

[0089] Specifically, the historical fault logs are the historical fault logs of unmanned transport vehicles in different scenarios, routes, and operational stages.

[0090] S122, extract keywords from historical fault logs and establish a fault knowledge graph.

[0091] Specifically, the fault knowledge graph includes fault entity information, the relationship between environmental parameters and fault entities, and event chains that characterize the fault development process.

[0092] Since historical fault logs may contain a large number of duplicate information commands and corresponding logical relationships sent by different vehicles, keyword extraction is used to clean the historical fault logs. A fault knowledge graph is then constructed based on fault entities, environmental parameters, and maintenance records to facilitate understanding by the subsequent pre-trained large language model.

[0093] Specifically, keyword extraction processing includes entity recognition, key extraction, and time extraction.

[0094] To construct a fault knowledge graph, in some specific embodiments of this application, S122, keyword extraction processing is performed on historical fault logs to establish a fault knowledge graph, including: S1221 to S1223.

[0095] S1221, Perform entity identification operation on historical fault logs to determine fault entity information.

[0096] Specifically, the fault entity information includes the fault entity, the fault type, and the fault entity's operating parameters; The entity concepts in the text of historical fault logs are identified by classification methods, and fault entity information is identified, such as fault entities: battery, controller; Fault types include, but are not limited to, communication interruption, motor failure, failure to run upon receiving instructions, scheduling deadlock, vehicle collision, inaccurate positioning due to missed vehicle location scanning, and human error. The operating parameters of the faulty entity include, but are not limited to, vehicle battery level, vehicle route, vehicle loss of control, received dispatch instructions, load information, current vehicle location, and next station location.

[0097] S1222, Perform a relationship extraction operation on the historical fault log to determine the association between environmental parameters and fault entities.

[0098] By classifying historical fault logs, we extract the relationships and logical problems between related entities, faults, and environmental parameters, such as failure to execute received instructions due to communication interruption or other reasons such as sensor malfunction or network fluctuation.

[0099] S1223, Perform an event extraction operation on the historical fault log to determine the fault event chain that represents the fault development process.

[0100] Specifically, the fault event chain that characterizes the fault development process includes time, location, route, event subject, event object, event cause, and fault cause.

[0101] By identifying the preset entities that trigger the events, the fault events are determined, and the event type is used as a trigger to analyze the fault event chain, such as the collision fault of unmanned transport logistics vehicles and the fault event chain of unmanned transport logistics vehicles. The event chain of the collision fault of unmanned transport logistics vehicles is: time-[specific time], location-[specific location], route-[vehicle route], event subject-[vehicle number], event object-[damaged parts], event cause-[collision cause], fault cause-[vehicle did not receive message].

[0102] In order to train the pre-trained large language model, in some specific embodiments of this application, S13, the pre-trained large language model is jointly trained using a fault knowledge graph and historical fault logs to determine the fault diagnosis model of the unmanned transport logistics vehicle. The fault diagnosis model of the unmanned transport logistics vehicle integrates a human-computer interaction module, which may include S131 to S139.

[0103] S131, determine the pre-trained large language model.

[0104] The pre-trained large language model uses open-source pre-trained large models, including but not limited to Lora, Chatglm3, LLaMA2, Lora, Chatglm3, and LLaMA2. In this application, Lora was selected as the pre-trained large language model.

[0105] S132, Load the pre-trained large language model.

[0106] One approach is to use an open-source deep learning framework, such as PyTorch, to load a pre-trained large language model.

[0107] S133, freeze the underlying weights of the pre-trained large language model.

[0108] This method involves freezing the weights of the underlying layers of the selected pre-trained large language model, freezing most of the parameters of the pre-trained large language model, training with only a small number of parameters, and retaining the features learned in the original pre-training task. By using Freeze for fine-tuning, it is possible to ensure that the knowledge learned by the model can be fully utilized, reduce the computational resources and time consumption during the fine-tuning process, and eliminate gradient updates, which is a low-cost training method.

[0109] S134 uses the cross-entropy loss function as the objective function for joint training of pre-trained large language models.

[0110] In this model, the correct fault diagnosis result is used as the label, and the difference between the model prediction result and the label is used as the loss value.

[0111] The labels for correct fault diagnosis results come from a continuously updated fault diagnosis knowledge base. By analyzing and organizing fault cases in the fault diagnosis knowledge base of unmanned transport logistics vehicles, the fault type corresponding to each fault log is determined, thereby determining the label for the correct fault diagnosis result.

[0112] S135, determine the joint training parameters of the pre-trained large language model.

[0113] The joint training parameters include hyperparameters. Hyperparameters include learning rate, number of training epochs, and batch size.

[0114] S136 defines a new task-related layer for a pre-trained large language model.

[0115] Among them, new layers or modules are added to the top or middle of the pre-trained large language model to adapt to the fault diagnosis task of unmanned transport logistics vehicles.

[0116] Taking LoRa fine-tuning model training as an example: In the task of diagnosing fault logs in unmanned transportation logistics vehicles, it is necessary to consider applying LoRa fine-tuning to key layers in the model related to fault log diagnosis, including layers related to feature extraction, classification, and prediction. The top or intermediate layers of a pre-trained large language model, specifically the fully connected layers responsible for processing the semantic information of the fault log text, can be used as targets for LoRa fine-tuning.

[0117] Initialize the Lora matrix and modify it during forward propagation, backpropagation, and gradient updates. Update the initialized Lora matrix using an optimization algorithm (such as Adam or SGD).

[0118] Besides applying LoRa fine-tuning, it can also be combined with other relevant task layers, such as convolutional layers (CNN layers) and recurrent layers (RNN layers), to better adapt to the fault log diagnosis task of unmanned transportation logistics vehicles. For example, after the fully connected layer with LoRa fine-tuning, a convolutional layer can be added to further extract local information from the fault log, or a recurrent layer can be added to analyze the time series information of the fault.

[0119] S137. Based on the joint training parameters of the pre-trained large language model, the pre-processed fault knowledge graph and historical fault logs are used to jointly train the pre-trained large language model. The backpropagation algorithm is used to determine the gradient of the objective function with respect to the model parameters. The model parameters of the pre-trained large language model are updated according to the preset learning rate until the preset number of training rounds is completed, and the trained large language model is determined.

[0120] For example, the data in the fault knowledge graph can be preprocessed, including text normalization, formatting, data augmentation, and data cleaning. Feature extraction is performed on historical fault log data to generate time-series feature vectors and fault type labels, which are then associated with entities in the fault knowledge graph.

[0121] The fault knowledge graph provides structured fault entity relationships and reasoning networks, which are then combined with unstructured information from historical fault logs for multimodal fusion. Through joint input of the fault knowledge graph and historical fault log data, cross-modal joint training is performed, enabling the model to simultaneously learn the entity relationships from the fault knowledge graph and the temporal information from the historical fault logs.

[0122] The preprocessed fault knowledge graph and historical fault logs are divided into training set, validation set and test set. The training set is used for model training, the validation set is used for adjusting model parameters and selecting models, and the test set is used for final evaluation of model performance. The commonly used ratio is 70% training set, 15% validation set and 15% test set.

[0123] The training set is divided into multiple batches and sequentially fed into the pre-trained large language model to facilitate model training. Before the start of each training cycle, the training set data is shuffled to prevent the model from learning the order of the data. Multithreading or multiprocessing is used to accelerate data loading.

[0124] The batch size is determined based on the training efficiency and the server configuration, and is generally set to 64. If the server configuration is sufficient, the batch size can be set to 128.

[0125] When training a pre-trained large language model, a fine-tuning strategy can be adopted. Based on the set training parameters of the pre-trained large language model, i.e., hyperparameters, the backpropagation algorithm is used to calculate the gradient of the loss value with respect to the model parameters.

[0126] The model parameters are updated based on the set learning rate to minimize the loss function.

[0127] Repeat the above process until the preset number of training rounds is completed.

[0128] S138, validate and test the trained large language model to determine its performance.

[0129] The training process involves using validation set data to validate the trained large language model, using test set data to test the trained large language model, comparing the prediction results of the trained large language model with the real labels, and using preset performance evaluation metrics such as accuracy, recall, and F1 score to evaluate the performance of the trained large language model in the fault diagnosis task.

[0130] Evaluate the performance of the trained large language model to ensure good performance and avoid overfitting or underfitting.

[0131] S139. Based on the performance of the trained large language model after verification and testing, perform hyperparameter optimization on the trained large language model, adjust the trained large language model to the optimal training parameters, and save the trained large language model corresponding to the optimal training parameters as the fault diagnosis model of the unmanned transport logistics vehicle.

[0132] Among them, the optimal training parameters are the optimal hyperparameters. Based on the performance of the large language model after validation and testing, the hyperparameters, such as the learning rate, regularization (such as L1 and L2 regularization coefficients), and batch size parameters, are fine-tuned to improve the performance of the large language model after training.

[0133] When the trained large language model achieves optimal performance on the validation set (i.e., the hyperparameter tuning values ​​are optimal), the trained large language model is saved as a fault diagnosis model for unmanned transport logistics vehicles, and the model's weights are saved to a specified path.

[0134] In this application, after completing steps S131 to S139, steps S137 to S139 can be repeated multiple times to iteratively update the unmanned transport logistics vehicle fault diagnosis model, so as to further improve the model performance, continuously monitor the model performance, and make fine adjustments and improvements according to actual needs.

[0135] Figure 4 This is a schematic diagram illustrating a process for training a pre-trained large language model according to an exemplary embodiment.

[0136] Reference Figure 4 As shown, the process of training a pre-trained large language model includes: Data processing: collection of fault logs from unmanned transport logistics vehicles, data preprocessing, and preparation of training and test data.

[0137] For the collection of fault logs from unmanned transport logistics vehicles, only historical fault troubleshooting experience needs to be provided, including the cause of the fault and the fault log text, which facilitates the model analysis of the fault logic.

[0138] Model training: Optimize the model architecture, initialize parameters, calculate the loss function, and optimize parameters using backpropagation.

[0139] Among these methods, the hyperparameters are repeatedly optimized using a loss function to achieve the best accuracy in judging and analyzing different types of faults.

[0140] Fault diagnosis: The model training part provides the complete training and optimization of the network, and the data processing part provides input fault log text, obtain fault troubleshooting feedback experience, and output fault diagnosis results.

[0141] Staff can gain troubleshooting feedback and experience while entering fault log text.

[0142] After training the unmanned logistics vehicle fault diagnosis model, the fault logs of the unmanned logistics vehicle are input into the model. The model outputs fault causes, troubleshooting suggestions, fault location information, and fault diagnosis information. Staff verify the authenticity of the fault logs and score the output of the fault diagnosis model. Developers can then optimize the model based on the feedback from the logs.

[0143] In some specific embodiments of this application, S14 involves inputting real-time multimodal fault information of the unmanned transport vehicle into the unmanned transport vehicle fault diagnosis model through the human-computer interaction module, and determining the fault cause, fault location information, fault troubleshooting suggestions, and fault troubleshooting feedback experience output by the unmanned transport vehicle fault diagnosis model. This may include: The fault diagnosis model for unmanned transport logistics vehicles in this application provides interactive prompts for staff and developers during the fault diagnosis process.

[0144] During use, the unmanned transport and logistics vehicle fault diagnosis model is embedded in the operation and maintenance software of staff and the development tools of developers. Staff and R&D personnel interact with the unmanned transport and logistics vehicle fault diagnosis model by sending on-board sensor data, inputting natural language fault descriptions through text, and directly uploading fault logs of unmanned transport and logistics vehicles. For example, they can describe in text that a vehicle has a problem with the failure to execute dispatch information instructions or upload the fault logs of the dispatched vehicle to obtain the fault cause, fault location information, and fault troubleshooting suggestions from the unmanned transport and logistics vehicle fault diagnosis model.

[0145] Staff and developers can also input newly observed information into the fault diagnosis model of unmanned transport logistics vehicles to obtain further fault-aiding diagnosis information.

[0146] For example, staff input multimodal fault information into the unmanned transport vehicle fault diagnosis model, and the unmanned transport vehicle fault diagnosis model outputs the fault cause and troubleshooting suggestions to the staff.

[0147] In this process, staff interact with the fault diagnosis model of the unmanned transport logistics vehicle to obtain the possible causes of the fault and the measures to be taken to address them.

[0148] For example, developers input multimodal fault information into the unmanned transportation logistics vehicle fault diagnosis model, and the unmanned transportation fault diagnosis big language model outputs fault cause and fault location information to the developers.

[0149] In this system, developers can interact with the unmanned transport vehicle fault diagnosis model to obtain the specific location of the unmanned transport vehicle fault log and return the logical problems that may occur at the specific location of the fault log, which facilitates developers to maintain and improve the software code regularly.

[0150] In some specific embodiments of this application, after troubleshooting the unmanned transport logistics vehicle, the unmanned transport logistics vehicle fault diagnosis model collects and outputs fault troubleshooting feedback experience.

[0151] After the unmanned logistics vehicle malfunctions and the staff confirms that the malfunction has been resolved, the unmanned logistics vehicle malfunction diagnosis model automatically collects the malfunction troubleshooting experience from the malfunction log text of the unmanned logistics vehicle, such as the specific logical reasons for the error, including: malfunction time, location, and route. This information is used to update the continuously updated malfunction diagnosis knowledge base, which can then be used to update the unmanned logistics vehicle malfunction diagnosis model.

[0152] Specifically, after the unmanned transport logistics vehicle fault diagnosis model is used for diagnosis, the unmanned transport logistics vehicle fault diagnosis model interacts with staff and developers, requesting them to confirm the true cause of the fault, the logical relationship in the text of the unmanned transport logistics vehicle fault log, and the problem handling results. After confirmation, these are used as input information to collect fault troubleshooting feedback experience in the unmanned transport logistics vehicle fault diagnosis model, and to update the continuously updated fault diagnosis knowledge base.

[0153] For example, if a fault is successfully eliminated under the guidance of the unmanned transport logistics vehicle fault diagnosis model, the fault type and fault text in the corresponding unmanned transport logistics vehicle fault log are input into a continuously updated fault diagnosis knowledge base.

[0154] For example, if a fault is not successfully resolved under the guidance of the unmanned transport vehicle fault diagnosis model, the guidance provided by the unmanned transport vehicle fault diagnosis model for this type of fault is recorded. The data in the continuously updated fault diagnosis knowledge base corresponding to this type of fault is then adjusted and updated. The unmanned transport vehicle fault diagnosis model is fine-tuned and updated until it can successfully guide the diagnosis of this type of fault, including determining the error location and logical relationship in the fault log. New faults can also be continuously proposed to assess the accuracy of the unmanned transport vehicle fault diagnosis model. If the accuracy of the unmanned transport vehicle fault diagnosis model is incorrect, the continuously updated fault diagnosis knowledge base is adjusted.

[0155] Both successful and unsuccessful diagnostic results are beneficial for fine-tuning the fault diagnosis model of unmanned logistics vehicles. A large number of fault logs from unmanned logistics vehicles can be recorded and used as an expert database—a continuously updated fault diagnosis knowledge base. For example, using 5000 identical or different fault logs from unmanned logistics vehicles as a continuously updated fault diagnosis knowledge base, and performing keyword extraction to establish a fault knowledge graph, can be used to train or update the fault diagnosis model of unmanned logistics vehicles. In particular, the different types of faults and different fault logs of the same fault in the continuously updated fault diagnosis knowledge base can improve the judgment accuracy of the fault diagnosis model of unmanned logistics vehicles.

[0156] In some specific embodiments of this application, S15, based on troubleshooting feedback experience, updates the continuously updatable fault diagnosis knowledge base and determines a new continuously updatable fault diagnosis knowledge base, which may include: S151 to S154.

[0157] S151, Keyword extraction and multi-dimensional feature analysis are performed on the fault troubleshooting feedback experience to determine the fault characteristics.

[0158] The keyword extraction process can employ a pre-defined keyword extraction model. Furthermore, the keyword extraction model is used to structure the fault troubleshooting feedback experience, including: fault entity identification, environmental parameter correlation analysis, and fault event reconstruction. Then, multi-dimensional feature analysis is performed to obtain the fault features in the fault troubleshooting feedback experience.

[0159] S152, perform similarity matching between fault features and knowledge graph nodes in the fault knowledge graph to determine the updated fault knowledge graph.

[0160] For example, if the entity matching degree between the fault feature and the fault knowledge node in the fault knowledge graph is lower than a preset threshold, a new knowledge graph node is generated, and an updated fault knowledge graph is determined.

[0161] Specifically, for new knowledge graph nodes, graph neural networks can be used to reason about node relationships, forming new knowledge image node relationships, which are then added to the original fault knowledge graph to determine the updated fault knowledge graph.

[0162] For example, if the entity matching degree between the fault feature and the knowledge graph node in the fault knowledge graph is not lower than a preset threshold, the knowledge graph node with the highest entity matching degree with the fault feature is updated according to the fault feature, and the updated fault knowledge graph is determined.

[0163] S153, incorporate troubleshooting feedback experience or a structured representation of troubleshooting feedback experience into the unmanned transport vehicle's fault log.

[0164] Specifically, troubleshooting feedback experience or a structured representation of troubleshooting feedback experience after structured processing can be directly summarized into the unmanned transport vehicle's fault log.

[0165] S154. Based on the updated fault knowledge graph and the fault logs of the unmanned transport logistics vehicle, determine a new, continuously updatable fault diagnosis knowledge base.

[0166] Among them, the fault logs of unmanned transport logistics vehicles, as historical cases extracted from fault troubleshooting feedback experience, are added to the historical fault log library to facilitate iterative updates of the pre-trained large language model.

[0167] Based on the above steps S151 to S154, for example, the fault troubleshooting feedback experience report is structured through a keyword extraction model to extract entity update features and environmental parameter association features. Node similarity matching and relationship reasoning are performed with the fault knowledge graph. When a new fault mode is detected, a graph expansion request is generated. Based on the graph expansion request, a continuously updated fault diagnosis knowledge base update process is initiated. A graph neural network is used to learn the relationship embedding of newly added fault nodes to generate a new continuously updated fault diagnosis knowledge base.

[0168] In some specific embodiments of this application, a fault diagnosis method for an unmanned transport logistics vehicle based on a large language model may further include S16.

[0169] S16. The unmanned transport vehicle fault diagnosis model is jointly trained based on the updated fault knowledge graph in the new continuously updated fault diagnosis knowledge base and the fault log of the unmanned transport vehicle, and the iteratively updated unmanned transport vehicle fault diagnosis model is determined.

[0170] Among them, the updated fault knowledge graph and historical fault logs are jointly trained through a hybrid training mode to complete the iterative update of the fault diagnosis model for unmanned transport logistics vehicles.

[0171] Specifically, steps S137 to S139 are performed using an updated fault knowledge graph and fault logs of unmanned transport logistics vehicles. The fault diagnosis model of unmanned transport logistics vehicles is iteratively trained and continuously updated to obtain new training text datasets and new, continuously updated fault diagnosis knowledge bases. This optimizes the fault diagnosis model of unmanned transport logistics vehicles and enriches the continuously updated fault diagnosis knowledge base, thereby improving the generalization of fault diagnosis.

[0172] The embodiments described above enable continuous online iterative updates of the model and knowledge base, forming a virtuous cycle optimization mechanism that can learn and adapt to new fault types and fault modes in a timely manner, thus achieving long-term effective fault diagnosis.

[0173] This application provides a fault diagnosis system and method for unmanned transport and logistics vehicles based on a large language model. These systems can assist manual fault diagnosis, reduce the difficulty of manual troubleshooting, improve fault diagnosis efficiency, and reduce diagnosis costs. Furthermore, they enable continuous online training and knowledge base updates, forming a virtuous cycle optimization mechanism that allows the system to learn and adapt to new fault types and modes in a timely manner, thereby achieving long-term effective fault diagnosis.

[0174] The specific embodiments of this application have been described above. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A fault diagnosis system for unmanned transport logistics vehicles based on a large language model, characterized in that, include: An unmanned transport logistics vehicle fault diagnosis model is used to input multimodal fault information of the unmanned transport logistics vehicle, integrate a human-machine interaction module, and output fault causes, fault location information, fault troubleshooting suggestions and fault troubleshooting feedback experience through the human-machine interaction module. The multimodal fault information includes on-board sensor data, natural language fault descriptions and unmanned transport logistics vehicle fault logs. A continuously updatable fault diagnosis knowledge base is used to store historical fault logs, historical fault causes, and historical fault troubleshooting experience. The continuously updatable fault diagnosis knowledge base also includes a fault knowledge graph. The fault knowledge graph and the historical fault logs are used to train and update the fault diagnosis model of the unmanned transport logistics vehicle. The continuously updatable fault diagnosis knowledge base is also used to self-update based on the fault troubleshooting feedback experience.

2. The fault diagnosis system for unmanned transport logistics vehicles based on a large language model according to claim 1, characterized in that, The human-computer interaction module is used by staff and developers to input the natural language fault description and the fault log of the unmanned transport logistics vehicle, and displays the fault cause, fault location information, fault troubleshooting suggestions and fault troubleshooting feedback experience in the form of a visual interface.

3. The fault diagnosis system for unmanned transport logistics vehicles based on a large language model according to claim 2, characterized in that, The fault log of the unmanned transport logistics vehicle includes fault events, fault locations, vehicle operating status parameters, system error messages, and error text descriptions.

4. The fault diagnosis system for unmanned transport logistics vehicles based on a large language model according to claim 1, characterized in that, The fault knowledge graph includes fault entity information, the relationship between environmental parameters and fault entities, and fault event chains that characterize the fault development process.

5. A fault diagnosis method for unmanned transport logistics vehicles based on a large language model, characterized in that, include: The system acquires real-time multimodal fault information of unmanned transport logistics vehicles. The multimodal fault information includes on-board sensor data, natural language fault descriptions, and unmanned transport logistics vehicle fault logs. The unmanned transport logistics vehicle fault logs include fault events, fault locations, vehicle operating status parameters, system error messages, and error text descriptions. Construct a continuously updated fault diagnosis knowledge base, which includes a fault knowledge graph and historical fault logs, and the historical fault logs summarize the causes of historical faults and the experience of troubleshooting historical faults. A pre-trained large language model is jointly trained using the fault knowledge graph and the historical fault logs to determine the fault diagnosis model of the unmanned transport logistics vehicle. The fault diagnosis model of the unmanned transport logistics vehicle integrates a human-computer interaction module. The human-computer interaction module inputs the real-time multimodal fault information of the unmanned transport vehicle into the fault diagnosis model of the unmanned transport vehicle, and determines the fault cause, fault location information, fault troubleshooting suggestions and fault troubleshooting feedback experience output by the fault diagnosis model of the unmanned transport vehicle. Based on the troubleshooting feedback experience, update the continuously updatable fault diagnosis knowledge base and determine a new continuously updatable fault diagnosis knowledge base.

6. The method according to claim 5, characterized in that, The construction of a continuously updated fault diagnosis knowledge base includes: Retrieve historical fault logs; Keyword extraction processing is performed on the historical fault logs to establish a fault knowledge graph. The fault knowledge graph includes fault entity information, the relationship between environmental parameters and fault entities, and fault event chains that characterize the fault development process.

7. The method according to claim 6, characterized in that, The keyword extraction process includes entity recognition, relation extraction, and event extraction. The step of extracting keywords from the historical fault logs and establishing a fault knowledge graph includes: Perform the entity identification operation on the historical fault log to determine the fault entity information, which includes the fault entity, fault type, and fault entity operating parameters. Perform the relationship extraction operation on the historical fault logs to determine the association between the environmental parameters and the fault entities; The event extraction operation is performed on the historical fault log to determine the fault event chain that characterizes the fault development process. The fault event chain includes time, location, route, event subject, event object, event cause, and fault cause.

8. The method according to claim 5, characterized in that, The step of updating the continuously updatable fault diagnosis knowledge base based on the fault troubleshooting feedback experience, and determining a new continuously updatable fault diagnosis knowledge base, includes: Keyword extraction and multi-dimensional feature analysis are performed on the troubleshooting feedback experience to determine the fault characteristics; The fault features are matched with knowledge graph nodes in the fault knowledge graph to determine the updated fault knowledge graph. The fault troubleshooting feedback experience or its structured representation is incorporated into the unmanned transport vehicle's fault log. Based on the updated fault knowledge graph and the fault logs of the unmanned transport logistics vehicle, a new, continuously updatable fault diagnosis knowledge base is determined.

9. The method according to claim 8, characterized in that, The step of performing similarity matching between the fault features and knowledge graph nodes in the fault knowledge graph to determine the updated fault knowledge graph includes: If the entity matching degree between the fault feature and the fault knowledge node in the fault knowledge graph is lower than a preset threshold, a new knowledge graph node is generated, and the updated fault knowledge graph is determined. If the entity matching degree between the fault feature and the knowledge graph node in the fault knowledge graph is not lower than the preset threshold, the knowledge graph node with the highest entity matching degree with the fault feature is updated according to the fault feature, and the updated fault knowledge graph is determined.

10. The method according to claim 6, characterized in that, The step of using the fault knowledge graph and the historical fault logs to jointly train a pre-trained large language model to determine the fault diagnosis model for unmanned transport logistics vehicles includes: Determine the pre-trained large language model; Load the pre-trained large language model; Freeze the underlying weights of the pre-trained large language model; The cross-entropy loss function is used as the objective function for the joint training of the pre-trained large language model. Determine the joint training parameters of the pre-trained large language model, the joint training parameters including hyperparameters; Define a new task-relevant layer for the pre-trained large language model; Based on the joint training parameters of the pre-trained large language model, the pre-processed fault knowledge graph and historical fault logs are used to jointly train the pre-trained large language model, and the gradient of the objective function with respect to the model parameters is determined by the backpropagation algorithm. The model parameters of the pre-trained large language model are updated according to the preset learning rate until the preset number of training rounds is completed, and the trained large language model is determined. The trained large language model is validated and tested to determine its performance. Based on the performance of the trained large language model after verification and testing, the hyperparameters of the trained large language model are optimized, the trained large language model is adjusted to the optimal training parameters, and the trained large language model corresponding to the optimal training parameters is saved as the fault diagnosis model of the unmanned transport logistics vehicle.

Images