Equipment fault maintenance method and system based on large model, and storage medium

Abstract

The invention provides an equipment fault maintenance method and system based on a large model, and a storage medium. The method comprises the following steps: determining fault equipment according to equipment fault description; obtaining current operation data and a current operation image of the fault equipment, and inputting the equipment fault description, the current operation data and the current operation image into a pre-trained large model for fault identification to obtain a target fault reason; determining a priority troubleshooting step according to the target fault reason, and sending the target fault reason and the priority troubleshooting step to the user; a troubleshooting feedback result for the priority troubleshooting step is received, and a real fault reason is determined according to the troubleshooting feedback result; and generating an equipment fault maintenance result according to the equipment fault description, the current operation data, the current operation image and the real fault reason. According to the embodiment of the invention, the target fault reason and the priority troubleshooting step are sent to the user, thereby effectively assisting the equipment fault maintenance, and improving the equipment fault maintenance efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fault maintenance, and in particular to a device fault maintenance method and system based on a large model and a storage medium. BACKGROUND

[0002] Device fault maintenance is a key link for stable operation of equipment in the fields of industrial manufacturing, transportation, etc. With the complication of equipment types and the integration of system structures, the coupling relationship between equipment is complex, and single fault often involves multiple system interactions, greatly increasing the difficulty of fault troubleshooting and processing.

[0003] In the existing device fault maintenance process, diagnosis and processing are generally based on manual experience, resulting in low efficiency of device fault maintenance. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a device fault maintenance method and system based on a large model and a storage medium to solve the problem of low efficiency of device fault maintenance in the prior art.

[0005] The embodiments of the present application are implemented in the following way. A device fault maintenance method based on a large model, the method comprising:

[0006] Obtaining a device fault description sent by a user, and determining a fault device according to the device fault description;

[0007] Obtaining current running data and a current running image of the fault device, and inputting the device fault description, the current running data and the current running image into a pre-trained large model for fault identification to obtain a target fault cause;

[0008] Determining a priority troubleshooting step according to the target fault cause, and sending the target fault cause and the priority troubleshooting step to the user;

[0009] Receiving a troubleshooting feedback result for the priority troubleshooting step, and determining a real fault cause according to the troubleshooting feedback result;

[0010] Generating a device fault maintenance result according to the device fault description, the current running data, the current running image and the real fault cause.

[0011] Preferably, before inputting the device fault description, the current running data and the current running image into the pre-trained large model for fault identification, the method further comprises:

[0012] Obtaining a sample fault description, sample operation data and sample operation images, and inputting the sample fault description, the sample operation data and the sample operation images into the large model for structured processing to obtain fault structure information, operation structure information and image structure information;

[0013] Performing semantic recognition on the fault structure information to obtain sample description semantics, and performing parameter feature extraction on the operation structure information to obtain sample parameter features;

[0014] Performing image feature extraction on the image structure information to obtain sample image features, and performing feature fusion on the sample description semantics, the sample parameter features and the sample image features to obtain sample fusion features;

[0015] Performing fault prediction according to the sample fusion features to obtain sample predicted faults, and determining sample troubleshooting steps according to the sample predicted faults;

[0016] Calculating a model loss according to the sample predicted faults and the sample troubleshooting steps, and performing parameter updating on the large model according to the model loss until the large model converges to obtain the pre-trained large model.

[0017] Preferably, calculating a model loss according to the sample predicted faults and the sample troubleshooting steps comprises:

[0018] Calculating a fault similarity between the sample predicted faults and sample true faults, and determining a first loss according to the fault similarity;

[0019] Calculating a step similarity between the sample troubleshooting steps and standard troubleshooting steps, and determining a second loss according to the step similarity;

[0020] Performing weighted operation on the first loss and the second loss to obtain the model loss.

[0021] Preferably, performing image feature extraction on the image structure information to obtain sample image features comprises:

[0022] Performing fault description according to the image structure information to obtain image fault description, and performing sentence division on the image fault description to obtain image description sentences;

[0023] Performing vector conversion on the image description sentences to obtain description sentence vectors, and combining the description sentence vectors to obtain a sentence vector set;

[0024] Obtaining a sample fault type of a sample device, and performing vector conversion on the sample fault type to obtain a sample type vector;

[0025] Calculate the inner product between the sample type vector and the set of sentence vectors to obtain a vector similarity, and determine the sample image feature according to the vector similarity.

[0026] Preferably, determining the sample image feature according to the vector similarity comprises:

[0027] Normalizing the vector similarity to obtain a type attention weight, and weighting and summing the set of sentence vectors according to the type attention weight to obtain the sample image feature.

[0028] Preferably, the parameter feature extraction of the running structure information obtains a sample parameter feature, comprising:

[0029] Obtain the running parameter value in the running structure information, and construct a matrix according to the running parameter value to obtain a running parameter matrix.

[0030] Obtain the parameter type of the running parameter value, obtain the description correlation between the parameter type and the image description sentence, and generate a running adjustment matrix according to the description correlation.

[0031] Adjust the running parameter matrix according to the running adjustment matrix to obtain a running feature matrix, and convert the running feature matrix into a vector to obtain the sample parameter feature.

[0032] Preferably, obtaining the description correlation between the parameter type and the image description sentence, and generating a running adjustment matrix according to the description correlation comprises:

[0033] Obtain the associated vocabulary of the parameter type, and perform vocabulary matching between the associated vocabulary and the image description sentence.

[0034] Determine an associated word set of the parameter type according to the vocabulary matching result, and obtain an associated value between each associated vocabulary in the associated word set and the corresponding parameter type.

[0035] Numerical mapping is performed on the associated value to obtain an associated mapping value, and a matrix is constructed with the associated mapping value as a matrix element to obtain the running adjustment matrix.

[0036] Another object of the embodiment of the application is to provide a device fault repair system based on a large model, which comprises:

[0037] A device determination module is configured to obtain a device fault description sent by a user, and determine a fault device according to the device fault description.

[0038] The fault identification module is configured to acquire current operation data and a current operation image of the faulty device, and input the device fault description, the current operation data and the current operation image into a pre-trained large model for fault identification to obtain a target fault cause;

[0039] The troubleshooting determination module is configured to determine a priority troubleshooting step according to the target fault cause, and send the target fault cause and the priority troubleshooting step to the user.

[0040] The result output module is configured to receive a troubleshooting feedback result for the priority troubleshooting step, and determine a real fault cause according to the troubleshooting feedback result.

[0041] The device fault maintenance result is generated according to the device fault description, the current operation data, the current operation image and the real fault cause.

[0042] Preferably, the fault identification module is further configured to:

[0043] The sample fault description, sample operation data and sample operation image are acquired, and the sample fault description, sample operation data and sample operation image are input into the large model for structured processing to obtain fault structure information, operation structure information and image structure information.

[0044] The fault structure information is subjected to semantic recognition to obtain sample description semantics, and the operation structure information is subjected to parameter feature extraction to obtain sample parameter features.

[0045] The image structure information is subjected to image feature extraction to obtain sample image features, and the sample description semantics, sample parameter features and sample image features are subjected to feature fusion to obtain sample fusion features.

[0046] The sample fusion features are subjected to fault prediction to obtain sample predicted faults, and sample troubleshooting steps are determined according to the sample predicted faults.

[0047] The model loss is calculated according to the sample predicted faults and the sample troubleshooting steps, and the large model is subjected to parameter update according to the model loss until the large model converges to obtain the pre-trained large model.

[0048] The embodiment of the application can automatically determine the fault equipment by obtaining the equipment fault description, can automatically analyze the fault reason of the fault equipment by inputting the equipment fault description, current running data and current running image into the pre-trained large model for fault identification to obtain the target fault reason, and can automatically determine the priority troubleshooting step based on the target fault reason, thereby effectively assisting the equipment fault maintenance and improving the equipment fault maintenance efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flowchart of the equipment fault maintenance method based on a large model provided by the first embodiment of the application;

[0050] Figure 2 is a structural schematic diagram of the equipment fault maintenance system based on a large model provided by the second embodiment of the application;

[0051] Figure 3 is a structural schematic diagram of the terminal equipment provided by the third embodiment of the application. DETAILED DESCRIPTION

[0052] In order to make the objectives, technical solutions and advantages of the application clearer, the application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0053] In order to illustrate the technical solutions of the application, the following specific embodiments are used for illustration.

[0054] Embodiment One

[0055] Please refer to Figure 1 is a flowchart of the equipment fault maintenance method based on a large model provided by the first embodiment of the application, which can be applied to any equipment or system. The equipment fault maintenance method based on a large model includes the following steps:

[0056] Step S10, obtaining the equipment fault description sent by the user, and determining the fault equipment according to the equipment fault description;

[0057] The equipment fault description can be transmitted in the form of voice or text. The entity recognition is performed on the equipment fault description, and the fault equipment is determined according to the entity recognition result.

[0058] Step S20, obtaining the current running data and the current running image of the fault equipment, and inputting the equipment fault description, the current running data and the current running image into the pre-trained large model for fault identification to obtain the target fault reason;

[0059] Wherein, the device identifier of the fault device is acquired, the current running data and the device address are acquired according to the device identifier and the Internet of Things query, the target shooting device is determined according to the device address, the target shooting device is controlled to collect images of the fault device, and the current running image is obtained.

[0060] In this step, through the integration of voice recognition, image recognition and Internet of Things collection module, the multi-modal sensing capability of fault scene information is realized, and the operation and maintenance personnel (user) can input the device fault description through voice or text, and at the same time, the running parameters of the fault device in the Internet of Things, such as temperature, current, vibration, etc. can be read in real time. In terms of picture processing, it supports uploading on-site pictures (such as error prompt screen, LED status light, etc.) or videos, and the large model can perform state recognition and comparative analysis based on computer vision (Computer Vision, CV). Based on the above multi-source information input, the large model after pre-training combines the knowledge base to identify faults, automatically judges the possible fault reasons, and obtains the target fault reason. In this embodiment, the large model is used as the language reasoning core, combined with the RAG (Retrieval-Augmented Generation) framework, to provide semantic retrieval of fault keywords or descriptions, and the most relevant knowledge fragments can be quickly extracted. At the same time, the large model understands the retrieval results and generates fault reasons close to the scene.

[0061] Optionally, before inputting the device fault description, the current running data and the current running image into the large model after pre-training for fault identification, it further includes:

[0062] Acquire sample fault description, sample running data and sample running image, and input the sample fault description, the sample running data and the sample running image into the large model for structured processing to obtain fault structure information, running structure information and image structure information;

[0063] Wherein, various types of knowledge assets are acquired through the following paths, such as: historical fault cases and maintenance processes; expert oral experience or maintenance notes; device technical manual, maintenance document, technical drawing; real-time monitoring data, alarm records of Internet of Things; image, video materials, such as device operation interface screenshot, camera monitoring record, etc. Data labeling is performed on the acquired various types of knowledge assets, sample fault description, sample running data and sample running image are generated according to the data labeling results, and the large model is used to perform structured processing on the sample fault description, the sample running data and the sample running image, which effectively facilitates the subsequent sample feature extraction;

[0064] The semantic recognition is performed on the fault structure information to obtain sample description semantics, and parameter feature extraction is performed on the operation structure information to obtain sample parameter features; wherein, through the semantic recognition on the fault structure information, the semantic information corresponding to the sample fault description can be effectively extracted;

[0065] The image feature extraction is performed on the image structure information to obtain sample image features, and the sample description semantics, the sample parameter features and the sample image features are fused to obtain sample fusion features;

[0066] The sample predicted faults are determined according to the sample predicted faults, wherein the feature similarity between the sample fusion features and preset fault features is calculated, the fault probability distribution is generated based on the feature similarity, and the sample predicted faults are determined based on the fault probability distribution; in this step, the sample predicted faults and the sample fusion features are matched with the troubleshooting step database to obtain the sample troubleshooting steps, and the corresponding relationship between different sample predicted faults, sample fusion features and corresponding sample troubleshooting steps is stored in the troubleshooting step database;

[0067] The model loss is calculated according to the sample predicted faults and the sample troubleshooting steps, and the large model is parameter updated according to the model loss until the large model converges, so as to obtain the pre-trained large model.

[0068] Further, the model loss is calculated according to the sample predicted faults and the sample troubleshooting steps, including: calculating the fault similarity between the sample predicted faults and the sample real faults, determining the first loss according to the fault similarity; calculating the step similarity between the sample troubleshooting steps and the standard troubleshooting steps, determining the second loss according to the step similarity; performing weighted operation on the first loss and the second loss to obtain the model loss; wherein, in the weighted operation process, the weighting coefficients of the first loss and the second loss can be set according to requirements.

[0069] Further, the image feature extraction is performed on the image structure information to obtain sample image features, including:

[0070] The image fault description is obtained according to the image structure information, and the image description sentences are obtained by performing sentence segmentation on the image fault description; wherein, the image fault description is obtained by performing text generation on the image structure information, and in this step, the text generation of the image structure information can be performed based on a text generator;

[0071] The image description sentence is converted into a vector to obtain a description sentence vector, and the description sentence vector is combined to obtain a sentence vector set; wherein the image description sentence is input into a vector encoder to be converted into a vector to obtain a description sentence vector;

[0072] A sample fault type of the sample device is obtained, the sample fault type is converted into a sample type vector, an inner product between the sample type vector and the sentence vector set is calculated to obtain a vector similarity, and the sample image feature is determined according to the vector similarity.

[0073] Preferably, determining the sample image feature according to the vector similarity comprises: normalizing the vector similarity to obtain a type attention weight, and weighting and summing the sentence vector set according to the type attention weight to obtain the sample image feature; wherein the vector similarity is effectively converted into an attention weight by normalizing the vector similarity.

[0074] In this step, the running structure information is subjected to parameter feature extraction to obtain a sample parameter feature, comprising:

[0075] A running parameter value in the running structure information is obtained, and a running parameter matrix is constructed according to the running parameter value; wherein the running parameter value is used as a matrix element to construct a matrix to obtain a running parameter matrix;

[0076] A parameter type of the running parameter value is obtained, a description correlation between the parameter type and the image description sentence is obtained, and a running adjustment matrix is generated according to the description correlation; wherein the description correlation is used to represent the correlation degree between the parameter type and the image description sentence.

[0077] The running parameter matrix is adjusted according to the running adjustment matrix to obtain a running feature matrix, and the running feature matrix is converted into a vector to obtain the sample parameter feature; wherein the running parameter matrix is effectively weighted according to the correlation degree between the parameter type and the image description sentence by adjusting the running parameter matrix according to the running adjustment matrix to obtain the running feature matrix.

[0078] In this embodiment, the description correlation between the parameter type and the image description sentence is obtained, and the running adjustment matrix is generated according to the description correlation, comprising:

[0079] obtain the associated vocabulary of the parameter type, and perform vocabulary matching between the associated vocabulary and the image description sentence; wherein the parameter type is matched with an associated query table to obtain the associated vocabulary, the associated query table storing a corresponding relationship between different parameter types and corresponding associated vocabulary, the image description sentence is segmented to obtain description segmentation, the description segmentation and the associated vocabulary are vector converted to obtain segmentation vectors and associated vectors, an associated vector similarity between the associated vectors and the segmentation vectors is calculated, and if any associated vector similarity is greater than a similarity threshold, it is determined that the vocabulary matching between the associated vocabulary and the image description sentence is successful;

[0080] determine an associated word set of the parameter type according to the vocabulary matching result, and obtain an associated value between each associated vocabulary in the associated word set and the corresponding parameter type; wherein the associated word set is generated according to the associated vocabulary matched successfully, the associated vocabulary and the corresponding parameter type are matched with the associated query table to obtain the associated value, and the associated query table stores a corresponding relationship between different associated vocabulary, parameter types and corresponding associated values;

[0081] perform numerical mapping on the associated value to obtain an associated mapping value, and perform matrix construction with the associated mapping value as a matrix element to obtain the running adjustment matrix.

[0082] Step S30, determine a priority troubleshooting step according to the target fault cause, and send the target fault cause and the priority troubleshooting step to the user;

[0083] The device fault description, the current running data, the fusion features corresponding to the current running image, and the target predicted fault and troubleshooting step database are matched to obtain the priority troubleshooting step, and the target fault cause and the priority troubleshooting step are sent to the user, thereby effectively facilitating the user to analyze the fault cause of the faulty device. In this step, based on the induction ability of the large model, the high-frequency fault scene and the corresponding standardized processing flow are refined, the clear path of fault processing is marked, and the flowchart is presented in the form of a graph. This process not only assists personnel decision-making, but also supports the subsequent reasoning engine as an execution reference.

[0084] Step S40, receive a troubleshooting feedback result for the priority troubleshooting step, and determine a real fault cause according to the troubleshooting feedback result;

[0085] The real fault cause of the current faulty device is determined according to the fault troubleshooting log information of the user on the faulty device in the troubleshooting feedback result.

[0086] Step S50, generate a device fault repair result according to the device fault description, the current running data, the current running image and the real fault cause;

[0087] The device fault maintenance result is obtained by corresponding storage of the device fault description, the current operation data, the current operation image and the real fault reason, and the device fault maintenance result is effectively convenient for subsequent parameter fine-tuning operation of the large model, so as to further improve the model effect of the large model.

[0088] In this step, all troubleshooting operations and results (success / failure) are fed back to the large model, and the model parameters of the large model are automatically updated. New fault types confirmed by artificial confirmation can also be included in the knowledge base of the large model through human-computer cooperation, realizing closed-loop knowledge evolution.

[0089] This embodiment takes “improving maintenance efficiency, reducing knowledge threshold, and enhancing system intelligence” as the goal, and proposes an intelligent equipment maintenance platform based on a multimodal large model (Multimodal LLM) and RAG technology. By integrating language, image, sound, structured data and other multi-source information, an interactive, self-learning and continuously optimized equipment fault troubleshooting system is created.

[0090] Through the large model, the experience knowledge is structured and standardized, which can assist all personnel in making unified decisions. Through the learning of historical faults and processing paths, a knowledge base conforming to equipment fault troubleshooting is constructed, and continuous self-learning is supported. Based on the RAG technology, multiple heterogeneous data sources can be integrated, the data sources have multiple channels, the problem of data islands and scattered data that cannot be shared is solved.

[0091] In this embodiment, the fault equipment can be automatically determined by obtaining the equipment fault description. The equipment fault description, the current operation data and the current operation image are input into the pre-trained large model for fault identification, the fault cause of the fault equipment can be automatically analyzed, the target fault cause is obtained, the priority troubleshooting step can be automatically determined based on the target fault cause, the target fault cause and the priority troubleshooting step are sent to the user, which effectively assists the device fault maintenance and improves the device fault maintenance efficiency.

[0092] Embodiment Two

[0093] Please refer to Figure 2 is a structural schematic diagram of a device fault maintenance system 100 based on a large model provided by the second embodiment of the present application, which comprises:

[0094] The device determination module 10 is configured to obtain the equipment fault description sent by the user, and determine the fault equipment according to the equipment fault description.

[0095] The fault identification module 11 is configured to acquire current operation data and a current operation image of the faulty device, and input the device fault description, the current operation data and the current operation image into a pre-trained large model for fault identification to obtain a target fault cause.

[0096] Optionally, the fault identification module 11 is further configured to acquire a sample fault description, sample operation data and a sample operation image, and input the sample fault description, the sample operation data and the sample operation image into the large model for structured processing to obtain fault structure information, operation structure information and image structure information.

[0097] The fault structure information is subjected to semantic recognition to obtain sample description semantics, and the operation structure information is subjected to parameter feature extraction to obtain sample parameter features.

[0098] The image structure information is subjected to image feature extraction to obtain sample image features, and the sample description semantics, the sample parameter features and the sample image features are subjected to feature fusion to obtain sample fusion features.

[0099] The sample fusion features are subjected to fault prediction to obtain sample predicted faults, and sample troubleshooting steps are determined according to the sample predicted faults.

[0100] A model loss is calculated according to the sample predicted faults and the sample troubleshooting steps, and the large model is subjected to parameter updating according to the model loss until the large model converges, thereby obtaining the pre-trained large model.

[0101] Further, the fault identification module 11 is further configured to calculate a fault similarity between the sample predicted faults and sample true faults, and determine a first loss according to the fault similarity.

[0102] A step similarity between the sample troubleshooting steps and standard troubleshooting steps is calculated, and a second loss is determined according to the step similarity.

[0103] The first loss and the second loss are subjected to weighted operation to obtain the model loss.

[0104] Still further, the fault identification module 11 is further configured to perform fault description according to the image structure information to obtain an image fault description, and divide the image fault description into image description sentences.

[0105] The image description sentences are subjected to vector conversion to obtain description sentence vectors, and the description sentence vectors are combined to obtain a sentence vector set.

[0106] obtain a sample fault type of the sample device, and perform vector conversion on the sample fault type to obtain a sample type vector;

[0107] calculate an inner product between the sample type vector and the set of sentence vectors to obtain a vector similarity, and determine the sample image feature according to the vector similarity.

[0108] Preferably, the fault identification module 11 is further configured to: perform normalization processing on the vector similarity to obtain a type attention weight, and perform weighted sum on the set of sentence vectors according to the type attention weight to obtain the sample image feature.

[0109] In this embodiment, the fault identification module 11 is further configured to: obtain a running parameter value in the running structure information, and perform matrix construction according to the running parameter value to obtain a running parameter matrix;

[0110] obtain a parameter type of the running parameter value, obtain a description correlation between the parameter type and the image description sentence, and generate a running adjustment matrix according to the description correlation;

[0111] adjust the running parameter matrix according to the running adjustment matrix to obtain a running feature matrix, and perform vector conversion on the running feature matrix to obtain the sample parameter feature.

[0112] Preferably, the fault identification module 11 is further configured to: obtain an associated vocabulary of the parameter type, and perform vocabulary matching between the associated vocabulary and the image description sentence;

[0113] determine an associated word set of the parameter type according to the vocabulary matching result, and obtain an associated value between each associated vocabulary in the associated word set and the corresponding parameter type, respectively;

[0114] perform numerical mapping on the associated value to obtain an associated mapping value, and perform matrix construction with the associated mapping value as a matrix element to obtain the running adjustment matrix.

[0115] The troubleshooting determination module 12 is configured to determine a priority troubleshooting step according to the target fault cause, and send the target fault cause and the priority troubleshooting step to the user.

[0116] The result output module 13 is configured to receive a troubleshooting feedback result for the priority troubleshooting step, and determine a real fault cause according to the troubleshooting feedback result.

[0117] generate a device fault maintenance result according to the device fault description, the current running data, the current running image, and the real fault cause.

[0118] Specifically, the specific implementation steps of the large model-based equipment fault maintenance system 100 include:

[0119] Step 1: Knowledge data integration

[0120] Organize and structure historical fault data and equipment information;

[0121] Build a multi-modal data access platform to complete text, image, voice, and data collection and format conversion;

[0122] Establish a unified knowledge storage structure to support vector retrieval.

[0123] Step 2: Large model access and capability training

[0124] Select appropriate large model services;

[0125] Introduce RAG architecture to build semantic retrieval and generation mechanism;

[0126] Preliminary annotation and fine-tuning training on enterprise fault corpus to improve industry adaptation.

[0127] Step 3: Multi-modal recognition module deployment

[0128] Access the speech recognition module to realize natural language description analysis;

[0129] Integrate the image recognition engine to train the typical equipment picture recognition model;

[0130] Connect the Internet of Things platform to realize real-time parameter acquisition and historical data comparison.

[0131] Step 4: Fault troubleshooting engine online trial operation

[0132] Establish SOP knowledge base (Standard Operating Procedure Knowledge Base) and troubleshooting suggestion generation module;

[0133] Support "human + machine" collaborative process handling mechanism to ensure human controllability.

[0134] In this embodiment, by obtaining the equipment fault description, the fault equipment can be automatically determined, by inputting the equipment fault description, the current running data and the current running image into the pre-trained large model for fault identification, the fault reason of the fault equipment can be automatically analyzed, the target fault reason is obtained, based on the target fault reason, the priority troubleshooting step can be automatically determined, by sending the target fault reason and the priority troubleshooting step to the user, the device fault maintenance is effectively assisted, and the device fault maintenance efficiency is improved.

[0135] Embodiment three

[0136] Figure 3 is a structural block diagram of a terminal device 2 provided by a third embodiment of the present application. As shown in the figure, the terminal device 2 of this embodiment comprises a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program of the device fault troubleshooting method based on a large model. The processor 20 implements the steps in each of the embodiments of the above-mentioned various device fault troubleshooting methods based on a large model when executing the computer program 22. Figure 3

[0137] For example, the computer program 22 can be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete the present application. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 22 in the terminal device 2. The terminal device can include, but is not limited to, the processor 20 and the memory 21.

[0138] The processor 20 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0139] The memory 21 can be an internal storage unit of the terminal device 2, such as a hard disk or a memory of the terminal device 2. The memory 21 can also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Further, the memory 21 can include both the internal storage unit and the external storage device of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0140] ​In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0141] If the integrated module is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. The computer readable storage medium can be non-volatile or volatile. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable storage medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content included in the computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable storage medium does not include electrical carrier signals and telecommunication signals.

[0142] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for equipment fault diagnosis and repair based on a large model, characterized in that, The method includes: Obtain the device fault description sent by the user, and determine the faulty device based on the device fault description; The current operating data and current operating image of the faulty device are obtained, and the device fault description, the current operating data and the current operating image are input into a pre-trained large model for fault identification to obtain the target fault cause; Based on the target fault cause, a priority troubleshooting step is determined, and the target fault cause and the priority troubleshooting step are sent to the user; Receive the troubleshooting feedback results for the priority troubleshooting steps, and determine the actual cause of the fault based on the troubleshooting feedback results; The equipment fault repair results are generated based on the equipment fault description, the current operating data, the current operating image, and the actual fault cause.

2. The equipment fault diagnosis and repair method based on a large model as described in claim 1, characterized in that, Before inputting the device fault description, the current operating data, and the current operating image into the pre-trained large model for fault identification, the process also includes: Obtain sample fault descriptions, sample operation data, and sample operation images, and input the sample fault descriptions, sample operation data, and sample operation images into the large model for structured processing to obtain fault structure information, operation structure information, and image structure information; Semantic recognition is performed on the fault structure information to obtain sample description semantics, and parameter feature extraction is performed on the operation structure information to obtain sample parameter features; Image features are extracted from the image structure information to obtain sample image features, and the sample description semantics, sample parameter features and sample image features are fused to obtain sample fused features; Fault prediction is performed based on the sample fusion features to obtain sample predicted faults, and sample investigation steps are determined based on the sample predicted faults. The model loss is calculated based on the sample prediction failure and the sample screening steps, and the parameters of the large model are updated based on the model loss until the large model converges, thus obtaining the pre-trained large model.

3. The equipment fault diagnosis and repair method based on a large model as described in claim 2, characterized in that, The model loss is calculated based on the predicted faults in the samples and the sample screening steps, including: Calculate the fault similarity between the predicted faults of the samples and the actual faults of the samples, and determine the first loss based on the fault similarity; Calculate the step similarity between the sample screening step and the standard screening step, and determine the second loss based on the step similarity; The first loss and the second loss are weighted and calculated to obtain the model loss.

4. The equipment fault diagnosis and repair method based on a large model as described in claim 2, characterized in that, Image feature extraction is performed on the image structure information to obtain sample image features, including: Based on the image structure information, a fault description is performed to obtain an image fault description, and the image fault description is segmented into sentences to obtain an image description sentence; The image description sentences are vectorized to obtain description sentence vectors, and the description sentence vectors are combined to obtain a sentence vector set; Obtain the sample fault type of the sample device, and perform vector transformation on the sample fault type to obtain the sample type vector; Calculate the inner product between the sample type vector and the sentence vector set to obtain the vector similarity, and determine the sample image features based on the vector similarity.

5. The equipment fault diagnosis and repair method based on a large model as described in claim 4, characterized in that, Determining the features of the sample image based on the vector similarity includes: The vector similarity is normalized to obtain the type attention weight, and the sentence vector set is weighted and summed according to the type attention weight to obtain the sample image features.

6. The equipment fault diagnosis and repair method based on a large model as described in claim 4, characterized in that, The operational structure information is subjected to parameter feature extraction to obtain sample parameter features, including: Obtain the running parameter values ​​from the running structure information, and construct a matrix based on the running parameter values ​​to obtain the running parameter matrix; Obtain the parameter type of the running parameter value, obtain the descriptive correlation between the parameter type and the image description sentence, and generate a running adjustment matrix based on the descriptive correlation; The operating parameter matrix is ​​adjusted according to the operating adjustment matrix to obtain the operating feature matrix, and the operating feature matrix is ​​then vector-transformed to obtain the sample parameter features.

7. The equipment fault diagnosis and repair method based on a large model as described in claim 6, characterized in that, Obtaining the descriptive correlation between the parameter type and the image description, and generating a runtime adjustment matrix based on the descriptive correlation, includes: Obtain the associated words of the parameter type, and match the associated words with the image description sentence; Based on the word matching results, determine the set of related words for the parameter type, and obtain the association value between each related word in the set of related words and the corresponding parameter type; The associated values ​​are numerically mapped to obtain associated mapping values, and the associated mapping values ​​are used as matrix elements to construct the operation adjustment matrix.

8. A large-scale model-based equipment fault diagnosis and repair system, characterized in that, The system includes: The device identification module is used to obtain the device fault description sent by the user and identify the faulty device based on the device fault description; The fault identification module is used to acquire the current operating data and current operating image of the faulty device, and input the device fault description, the current operating data and the current operating image into the pre-trained large model for fault identification to obtain the target fault cause; The troubleshooting and determination module is used to determine the priority troubleshooting steps based on the target fault cause, and send the target fault cause and the priority troubleshooting steps to the user; The result output module is used to receive the investigation feedback results for the priority investigation steps, and determine the actual cause of the fault based on the investigation feedback results; The equipment fault repair results are generated based on the equipment fault description, the current operating data, the current operating image, and the actual fault cause.

9. The equipment fault diagnosis and repair system based on a large model as described in claim 8, characterized in that, The fault identification module is also used for: Obtain sample fault descriptions, sample operation data, and sample operation images, and input the sample fault descriptions, sample operation data, and sample operation images into the large model for structured processing to obtain fault structure information, operation structure information, and image structure information; Semantic recognition is performed on the fault structure information to obtain sample description semantics, and parameter feature extraction is performed on the operation structure information to obtain sample parameter features; Image features are extracted from the image structure information to obtain sample image features, and the sample description semantics, sample parameter features and sample image features are fused to obtain sample fused features; Fault prediction is performed based on the sample fusion features to obtain sample predicted faults, and sample investigation steps are determined based on the sample predicted faults. The model loss is calculated based on the sample prediction failure and the sample screening steps, and the parameters of the large model are updated based on the model loss until the large model converges, thus obtaining the pre-trained large model.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images