Fault diagnosis and solution recommendation method, device, system and storage medium

Abstract

The embodiment of the application provides a fault diagnosis and solution recommendation method, device, system and computer readable storage medium. The method comprises: obtaining original data containing a fault problem of a target device; analyzing the original data containing the fault problem to obtain problem description information; analyzing the problem description information to obtain a diagnosis report; obtaining a video and / or document solution of the fault based on a cloud knowledge graph according to the diagnosis report, and recommending the solution to a user; the knowledge graph comprises: a node representing the fault, a node representing the video solution and / or the document solution, and a plurality of edges representing the relationship between the nodes. The technical solution in the embodiment of the application can realize intelligent fault diagnosis and solution recommendation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, in particular to a fault diagnosis and solution recommendation method, device, system and computer readable storage medium. BACKGROUND

[0003] Digital technology refers to a technology that uses computers and networks to realize digitization. Digital technology has been applied to various industries and fields, for example, to traditional manufacturing plants. A digital factory refers to using computer hardware and software technology to provide digital and information services for traditional manufacturing plants. A digital factory integrates various systems and databases of factories, products and controls, etc., and improves the flexibility and efficiency of the manufacturing process of the factory through visualization, simulation, big data, etc.

[0004] In a modern digital factory, workers have difficulty in timely solving unexpected problems, so an intelligent diagnosis and solution recommendation system for the factory situation is needed. SUMMARY

[0005] Therefore, in the embodiments of the present application, on the one hand, a fault diagnosis and solution recommendation method is provided, and on the other hand, a fault diagnosis and solution recommendation device, system and computer readable storage medium are provided, which can realize fault diagnosis and solution recommendation in a digital factory and improve the flexibility and efficiency of fault diagnosis and solution recommendation.

[0006] To solve the above technical problems, the technical scheme of the present application is as follows:

[0007] A fault diagnosis and solution recommendation method comprises: acquiring original data containing a fault of a target device; analyzing the original data containing the fault to obtain a problem description information; analyzing the problem description information to obtain a diagnosis report; acquiring a video and / or document solution for the fault based on a cloud knowledge graph according to the diagnosis report, and recommending the solution to a user; wherein the cloud knowledge graph comprises nodes representing faults, video solutions and / or document solutions, respectively, and a plurality of edges representing relationships between the nodes.

[0008] In an embodiment, the original data containing the fault comprises: a data stream containing error signals, device identification and time information; the analysis of the original data containing the fault obtains a problem description information, comprising: obtaining a corresponding task document from a cloud task management database according to the device identification and time information of the target device; the cloud task management database stores task documents of standard processes and steps of tasks corresponding to each device; dividing the data stream into corresponding data segments according to the standard processes and steps in the task document; determining the target step where the fault exists by locating the error signals in the data segment; comparing and analyzing the task document and the part of the data stream corresponding to the target step to generate the problem description information.

[0009] In an embodiment, the obtaining of the original data containing the fault of the target device comprises: collecting data streams of each target device with device identification and time information based on the OPC UA protocol according to a set frequency; in response to receiving a fault report of a user, extracting the corresponding data stream of the target device from the collected data streams according to the device identification and time information provided by the fault report.

[0010] In an embodiment, the analysis of the problem description information obtains a diagnosis report, comprising: inputting the problem description information into a pre-trained fault diagnosis model based on a convolutional neural network to obtain a diagnosis report output by the fault diagnosis model; the fault diagnosis model is trained by using a large number of historical problem description information as input samples and a historical diagnosis report corresponding to each historical problem description information as output samples.

[0011] In an embodiment, the original data containing the fault comprises: a fault multimedia content; the fault multimedia content comprises: a video containing a fault process or a photo containing a fault area; the analysis of the original data containing the fault obtains a problem description information, comprising: inputting the fault multimedia content into a pre-trained fault analysis model based on a neural network to obtain a problem description information output by the fault analysis model; the fault analysis model is trained by using a large number of historical fault multimedia contents stored in a cloud image database as input samples and a historical problem description information corresponding to each historical fault multimedia content as output samples.

[0012] In an embodiment, the fault analysis model comprises a multimedia content classification model and a plurality of different types of fault analysis sub-models; wherein the multimedia content classification model is configured to classify the fault multimedia content, and provide the fault multimedia content to a corresponding type of fault analysis sub-model according to the classification result; each fault analysis sub-model is configured to output corresponding problem description information according to the input fault multimedia content.

[0013] In an embodiment: the problem description information is fed back to a user for checking, and the user-confirmed problem description information is received; the user-confirmed problem description information is taken as the finally determined problem description information; and the finally determined problem description information and the fault multimedia content are stored as a new record item in the cloud image database, so that the fault analysis model is optimized based on the updated cloud multimedia management database.

[0014] A fault diagnosis and solution recommendation device, comprising: a data acquisition module configured to acquire original data containing faults of a target device; a fault analysis module configured to analyze the original data containing faults to obtain problem description information; a fault diagnosis module configured to analyze the problem description information to obtain a diagnosis report; and a solution recommendation module configured to acquire a video and / or document solution for the fault based on a cloud knowledge graph according to the diagnosis report, and recommend the solution to a user; wherein the cloud knowledge graph comprises nodes respectively representing faults, video solutions and / or document solutions, and a plurality of edges representing relationships between the nodes.

[0015] In an embodiment, the original data containing faults comprises: data streams containing device identifiers and time information, and / or; fault multimedia content; the fault multimedia content comprises: a video containing a fault process or a photo containing a fault area.

[0016] In an embodiment, the fault analysis module comprises a data stream analysis module and / or a multimedia content analysis module; the data stream analysis module is configured to obtain a corresponding task document from a cloud task management database according to the device identifier and the time information of the target device; the task management database stores task documents of standard processes and steps of tasks corresponding to respective devices; the data stream is divided into corresponding data segments according to the standard processes and steps in the task document; the target step with a fault is determined by locating an error signal in the data segment; the task document and the part of the data stream corresponding to the target step are compared and analyzed to generate the problem description information; the multimedia content analysis module is configured to input the fault multimedia content into a pre-trained neural network-based fault analysis model to obtain the problem description information output by the fault analysis model; the fault analysis model is trained using a large amount of historical fault multimedia content stored in a cloud multimedia management database as input samples and using historical problem description information corresponding to each historical fault multimedia content as output samples.

[0017] A fault diagnosis and solution recommendation device, comprising at least one memory and at least one processor, wherein: the at least one memory is configured to store a computer program; the at least one processor is configured to invoke the computer program stored in the at least one memory to execute the fault diagnosis and solution recommendation method as described in any one of the above embodiments.

[0018] A fault diagnosis and solution recommendation system, comprising the fault diagnosis and solution recommendation device as described in any one of the above embodiments; a cloud task management server configured to store task documents of standard processes and steps of tasks corresponding to respective devices; a cloud image server configured to store historical fault multimedia content and corresponding historical problem description information; a cloud teaching content knowledge graph comprising nodes respectively representing faults, video solutions and / or document solutions and a plurality of edges representing relationships between the nodes; and a diagnosis management database 14 configured to store a large amount of historical problem description information and corresponding historical diagnosis reports.

[0019] A computer-readable storage medium having a computer program stored thereon; the computer program can be executed by a processor and implement the fault diagnosis and solution recommendation method as described in any one of the above embodiments.

[0020] From the above technical solutions, it can be seen that in the present application, all fault analysis, fault diagnosis and solution recommendation are completed by a third party, and the user only needs to perform some simple operations without analyzing the fault reason, thereby realizing intelligent fault diagnosis and solution recommendation and improving the flexibility and efficiency of fault diagnosis and recommendation.

[0021] BRIEF DESCRIPTION OF DRAWINGS

[0022] To better understand this application, embodiments of this application will be described in detail below with reference to the accompanying drawings, so that those skilled in the art can more clearly understand the above and other features and advantages of this application, in which:

[0023] Figure 1 This is an exemplary flowchart of a fault diagnosis and solution recommendation method in an embodiment of this application;

[0024] Figure 2 This is a schematic diagram illustrating an example application scenario of this application;

[0025] Figure 3 Based on the embodiments of this application Figure 2 An exemplary flowchart of the fault diagnosis and solution recommendation method for the application scenario shown;

[0026] Figure 4 As an example in this application Figure 2 A schematic diagram illustrating the working principle of the data flow analysis package in the application scenario shown;

[0027] Figure 5 for Figure 4 The diagram shows a flowchart of the data flow analysis method executed by the data flow analysis package.

[0028] Figure 6 As an example in this application Figure 2 A schematic diagram illustrating the working principle of the multimedia content analysis package in the application scenario shown.

[0029] Figure 7 In the embodiments of this application, by Figure 6 The flowchart shown is a process for performing multimedia content analysis using the multimedia content analysis package.

[0030] Figure 8 This is an exemplary structural diagram of the recommended device for fault diagnosis and solution in the embodiments of this application;

[0031] Figure 9 This is an exemplary structural diagram of another fault diagnosis and solution recommended device in the embodiments of this application.

[0032] The reference numerals in the attached figures are as follows:

[0033]

[0034]

[0035] WAYS OF CARRYING OUT THE APPLICATION

[0036] In the embodiments of the present application, it is considered that in the digital factory, some devices such as numerical control machine tools have their own diagnosis programs at present, but many of them are based on the established mode for judgment, and cannot diagnose according to the characteristics of specific tasks. In addition, the diagnosis programs of some control systems also give some solutions, but these solutions are usually based on pre-stored documents. In addition, most of the diagnosis programs are based on the data collected by the device itself to judge, and it is difficult to diagnose the faults that need to be detected through sensors.

[0037] To this end, an intelligent fault diagnosis and solution recommendation scheme is proposed in the embodiments of the present application.

[0038] In order to make the purpose, technical scheme and advantages of the present application more clear, the technical scheme of the present application will be described in detail below with reference to the drawings and examples.

[0039] Figure 1 An example flowchart of a fault diagnosis and solution recommendation method in the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the method can include the following steps: Figure 1

[0040] Step 101: Obtain original data of a target device containing fault problems.

[0041] The original data can be obtained by a robot, for example, a data collection module of a knowledge transmission robot.

[0042] In a specific implementation, the original data can include a data stream reported by a device based on OPC Unified Architecture (OPC-UA), a device identifier (for example, a machine number), and a time information (for example, a time stamp) data stream. The data stream here can include data collected by the device itself and data collected by sensors.

[0043] In an example, the above data stream, device identifier, and time information can be obtained at a set frequency. In response to receiving a fault report from a user, the corresponding data stream of the target device can be extracted from the obtained data stream according to the device identifier and time information provided by the fault report. For example, when a fault occurs, the user can call a cloud robot, and report the device identifier of the target device that has occurred a fault and the time information when the fault occurs to the cloud robot.

[0044] In addition, the original data can also include fault multimedia content. The fault multimedia content can include a video containing a fault process or a photo containing a fault area. For example, when a fault occurs, the user can take a video or a photo, and send the video or the photo to the robot.

[0045] Step 102: Analyze the original data containing the fault problems, and obtain problem description information.

[0046] ​In a specific implementation, the original data containing the fault problem can be analyzed by a fault analysis package of the robot.

[0047] When the original data includes a data stream with device identification and time information, the fault analysis package can include a data stream analysis package that can find a corresponding task document from a task management database according to the device identification and time information of the target device. The task management database stores task documents including standard processes and steps of various tasks of various devices. The data stream analysis package divides the data stream collected from the target device into corresponding data segments according to the standard processes and steps in the task document, and determines the target step with the fault by locating the error signal in the data segment. The data stream analysis package compares the corresponding part in the task document with the corresponding part in the data stream, and analyzes the fault cause on this basis to generate problem description information.

[0048] In one example, the task management database can be a cloud task management database. Through the cloud task management database, the task documents stored in the cloud task management database can be shared by different factories.

[0049] When the original data contains multimedia content, the fault analysis package can include a multimedia content analysis package that can input the fault multimedia content into a pre-trained fault analysis model based on a convolutional neural network to obtain problem description information output by the fault analysis model. The fault analysis model can be trained by taking a large number of historical fault multimedia content as input samples and taking historical problem description information corresponding to the fault multimedia content as output samples. The historical fault multimedia content and the historical problem description information corresponding to the fault multimedia content can be stored in an image database.

[0050] In one example, the fault analysis model can include a multimedia content classification model and a plurality of fault analysis sub-models. The multimedia content classification model is used to classify the multimedia content, and the multimedia content is input into the corresponding fault analysis sub-model according to the classification result. Each fault analysis sub-model is used to output corresponding problem description information according to the input multimedia content.

[0051] In addition, the multimedia content analysis package feeds back the problem description information to the user for viewing and receives the problem description information confirmed by the user. When the problem description information fed back by the multimedia content analysis package meets the actual problem, the user directly confirms, and at this time, the problem description information confirmed by the user is consistent with the problem description information fed back by the multimedia content analysis package; when the problem description information fed back by the multimedia content analysis package does not match the actual problem, the user will manually write new problem description information or modify the feedback problem description information, and the problem description information confirmed by the user is the new problem description information written or modified by the user.

[0052] Then, the user-confirmed problem description information is taken as the final determined problem description information, i.e., correct problem description information. The correct fault description information and the fault multimedia content can be taken as a new historical sample to optimize the fault analysis model. When there is an image database, the correct problem description information and the fault multimedia content can be stored in the image database as a new record item, and then the fault analysis model is optimized according to the updated image database.

[0053] The image database can be a cloud image database. Since the fault multimedia content and the problem description information in the cloud image database are huge, the fault analysis model can be fully trained and timely updated, and therefore the accuracy of the fault analysis model will be very high.

[0054] In the embodiments of the present application, the problem description information can include a fault type and a corresponding fault description.

[0055] In step 103, the problem description information is analyzed, and a diagnosis report is obtained.

[0056] In one example, the problem description information can be analyzed by a diagnosis package of the robot. The diagnosis package can input the problem description information into a pre-trained fault diagnosis model based on a convolutional neural network, and obtain a diagnosis report output by the fault diagnosis model. The fault diagnosis model can be trained by taking a large amount of historical problem description information as input samples and taking a historical diagnosis report corresponding to each piece of historical problem description information as an output sample. In one example, the historical problem description information and the corresponding historical diagnosis report can be stored in a diagnosis management database.

[0057] The diagnosis management database can be a cloud diagnosis management database. With the cloud diagnosis management database, since the diagnosis and diagnosis report in the cloud diagnosis management database are huge, the fault diagnosis model can be fully trained and timely updated, and therefore the accuracy of the fault diagnosis model will be very high.

[0058] In step 104, according to the diagnosis report, a video solution and / or a document solution of the fault are obtained based on a knowledge graph, and the solutions are recommended to the user. The knowledge graph can include nodes representing the fault, nodes representing the video solution and / or the document solution, and a plurality of edges representing the relationship between the nodes.

[0059] In one example, the video solution and / or the document solution of the fault can be obtained by a recommendation engine of the robot.

[0060] The knowledge graph can be a teaching resource knowledge graph, and the knowledge graph can be a cloud knowledge graph. Through the cloud knowledge graph, teaching resources can be shared by different factories, and since the teaching resources are rich, the user can obtain timely and effective guidance.

[0061] The above scheme will be illustrated with an example below.

[0062] Figure 2 This is a schematic diagram illustrating an example application scenario of this application. For example... Figure 2 As shown, this program scenario includes a database layer 1 and an analysis layer 2.

[0063] Database layer 1 includes a task management database 11, an image database 12, a teaching resource knowledge graph 13, and a diagnostic management database 14. Databases 11, 12, 13, and 14 can be cloud databases or local databases.

[0064] The task management database 11 stores task documents, including standard procedures and steps for each task on each device. The image database 12 stores a large amount of historical fault multimedia content and corresponding historical problem descriptions. The teaching resource knowledge graph 13 includes nodes representing faults, nodes representing video solutions and / or document solutions, and multiple edges representing the relationships between nodes. The diagnostic management database 14 stores a large amount of historical problem descriptions and corresponding historical diagnostic reports.

[0065] The analysis layer 2 includes a data flow analysis package 21, a multimedia content analysis package 22, a diagnostic package 23, and a recommendation engine 24. The data flow analysis package 21 and the multimedia content analysis package 22 can be included in the robot's analysis module, the diagnostic package 23 can be the robot's diagnostic module, and the recommendation engine 24 can be the robot's recommendation module.

[0066] Data streams from machines such as motor 31, PLC 32, and CNC 33, containing device identification and time information, can be transmitted to the robot's data collection module and provided to the data stream analysis package 21 based on the OPC-UA protocol. Photos or videos taken by user 34 can be sent to the data collection module and then provided to the multimedia content analysis package 22.

[0067] The diagnostic report output by diagnostic package 23 and the solution recommended by recommendation engine 24 can be displayed on screen 4, such as the robot's screen.

[0068] Of course, in another example, the robot could be replaced by other devices to perform diagnostics and recommend solutions.

[0069] Figure 3 Based on the embodiments of this application Figure 2 An exemplary flowchart illustrating the fault diagnosis and solution recommendation method for the application scenario shown. Figure 3 As shown, the method may include the following procedures.

[0070] Step 301, the user calls the robot, i.e., the robot receives the fault report.

[0071] Step 302, the data stream of the machine is collected and the corresponding task document is obtained from the task management database.

[0072] Step 303, the data stream and the task document are provided to the data stream analysis package, and the data stream analysis package analyzes the data stream based on the comparison between the data stream and the task document.

[0073] Step 304, it is determined whether the correct problem description information is obtained. When the correct problem description information is obtained, step 310 is performed; otherwise, step 305 is performed.

[0074] Step 305, instructing the user to take a photo of the problem existing fault.

[0075] Step 306, receiving the photo uploaded by the user.

[0076] Step 307, providing the photo to the multimedia content analysis package, and the multimedia content analysis package analyzes the photo.

[0077] Step 308, feeding back the obtained problem description information to the user, and determining by the user whether the correct problem description information is obtained. When the correct problem description information is obtained, step 310 is performed; otherwise, step 309 is performed.

[0078] Step 309, receiving the problem description information modified or newly written by the user, and regarding the modified or newly written problem description information as the correct problem description information.

[0079] Step 310, providing the problem description information to the diagnosis package generated based on the convolutional neural network.

[0080] Step 311, the diagnosis package analyzes the problem description information and generates a diagnosis report.

[0081] Step 312, providing the diagnosis report to the recommendation engine, and the recommendation engine recommends a corresponding solution based on the knowledge graph.

[0082] Step 313, updating the image database based on the feedback from the user on the problem description information, and updating the fault analysis model based on the updated image database.

[0083] Figure 4 For an example in this application Figure 2 the working principle of the data stream analysis package 21 in the application scenario shown. As Figure 4 shown, its working principle can include the following processes.

[0084] Block 401, obtain a task list from the task management database 11.

[0085] Block 402, obtain a machine number and a timestamp based on OPC UA.

[0086] Block 403, obtain a corresponding task file according to the task list, the machine number and the timestamp.

[0087] Block 404, divide the data stream obtained based on OPC UA into multiple data segments according to the process and the step in each process in the task file, determine the target step of the fault by locating the error signal in the data segment, then compare the corresponding part in the task document with the corresponding part in the data stream, and analyze the fault cause on this basis to generate problem description information.

[0088] Figure 5 For Figure 4 The flowchart of the data stream analysis method executed by the data stream analysis package shown in FIG. 8. As shown in FIG. 8, the method can include the following processes. Figure 5

[0089] Step 501, in response to the fault report received in step 301, receive first data from the machine.

[0090] Step 502, record the data stream in the first data.

[0091] Step 503, read the machine number and the timestamp from the first data.

[0092] Step 504, in response to the fault report received in step 301, read second data from the task management database.

[0093] Step 505, read the task list from the second data.

[0094] Step 506, find the corresponding task according to the machine number and the timestamp.

[0095] Step 507, read the task document corresponding to the task.

[0096] Step 508, divide the data stream into multiple data segments based on the process and the step in each process in the task document.

[0097] Step 509, determine and extract the part or target step with the error signal.

[0098] Step 510, extract the corresponding part in the task document.

[0099] Step 511, compare the extracted part with the extracted corresponding part, and analyze based on the comparison.

[0100] ​Step 512: Generate problem description information.

[0101] Figure 6 As an example in this application Figure 2 The diagram illustrates the working principle of the multimedia content analysis package 22 in the application scenario shown. Figure 6 As shown, its working principle may include the following processes.

[0102] Box 601: Users take photos using an application on their mobile phones and upload them.

[0103] In box 602, the photo is input into a fault analysis model trained on a large amount of historical fault multimedia content and problem description information stored in image database 12. The fault analysis model classifies the photo and outputs corresponding problem description information. The problem description information is fed back to the user, who confirms that the description is correct. Image database 12 can be updated based on user feedback, and then the fault analysis model is updated based on the updated image database 12.

[0104] Figure 7 In the embodiments of this application, by Figure 6 The flowchart shown illustrates the multimedia content analysis method performed by the multimedia content analysis package. Figure 7 As shown, the method may include the following procedures.

[0105] Step 701: Analyze the historical fault multimedia content and problem description information pairs stored in the image database 12.

[0106] Step 702: Establish a fault analysis model based on a convolutional neural network and train it using historical fault multimedia content and problem description information stored in the image database 12.

[0107] Step 703: Take and upload photos.

[0108] Step 704: In response to receiving the photo, input the photo into the fault analysis model.

[0109] Step 705: The fault analysis model classifies the photos and generates corresponding problem description information.

[0110] Step 706: The problem description information is displayed on the robot's screen, and the robot awaits user confirmation.

[0111] Step 707: Determine whether the correct problem description information has been obtained. If the correct problem description information has been obtained, proceed to step 709; otherwise, proceed to step 708.

[0112] Step 708: Receive the problem description information modified or newly written by the user, and regard the modified or newly written problem description information as the correct problem description information.

[0113] Step 709: Generate a new record item consisting of a correct problem description and fault multimedia content.

[0114] Step 710: Provide the diagnostic package with the correct problem description information.

[0115] Step 711: Add the new record to the image database, and then update the fault analysis model based on the updated image database.

[0116] Figure 8 This is an exemplary structural diagram of a recommended device for fault diagnosis and solutions in the embodiments of this application. This device can be used to perform… Figures 1 to 7 The method is illustrated. For details not disclosed in the device embodiments of this application, please refer to the corresponding descriptions in the method embodiments of this application; they will not be repeated below. Figure 8 As shown, the device may include a data acquisition module 81, a fault analysis module 82, a fault diagnosis module 83, and a solution recommendation module 84.

[0117] The data acquisition module 81 is used to acquire raw data including the fault problem of the target device. The raw data may include a data stream with device identification and time information, and / or fault multimedia content. The fault multimedia content may include: a video showing the fault process or a photograph showing the fault area.

[0118] The fault analysis module 82 is used to analyze the raw data containing fault problems to obtain problem description information.

[0119] When the raw data includes a data stream with device identification and time information, the fault analysis module 82 may include a data stream analysis module 821. The data stream analysis module 821 is configured to search for the corresponding task document from the task management database based on the device identification and time information of the target device; divide the data stream collected from the target device into corresponding data segments according to the standard process and steps in the task document; determine the target step of the fault by locating the error signal in the data segment; compare the part corresponding to the target step in the task document with the part in the data stream; and analyze the cause of the fault based on this comparison to generate problem description information.

[0120] When the original data contains multimedia content, the fault analysis module 82 may include a multimedia content analysis module 822. The multimedia content analysis module 822 is used to input the fault multimedia content into a pre-trained fault analysis model based on a convolutional neural network to obtain the problem description information output by the fault analysis model.

[0121] Furthermore, the multimedia content analysis module 822 can further provide problem description information to the user for inspection and receive the problem description information confirmed by the user. Then, the multimedia content analysis module 822 uses the problem description information confirmed by the user as the final problem description information.

[0122] The fault diagnosis module 83 is used to analyze the problem description information to obtain a diagnostic report. The fault diagnosis module 803 can input the problem description information into a pre-trained fault diagnosis model based on a convolutional neural network to obtain a diagnostic report output by the fault diagnosis model.

[0123] The solution recommendation module 84 is used to obtain fault video and / or document solutions based on the diagnostic report and recommend video and / or document solutions to the user. The knowledge graph includes: nodes representing faults, nodes representing video and / or document solutions, and multiple edges representing the relationships between nodes.

[0124] In fact, the fault diagnosis and solution recommendation device provided in this embodiment of the application can be implemented in various ways. For example, it can be compiled into a plugin installed in a smart terminal using an application programming interface that conforms to specific rules, or it can be packaged into an application for users to download and use.

[0125] When compiled as a plugin, the fault diagnosis and solution recommendation device can be implemented in various plugin forms, such as OCX, DLL, and CAB. The fault diagnosis and solution recommendation device provided by this implementation method can also be implemented using specific technologies, such as Flash plugin technology, RealPlayer plugin technology, MMS plugin technology, MIDI personnel plugin technology, or ActiveX plugin technology.

[0126] The fault diagnosis and solution recommendations provided by this implementation method can be stored in various storage media in either instruction-store or instruction-set-store mode. These storage media include, but are not limited to: floppy disks, optical disks, DVDs, hard disks, flash memory, USB flash memory, CF cards, SD cards, MMC cards, SM cards, Memory Sticks, and xD cards.

[0127] Furthermore, the fault diagnosis and solution recommendation method provided by this embodiment of the application can also be applied to flash memory (Nand-flash) based storage media, such as USB flash drives, CF cards, SD cards, SDHC cards, MMC cards, SM cards, Memory Sticks, and xD cards.

[0128] It should be clear that the operating system that operates in a computer can perform some or all of the actual operations not only by executing program code that the computer reads from the storage medium, but also by using instructions based on the program code, in order to achieve the functions of any of the embodiments described above.

[0129] For example, Figure 9 This is an exemplary structural diagram of a recommended device for another fault diagnosis and solution embodiment in this application. This device can be used to perform… Figures 1-7 The method shown, or used to implement Figure 8 The equipment in the system. For example... Figure 9 As shown, the device may include at least one memory 91 and at least one processor 92. Additionally, it may include other components such as communication ports, input / output controllers, network communication interfaces, etc. These components communicate via a bus 93, etc.

[0130] At least one memory 91 is used to store a computer program 911. In one example, the computer program can be understood to include... Figure 8 The device shown includes various modules. Additionally, at least one memory 91 can store an operating system, etc. Operating systems include, but are not limited to: Android, Symbian, Windows, Linux, etc.

[0131] At least one processor 92 is used to invoke a computer program stored in at least one memory 91 to execute the fault diagnosis and recommended solution methods described in the examples of this application. The processor 92 may be a CPU, processing unit / module, ASIC, logic module, or programmable gate array, etc., and it can receive and send data through a communication port.

[0132] The input / output controller has a display and input devices for inputting, outputting, and displaying relevant data.

[0133] This application also provides a fault diagnosis and solution recommendation system. The system may include... Figure 8 or Figure 9 The equipment shown, and Figure 2 The databases shown are: Task Management Database 11, Image Database 12, Teaching Resource Knowledge Graph 13, and Diagnostic Management Database 14.

[0134] In this embodiment, since all fault analysis, fault diagnosis, and solution recommendations are completed by a third party, users only need to perform some simple operations without needing to analyze the cause of the fault, thereby achieving intelligent fault diagnosis and solution recommendation and improving the efficiency of fault diagnosis and solution recommendation.

[0135] It should be understood that the use of “and / or” in this document is intended to include any and all possible combinations of one or more of the associated listed items.

[0136] The number of embodiments in this application is for illustrative purposes only and does not represent the advantages of the embodiments.

[0137] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for fault diagnosis and solution recommendation, characterized in that, The method includes: Acquire raw data, including fault information, from target equipment in a digital factory; Based on the type of the original data, the original data containing the fault is analyzed to obtain a problem description information; The problem description information is analyzed to obtain a diagnostic report; Based on the diagnostic report, a solution for the fault is obtained using a cloud knowledge graph, including video and / or document solutions, and the solution is recommended to the user. The cloud knowledge graph includes nodes representing the fault, video solutions, and / or document solutions, and multiple edges representing the relationships between the nodes. The original data containing the fault includes: a data stream with device identification and time information or fault multimedia content, wherein the fault multimedia content includes: a video containing the fault process or a photograph containing the fault area; The analysis of the raw data containing the fault yields a problem description, including: When the raw data includes the data stream, the corresponding task document is retrieved from a cloud task management database based on the device identifier and time information of the target device. The cloud task management database stores task documents containing standard processes and steps for each task corresponding to each device. Based on the standard processes and steps in the task documents, the data stream is divided into corresponding data segments. The target step with a fault is determined by locating error signals in the data segments. The task documents and the portions of the data stream corresponding to the target step are compared and analyzed to generate the problem description information. When the original data includes faulty multimedia content, the faulty multimedia content is input into a pre-trained neural network-based fault analysis model to obtain the problem description information output by the fault analysis model. The fault analysis model is trained using a large amount of historical faulty multimedia content stored in a cloud image database as input samples and using the historical problem description information corresponding to each historical faulty multimedia content as output samples.

2. The method according to claim 1, characterized in that, The acquisition of raw data, including fault data, of the target equipment in the digital factory includes: According to the set frequency, data streams with device identifiers and time information are collected from each target device based on the OPC UA protocol; In response to receiving a fault report from a user, the corresponding data stream of the target device is extracted from the collected data stream based on the device identifier and time information provided in the fault report.

3. The method according to claim 1, characterized in that, The analysis of the problem description information yields a diagnostic report, including: The problem description information is input into a pre-trained fault diagnosis model based on a convolutional neural network to obtain a diagnosis report output by the fault diagnosis model. The fault diagnosis model is trained using a large amount of historical problem description information as input samples and using historical diagnosis reports corresponding to each historical problem description information as output samples.

4. The method according to claim 3, characterized in that, The fault analysis model includes a multimedia content classification model and multiple fault analysis sub-models of different types; among them... The multimedia content classification model is used to classify the faulty multimedia content, and provides the faulty multimedia content to the corresponding type of fault analysis sub-model according to the classification results. Each fault analysis sub-model is used to output corresponding problem description information based on the input fault multimedia content.

5. The method according to claim 1, characterized in that, Further includes: The problem description information is fed back to the user for inspection, and the user's confirmation of the problem description information is received. The problem description information confirmed by the user shall be used as the final determined problem description information; The finalized problem description information and the fault multimedia content are stored as a new record in the cloud image database, so that the fault analysis model can be optimized based on the updated cloud image database.

6. A fault diagnosis and solution recommendation device, characterized in that, include: The data acquisition module is used to acquire raw data, including fault data, of the target equipment in the digital factory; The fault analysis module is used to analyze the original data containing faults based on the type of the original data to obtain a problem description information; The fault diagnosis module is used to analyze the problem description information and obtain a diagnosis report; The solution recommendation module is used to obtain video and / or document solutions for the fault based on the diagnostic report and a cloud knowledge graph, and recommend the solutions to the user; wherein, the cloud knowledge graph includes: nodes representing the fault, video solutions and / or document solutions respectively, and multiple edges representing the relationships between nodes; The original data containing the fault includes: a data stream with device identification and time information or fault multimedia content, wherein the fault multimedia content includes: a video containing the fault process or a photograph containing the fault area; The analysis of the raw data containing the fault yields a problem description, including: When the raw data includes the data stream, the corresponding task document is retrieved from a cloud task management database based on the device identifier and time information of the target device. The cloud task management database stores task documents containing standard processes and steps for each task corresponding to each device. Based on the standard processes and steps in the task documents, the data stream is divided into corresponding data segments. The target step with a fault is determined by locating error signals in the data segments. The task documents and the portions of the data stream corresponding to the target step are compared and analyzed to generate the problem description information. When the original data includes faulty multimedia content, the faulty multimedia content is input into a pre-trained neural network-based fault analysis model to obtain the problem description information output by the fault analysis model. The fault analysis model is trained using a large amount of historical faulty multimedia content stored in a cloud image database as input samples and using the historical problem description information corresponding to each historical faulty multimedia content as output samples.

7. A fault diagnosis and solution recommendation device, characterized in that, It includes at least one memory and at least one processor, wherein: The at least one memory is used to store computer programs; The at least one processor is used to call a computer program stored in the at least one memory to execute the fault diagnosis and solution recommendation method as described in any one of claims 1 to 5.

8. A fault diagnosis and solution recommendation system, characterized in that, include: Recommended equipment for fault diagnosis and solutions as described in any one of claims 6 to 7; The cloud task management database is used to store task documents containing the standard processes and steps for each task corresponding to each device. A cloud image database is used to store historical fault multimedia content and corresponding historical problem descriptions; The cloud knowledge graph includes nodes representing faults, video solutions, and / or document solutions, as well as multiple edges representing the relationships between nodes.

9. A computer-readable storage medium having a computer program stored thereon; characterized in that, The computer program can be executed by a processor to implement the fault diagnosis and solution recommendation method as described in any one of claims 1 to 5.

Images