Ship operation and maintenance guidance method based on AI and knowledge graph

Abstract

The invention relates to the technical field of ship operation and maintenance guidance, in particular to a ship operation and maintenance guidance method based on AI and a knowledge graph, by combining a natural language large model, an operation and maintenance large model and a knowledge graph network, an operator can ask questions through a natural and fuzzy language, and only needs to simply speak operation and maintenance requirements, so that the operation and maintenance guidance can be realized. An operator does not need to actively search problems, and the system can automatically push a solution. And meanwhile, the pushed solution object can be associated to an actual spatial position through an AR technology, so that an operator can conveniently and quickly position and dispose the object, and the operator can be intuitively and quickly guided to process, operate and maintain problems.

Description

Technical Field

[0001] The present invention relates to the technical field of ship operation and maintenance guidance, and in particular to a ship operation and maintenance guidance method based on AI and knowledge graph. Background Art

[0002] At present, domestic ships are becoming larger, more complex, and more high-value-added. The number of parts they contain is also increasing, and their operation and maintenance are becoming more and more difficult. In the past, shipyards generally delivered drawings, documents, and manuals to shipowners to serve operation and maintenance. Subsequently, against the background of rapid development of digitalization, the IETM (Interactive Electronic Technical Manual) interactive electronic manual came into being. It uses electronic methods to structure and integrate multimodal information such as text, models, pictures, videos, animations, etc. in the operation and maintenance manuals in traditional paper or technical materials to build a standardized structure tree for equipment use, maintenance, training, and other work. However, the solution pushed in this way cannot be located at the location of the corresponding operation and disposal parts on the actual ship. The operator needs to correspond to the location himself, which brings inconvenience to the operation and maintenance operations. Summary of the invention

[0003] In view of the shortcomings of the prior art mentioned above, the purpose of the present invention is to provide a ship operation and maintenance guidance method based on AI and knowledge graph, which is used to solve the problem that the pushed solution in the prior art cannot be located at the position of the corresponding operation and disposal parts on the actual ship, and the operator needs to locate the corresponding position himself, which brings inconvenience to operation and maintenance.

[0004] To achieve the above objectives and other related objectives, the present invention provides a ship operation and maintenance guidance method based on AI and knowledge graph, the guidance method comprising:

[0005] Build a large natural language model to recognize and process the text generated by the operator through AR terminal interaction;

[0006] Build a large model in the field of operation and maintenance to analyze failures and problems in the operation and maintenance process, and proactively push answers or solutions;

[0007] Construct a ship operation and maintenance knowledge graph network, which includes the troubleshooting process, appearance pictures, videos, and normal operating parameters of each system and equipment failure;

[0008] Using the AR terminal to input operation and maintenance requirements, and using the natural language big model to process the requirements;

[0009] Performing retrieval and reasoning on the processed requirements to obtain solutions;

[0010] Standardizing the solution, associating the name of the object model in the solution with the three-dimensional model, and associating the corresponding three-dimensional model with the solution;

[0011] The solution is displayed on the AR terminal by using the virtual-real superposition method.

[0012] Optionally, displaying the solution on the AR terminal includes displaying the solution directly on the AR terminal and displaying the solution at a corresponding three-dimensional model of the AR terminal.

[0013] Optionally, the step of performing retrieval and reasoning on the processed demand includes:

[0014] Using the knowledge graph network to perform retrieval reasoning on the processed requirements;

[0015] Determining whether the retrieval reasoning is successful;

[0016] If so, the result of the knowledge graph network retrieval reasoning is used as the solution;

[0017] If not, use the big model of the operation and maintenance domain to search and reason about the demand;

[0018] The result of retrieval and reasoning of the large model in the operation and maintenance field is used as the solution.

[0019] Optionally, the basis for judging whether the retrieval reasoning is successful is the complexity of the demand.

[0020] Optionally, the operation and maintenance big model is a cross-modal big model, and the training sources of the operation and maintenance big model include fault trees, mechanism models of various systems, standard operation and maintenance processes, vibration and noise data, and fault images.

[0021] Optionally, the knowledge graph network is a multimodal knowledge graph network.

[0022] Optionally, the requirements include inquiries into system functions, equipment parameters, and solutions to operational and maintenance issues.

[0023] Optionally, the input operation maintenance requirement is performed by inputting through AR voice or AR gesture.

[0024] Optionally, the AR terminal is AR glasses.

[0025] In a ship operation and maintenance guidance method based on AI and knowledge graph of the present invention, by combining the natural language big model, the operation and maintenance big model with the knowledge graph network, the operator can ask questions in natural and fuzzy language, and only needs to simply state the operation and maintenance requirements, without the operator actively searching for the problem, and the system will automatically push the solution. At the same time, the pushed solution object can be associated with the actual space position through AR technology, which is convenient for the operator to quickly locate the disposal object and intuitively and quickly guide the operator to deal with the operation and maintenance problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a ship operation and maintenance guidance method based on AI and knowledge graph according to an embodiment of the present invention;

[0027] Figure 2 It is a flowchart of the steps of searching and reasoning the processed operation and maintenance requirements in one embodiment of the present invention. DETAILED DESCRIPTION

[0028] Refer to the following Figure 1 and Figure 2 To describe a ship operation and maintenance guidance method based on AI and knowledge graph of the present invention. In the description of this embodiment, it should be understood that although the steps in the flowcharts of the embodiments of the present invention are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this document, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in each embodiment may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0029] Unless otherwise expressly specified and defined, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features, that is, include one or more of the features. In the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. When a feature "includes or contains" one or some of the features it covers, unless otherwise specifically described, this indicates that other features are not excluded and may further include other features. Terms such as "set", "install", "connect", "connect", "fix", "couple" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. A person of ordinary skill in the art should be able to understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0030] In addition, in the description of this embodiment, the first feature being "above" or "below" the second feature may include the first and second features being in direct contact, or may include the first and second features not being in direct contact but being in contact through another feature between them. That is, in the description of this embodiment, the first feature being "above", "above", and "above" the second feature includes the first feature being directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. The first feature being "below", "below", or "below" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0031] In the description of the present embodiment, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0032] like Figure 1 As shown, an embodiment of the present invention provides a ship operation and maintenance guidance method based on AI and knowledge graph, and the guidance method includes:

[0033] Step S10, construct a large natural language model to recognize and process the text generated by the operator through the AR terminal interaction.

[0034] Step S20, constructing a large model in the field of operation and maintenance. The large model in the field of operation and maintenance is used to analyze faults and problems in the process of operation and maintenance, and actively push answers or solutions. The training sources of the large model of operation and maintenance include fault trees, mechanism models of various systems, standard operation and maintenance processes, vibration and noise data, and fault images, so that the analysis of the large model of operation and maintenance is more comprehensive and the input types accepted are more diversified.

[0035] Step S30, constructing a ship operation and maintenance knowledge graph network. The knowledge graph network includes the troubleshooting process, appearance pictures, videos, and normal operation parameters of each system and equipment failure. Optionally, the knowledge graph network is a multimodal knowledge graph network.

[0036] Step S40, inputting operation and maintenance requirements using the AR terminal, and processing the requirements using the natural language big model. Optionally, the AR terminal is AR glasses. Using AR glasses, inputting through AR voice or AR gestures. Using the natural language big model to process the operation and maintenance requirements, it is convenient to convert the operation and maintenance requirements into a form that is convenient for subsequent retrieval and reasoning. The operation and maintenance requirements include inquiring about system functions, equipment parameters, and solutions to operation and maintenance problems. For example, before starting the engine, the crew of the generator room wears AR glasses and asks through voice input: "I see a little condensation water on the flange of the generator outlet pressure valve, will it affect the normal operation of the unit." The natural language big model converts the above questions into text, and then simplifies them into the following content: "What parts and equipment are next to the generator outlet pressure valve", "Which parts and equipment near the outlet pressure valve will discharge water vapor or local low temperature", "What is the reason for condensation water on the flange", "What factors are related to the reason for condensation water on the flange of the generator outlet pressure valve", "What are the methods to avoid condensation water on the flange", "Will the condensation water on the generator outlet pressure valve affect the operation of the generator set", etc., so as to facilitate subsequent retrieval and reasoning.

[0037] Step S50, searching and reasoning the processed operation and maintenance requirements to obtain corresponding solutions.

[0038] Furthermore, if Figure 2 As shown in FIG. 1 , the steps of retrieving and reasoning the processed operation and maintenance requirements include:

[0039] Step S51, using the knowledge graph network to search and reason about the processed operation and maintenance requirements;

[0040] Step S52, determine whether the retrieval reasoning is successful, if so, execute step S53, if not, execute step S54.

[0041] Step S53, using the result of knowledge graph network retrieval reasoning as a solution.

[0042] Step S54: Use the big model of the operation and maintenance domain to search and reason about the processed operation and maintenance requirements.

[0043] Step S55, using the result of retrieval and reasoning of the large model in the operation and maintenance field as a solution.

[0044] Furthermore, the basis for judging whether the retrieval reasoning is successful is the complexity of the input operation and maintenance requirements. When the requirements are relatively complex, it is judged that the knowledge graph network retrieval reasoning fails.

[0045] For example, the knowledge graph is used to search and reason about the processed problems. First, the objects in and around the generator outlet pressure valve are searched. Then, whether there are parts and equipment around it that can produce water or water vapor, and whether there are equipment parts around it that can produce local low temperatures to condense steam into water. Subsequently, a search is also conducted to see whether a small amount of water at the flange pipe mouth will affect the normal operation of the generator. Since the above problems are relatively complex, the knowledge graph network judgment reasoning is unsuccessful. At this time, the large model in the operation and maintenance field reasons about the processed problems and gives the solution: the condensed water at the flange of the outlet pressure valve may be caused by the leakage of the steam valve above and the contact with the condensed water generated by the outlet pressure valve with a lower temperature. Please check whether the pressure gauge of the steam inlet valve above is normal. If it is normal, no need to pay attention. If it is not normal, close the steam valve.

[0046] Step S60, standardize the solution, that is, associate the name of the object model in the solution with the three-dimensional model, and associate the corresponding three-dimensional model with the solution. For example, ① check the steam valve pressure gauge; ② determine whether the generator steam valve pressure gauge is within 50MPa, if so, push the system normal and no need to pay attention to the prompt, otherwise push the generator steam valve close prompt. After reading, the system associates "generator steam valve pressure gauge" and "generator steam valve" with the generator steam valve pressure gauge model and generator steam valve model in the three-dimensional model respectively, and associates the three-dimensional model with the solution.

[0047] Step S70, using the virtual-real superposition method, the solution is displayed on the AR terminal, including displaying the solution directly on the AR terminal and displaying the solution on the corresponding three-dimensional model of the AR terminal. For example, the operator can see the generator steam valve pressure gauge position is highlighted in the AR glasses, and a prompt box is displayed to inform the operator to check the pressure indication number. After the operator checks, the AR interaction clicks "outside" the normal range, and then the AR glasses will see the generator steam valve highlighted and pop up a prompt "Close the generator steam valve". The operator closes the generator steam valve according to the prompt.

[0048] By combining the natural language big model, the operation and maintenance big model with the knowledge graph network, operators can ask questions in natural and ambiguous language, and only need to simply state the operation and maintenance requirements. The system will automatically push solutions without the operator actively searching for the problem. At the same time, the pushed solution object can be associated with the actual space location through AR technology, which is convenient for the operator to quickly locate the disposal object and intuitively and quickly guide the operator to deal with the operation and maintenance problem.

[0049] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A ship operation and maintenance guidance method based on AI and knowledge graph, characterized in that: The guidance method includes: Build a large natural language model to recognize and process the text generated by the operator through AR terminal interaction; Build a large model in the field of operation and maintenance to analyze failures and problems in the operation and maintenance process, and proactively push answers or solutions; Construct a ship operation and maintenance knowledge graph network, which includes the troubleshooting process, appearance pictures, videos, and normal operating parameters of each system and equipment failure; Using the AR terminal to input operation and maintenance requirements, and using the natural language big model to process the requirements; Performing retrieval and reasoning on the processed requirements to obtain solutions; Standardizing the solution, associating the name of the object model in the solution with the three-dimensional model, and associating the corresponding three-dimensional model with the solution; The solution is displayed on the AR terminal by using the virtual-real superposition method.

2. The ship operation and maintenance guidance method based on AI and knowledge graph according to claim 1 is characterized in that: The displaying of the solution on the AR terminal includes directly displaying the solution on the AR terminal and displaying the solution at the corresponding three-dimensional model of the AR terminal.

3. The ship operation and maintenance guidance method based on AI and knowledge graph according to claim 1 is characterized in that: The step of performing retrieval and reasoning on the processed requirements includes: Using the knowledge graph network to perform retrieval reasoning on the processed requirements; Determining whether the retrieval reasoning is successful; If so, the result of the knowledge graph network retrieval reasoning is used as the solution; If not, use the big model of the operation and maintenance domain to search and reason about the demand; The result of retrieval and reasoning of the large model in the operation and maintenance field is used as the solution.

4. The ship operation and maintenance guidance method based on AI and knowledge graph according to claim 3 is characterized in that: The basis for judging whether the retrieval reasoning is successful is the complexity of the demand.

5. The ship operation and maintenance guidance method based on AI and knowledge graph according to claim 1 is characterized in that: The operation and maintenance big model is a cross-modal big model, and the training sources of the operation and maintenance big model include fault trees, mechanism models of various systems, standard operation and maintenance processes, vibration and noise data, and fault images.

6. The ship operation and maintenance guidance method based on AI and knowledge graph according to claim 1 is characterized in that: The knowledge graph network is a multimodal knowledge graph network.

7. The ship operation and maintenance guidance method based on AI and knowledge graph according to claim 1 is characterized in that: The requirements include inquiring about system functions, equipment parameters, and solutions to operational and maintenance issues.

8. The ship operation and maintenance guidance method based on AI and knowledge graph according to claim 1 is characterized in that: The input operation and maintenance requirements are met by inputting through AR voice or AR gestures.

9. The ship operation and maintenance guidance method based on AI and knowledge graph according to claim 1 is characterized in that: The AR terminal is AR glasses.

Images