A ship fault diagnosis method, device and electronic equipment based on topological constraints

Abstract

The application provides a ship fault diagnosis method and device based on topological constraints and an electronic device. The method comprises: acquiring ship multi-modal fault data, mapping the ship multi-modal fault data to fault nodes in a ship knowledge graph, constructing a fault context subgraph, and constructing a hybrid semantic query vector with the multi-modal fault data; retrieving an initial set in a ship maintenance database based on the hybrid semantic query vector, mapping each text segment in the initial set to an associated node in the ship knowledge graph, calculating the topological connectivity of each associated node to each fault node, and screening a candidate set; constructing a corresponding causal path subgraph based on the associated nodes of each text segment in the candidate set, calculating the graph edit distance score of the causal path subgraph and the fault context subgraph, and calculating the cross semantic attention score of the hybrid semantic query vector and the candidate set, screening a reference text segment based on the two scores, and using a large language model to infer a fault diagnosis suggestion.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology, specifically to a ship fault diagnosis method, device, and electronic equipment based on topology constraints. Background Technology

[0002] With the rapid development of shipping automation and the increasing size of ships, the complexity of ship electromechanical equipment systems has increased exponentially. However, current ship operation and maintenance practices generally face the following technical challenges: First, ship fault diagnosis relies heavily on the extensive practical experience of marine engineers. As experienced technical experts gradually retire, younger crew members lack sufficient practical experience, leading to low efficiency in fault handling, high risk of misjudgment, and difficulty in effectively passing on implicit expert experience during crew rotation.

[0003] Secondly, with the development of general artificial intelligence, Retrieval-Augmented Generation (RAG) technology has begun to be introduced into the industrial operations and maintenance field to address the problem of knowledge silos. However, conventional RAG technology mainly relies on the cosine similarity of text semantic vectors for retrieval. Due to the complex physical cascading and topological dependencies of various ship systems (such as power, lubrication, and cooling systems), simple vector semantic retrieval cannot perceive the topological connection logic of the equipment. This can easily lead to the system recalling maintenance manual fragments that are highly similar in literal semantics but have no relation in physical topology, thereby inducing a serious logical illusion in the large language model, which cannot meet the high safety requirements of ship industrial operations and maintenance scenarios.

[0004] Furthermore, existing expert systems or fault feedback mechanisms often rely on simple manual scoring or frequency accumulation to update diagnostic rules. This static feedback model ignores the nonlinear aging characteristics of ship equipment (i.e., old maintenance experience becomes obsolete over time) and fails to consider the heterogeneous impact of different sea states, speeds, and load conditions on fault causes. This results in the system's inability to achieve accurate adaptive diagnosis based on the current operating environment, making it difficult to truly transform the tacit experience of marine engineers into high-value digital assets.

[0005] Therefore, developing an intelligent diagnostic system capable of accurate identification of ship faults, multimodal interaction, traceable answers, and self-learning closed-loop evolution has become a key issue that needs to be addressed in the digital transformation of the shipping industry. Summary of the Invention

[0006] In view of this, this application proposes a ship fault diagnosis method, device and electronic equipment based on topology constraints to overcome the technical problems in existing ship operation and maintenance, such as the difficulty in passing on expert experience, serious fault information silos, and high economic losses caused by passive operation and maintenance.

[0007] Specifically, this application is implemented through the following technical solution: According to a first aspect of the embodiments of this specification, a ship fault diagnosis method based on topology constraints is provided, the method comprising the following steps: Step S1: Obtain multimodal fault data of the target ship fault, including fault description text, voice fault commands, and images of the fault location; Step S2: The entity chain index algorithm is used to map the multimodal fault data to the ship knowledge graph used to characterize the ship's physical topology and fault causal logic, to obtain the fault nodes corresponding to the multimodal data. A fault context subgraph is constructed based on all fault nodes, and a hybrid semantic query vector is constructed based on the fault context subgraph and the multimodal fault data. Step S3: Based on the hybrid semantic query vector, retrieve an initial maintenance semantic vector set from the pre-constructed ship maintenance database. The ship maintenance database stores text fragments of ship maintenance manuals and technical specifications and their corresponding maintenance semantic feature vectors. Use the entity chaining algorithm to map each text fragment to the ship knowledge graph to obtain the associated nodes corresponding to each text fragment, and calculate the topological connectivity from each associated node to each fault node. Based on the topological connectivity, retrieve a candidate maintenance semantic vector set from the initial maintenance semantic vector set. Step S4: For each text segment in the candidate maintenance semantic vector set, construct a corresponding causal path subgraph based on its associated nodes, calculate the graph edit distance score between the causal path subgraph and the fault context subgraph; and calculate the cross semantic attention score between the hybrid semantic query vector and each maintenance semantic feature vector in the candidate maintenance semantic vector set. Based on the graph edit distance score and the cross semantic attention score, select reference text segments from the candidate maintenance semantic vector set and obtain the causal logic path corresponding to the reference text segment. Step S5: Use a large language model to perform fault reasoning on the reference text fragment and its corresponding causal logic path to obtain fault diagnosis suggestions.

[0008] According to a second aspect of the embodiments of this specification, a ship fault diagnosis device based on topology constraints is provided, the device comprising: The fault data acquisition unit is used to acquire multimodal fault data of the target ship fault, including fault description text, voice fault commands, and images of the fault location. The query vector construction unit is used to map the multimodal fault data to a ship knowledge graph that represents the ship's physical topology and fault causal logic using the entity chain index algorithm, to obtain the fault nodes corresponding to the multimodal data, construct a fault context subgraph based on all fault nodes, and construct a hybrid semantic query vector based on the fault context subgraph and the multimodal fault data. A topology constraint retrieval unit is used to retrieve an initial set of maintenance semantic vectors from a pre-constructed ship maintenance database based on the hybrid semantic query vectors. The ship maintenance database stores text fragments of ship maintenance manuals and technical specifications and their corresponding maintenance semantic feature vectors. An entity chain indexing algorithm is used to map each text fragment to the ship knowledge graph to obtain the associated nodes corresponding to each text fragment, and the topological connectivity from each associated node to each fault node is calculated. Based on the topological connectivity, a candidate set of maintenance semantic vectors is retrieved from the initial set of maintenance semantic vectors. The text reordering unit constructs a corresponding causal path subgraph based on its associated nodes for each text segment in the candidate maintenance semantic vector set, calculates the graph edit distance score between the causal path subgraph and the fault context subgraph, and calculates the cross-semantic attention score between the hybrid semantic query vector and each maintenance semantic feature vector in the candidate maintenance semantic vector set. Based on the graph edit distance score and the cross-semantic attention score, it selects reference text segments from the candidate maintenance semantic vector set and obtains the causal logic path corresponding to the reference text segment. The ship fault diagnosis unit uses a large language model to perform fault reasoning on the reference text fragments and their corresponding causal logic paths to obtain fault diagnosis suggestions.

[0009] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising: a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in the first aspect.

[0010] The embodiments of this application have at least the following technical effects: First, this application anchors the image and text features of the target fault to the relevant fault nodes in the ship knowledge graph, and extracts K-hop neighbors to construct a fault context subgraph, transforming discrete multimodal signals into hybrid semantic query vectors carrying equipment topological location attributes, which greatly improves the accuracy of multimodal feature fusion in highly interfering industrial environments.

[0011] Second, in response to the pain point of conventional RAG technology blindly pursuing semantic similarity while ignoring physical mechanisms, this application introduces graph topological connectivity based on random walks and combines it with graph editing distance for secondary rearrangement. This can filter out semantically similar but physically inaccessible interference information, ensuring that the generated diagnostic suggestions are reasonable in terms of engineering mechanisms and accompanied by a traceable graph evidence chain. Attached Figure Description

[0012] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of a ship fault diagnosis method based on topology constraints. Figure 2 This is a schematic diagram of the architecture of a ship fault diagnosis system shown in an exemplary embodiment of this application; Figure 3 This is a schematic diagram illustrating a diagnostic process based on a ship knowledge graph, as shown in an exemplary embodiment of this application. Figure 4 This is a schematic diagram illustrating a ship fault diagnosis process according to an exemplary embodiment of this application; Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment of this application; Figure 6 This is a block diagram illustrating a ship fault diagnosis device based on topology constraints, as shown in an exemplary embodiment of this application. Detailed Implementation

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0014] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0015] This application provides a ship fault diagnosis method based on topology constraints. This method can be executed by a ship fault diagnosis system, which can be deployed on ship control equipment or in the cloud.

[0016] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of a ship fault diagnosis method based on topology constraints, as shown in this application. Figure 1 As shown, the ship fault diagnosis method includes the following steps: Step S1: Obtain multimodal fault data of the target ship fault, including fault description text, voice fault commands, and images of the fault location.

[0017] Step S2: The entity chain index algorithm is used to map the multimodal fault data to a ship knowledge graph that represents the ship's physical topology and fault causal logic to obtain the fault nodes corresponding to the multimodal data. A fault context subgraph is constructed based on all fault nodes, and a hybrid semantic query vector is constructed based on the fault context subgraph and the multimodal fault data.

[0018] In this embodiment, the ship knowledge graph is a graph-based knowledge database used to structurally describe the physical topology relationships and fault propagation logic of ship systems and equipment. Specifically, this knowledge graph contains various types of nodes and edges. Figure 3 As shown, node types include, for example, device / component nodes, subsystem nodes, fault phenomenon nodes, fault cause nodes, and maintenance solution nodes. Each node represents a unique physical entity or abstract concept. Directed edges between nodes define their relationships; for example, device / component nodes and fault phenomenon nodes are connected by "behave as" type relationship edges, and fault phenomenon nodes and fault cause nodes are connected by "caused by" or "caused by" causal edges.

[0019] In the ship knowledge graph of this embodiment, its nodes and edges together constitute a topological space reflecting the physical connectivity and fault causal chains of the ship's electromechanical systems. The length of the connection path between any two nodes and the edge type on the path characterize the engineering mechanism correlation between the corresponding physical entities or fault logic. The subsequent calculation of topological connectivity and graph edit distance (GED) analysis in this embodiment are all completed within the topological space defined by this ship knowledge graph.

[0020] Step S3: Based on the hybrid semantic query vector, retrieve an initial maintenance semantic vector set from the pre-constructed ship maintenance database. The ship maintenance database stores text fragments of ship maintenance manuals and technical specifications and their corresponding maintenance semantic feature vectors. Use the entity chain index algorithm to map each text fragment to the ship knowledge graph to obtain the associated nodes corresponding to each text fragment, and calculate the topological connectivity from each associated node to each fault node. Based on the topological connectivity, retrieve a candidate maintenance semantic vector set from the initial maintenance semantic vector set.

[0021] This embodiment can pre-segment unstructured text data such as ship maintenance manuals, technical specifications, and historical maintenance reports into text blocks, and use a pre-trained embedded model to convert each text block into a fixed-dimensional maintenance semantic feature vector. Then, the maintenance semantic feature vector is associated with the corresponding original text fragment and stored to obtain the ship maintenance database of this embodiment.

[0022] In the process of ship fault diagnosis, the cosine similarity or other distance metric between the hybrid semantic query vector and each maintenance semantic feature vector in the ship maintenance database is calculated. Based on the semantic similarity, a batch of original maintenance data fragments that are most relevant to the target fault description at the textual semantic level are initially screened and recalled. Then, the initial screening and recall results are further filtered based on topological connectivity to obtain a set of candidate maintenance semantic vectors.

[0023] Step S4: For each text segment in the candidate maintenance semantic vector set, construct a corresponding causal path subgraph based on its associated nodes, calculate the graph edit distance score between the causal path subgraph and the fault context subgraph; and calculate the cross-semantic attention score between the hybrid semantic query vector and each maintenance semantic feature vector in the candidate maintenance semantic vector set. Based on the graph edit distance score and the cross-semantic attention score, select reference text segments from the candidate maintenance semantic vector set and obtain the causal logic path corresponding to the reference text segment.

[0024] This embodiment uses the entity chain indexing algorithm to map each text fragment to the ship knowledge graph to obtain associated nodes. Based on the neighbor nodes and relationship edges within the K-hop range of the associated nodes, a causal path subgraph is constructed.

[0025] The causal logic path refers to the directed path formed in the ship knowledge graph, starting from the fault node in the fault context subgraph, passing through one or more preset mechanism relationship edges, and finally reaching the associated node corresponding to the corresponding text fragment (the associated node is usually the fault cause node or maintenance plan node).

[0026] The predefined mechanistic relationship edges refer to edge types predefined in the ship knowledge graph that characterize the physical connection relationships or fault propagation causal relationships between equipment / components. Specifically, they include, but are not limited to: "Leads_To" edges and "Caused_By" edges used to characterize fault causal transmission; and "Has_Component" edges and "Connected_To" edges used to characterize physical structural dependencies. These mechanistic relationship edges constitute the topological foundation for fault logic reasoning in the ship knowledge graph, distinguishing them from purely semantic-level classification or indexing relationships.

[0027] Step S5: Use a large language model to perform fault reasoning on the reference text fragment and its corresponding causal logic path to obtain fault diagnosis suggestions.

[0028] In this embodiment, the reference text fragment and its corresponding causal logic path are processed according to a preset prompt template to form a structured input prompt, which is then input into a large language model to generate a traceable diagnostic suggestion with an attached chain of evidence.

[0029] The Prompt template in this embodiment can be configured as needed. For example, it may include role settings, factual context, fault description, and output constraints. The role settings instruct the large language model to play the role of a marine engineering expert, constraining it to reason only based on the provided context; the factual context section fills in the reference text fragment and the causal logic path; the fault description section fills in the fault phenomenon text after multimodal data fusion; and the output constraints require the model to output structured diagnostic suggestions (including root cause analysis and maintenance plans) and mark the source of evidence for each conclusion (such as the original text position in the reference fragment or the node in the causal logic path).

[0030] In some embodiments, the faulty node in step S2 is obtained through the following steps: The voice fault command is subjected to speech recognition to obtain the fault command text; and the image of the fault location is subjected to target detection to obtain the fault location label; the fault description text, the fault command text, and the fault location label are mapped to the ship knowledge graph through the entity chain indexing algorithm to obtain at least one fault node.

[0031] In some embodiments, the fault context subgraph in step S2 is obtained through the following steps: Using each fault node as an anchor point, extract its neighbor nodes and relational edges within a range of K hops in the ship knowledge graph to construct a fault context subgraph; K is a preset positive integer.

[0032] In some embodiments, the hybrid semantic query vector in step S2 is obtained through the following steps: The fault context subgraph is aggregated using a graph neural network (GNN) to obtain a graph representation vector; the semantic embedding vector corresponding to the multimodal fault data is calculated; and the graph representation vector and the semantic embedding vector are concatenated using a tensor to obtain the hybrid semantic query vector.

[0033] This embodiment employs a deep learning-based speech recognition model, such as Whisper or a similar end-to-end speech recognition network, to transcribe user-inputted voice fault commands into text. For example, if the user inputs "host oil pressure low," the system transcribes it into the text "host oil pressure low."

[0034] This embodiment employs a convolutional neural network-based object detection model, such as the YOLO series or Faster R-CNN, to perform real-time detection on user-uploaded images of faulty parts, identify the equipment component regions in the images, and output corresponding faulty part labels. For example, if a user takes a photo of the appearance of an oil filter, the object detection model will output the label "oil filter".

[0035] The fault description text, fault instruction text, and fault location tags obtained in the above steps are integrated to obtain a multi-source textual fault feature set. Then, the Entity Linking algorithm is used to map each text element in the fault feature set to a pre-constructed ship knowledge graph.

[0036] To enrich the fault context information, the system also uses the fault node as an anchor point to extract its neighbor nodes and relational edges within a K-hop range from the ship's knowledge graph. The value of K can be preset according to the density of the ship's knowledge graph and the actual application scenario. A fault context subgraph is constructed based on all anchor nodes and their K-hop neighbor nodes. Through this expansion, the fault node not only includes entities directly related to the user input, but also covers its neighboring entities in terms of physical topology and causal logic, thus constructing a fault context subgraph with richer structural information.

[0037] The graph neural network in this embodiment can adopt a Graph Attention Network (GAT) or Graph Convolutional Network (GCN) architecture. Taking GAT as an example, it assigns different attention weights to neighboring nodes through a self-attention mechanism, thereby aggregating the feature information of neighboring nodes and updating the representation vector of each node. After iterative propagation and aggregation through multiple layers of graph neural networks, the topological structure information of the entire subgraph is encoded into a fixed-dimensional graph representation vector. This graph representation vector implicitly contains the physical topological relationship and causal propagation path information between the faulty node and its surrounding nodes. Simultaneously, for the fault description text and the fault instruction text obtained through speech recognition transcription, a pre-trained language model, such as BERT, RoBERTa, or an embedding model fine-tuned for the industrial field, is used to encode them into semantic embedding vectors. These semantic embedding vectors represent the semantic connotation of the user-input fault information.

[0038] The graph representation vector and the semantic embedding vector are tensor concatenated along the feature dimension to generate a hybrid semantic query vector. This hybrid semantic query vector carries both fault description semantic content from the multimodal input data and the physical topological location and causal context of the fault node from the ship knowledge graph.

[0039] Thus, this application, through the above embodiments, uniformly maps discrete, heterogeneous multimodal fault information to the topological space of the ship knowledge graph, eliminating semantic ambiguity between modalities; extracts structured features of the fault context subgraph through graph neural networks, enabling the query vector to carry topological constraint information such as the physical location of the fault node, equipment affiliation, and fault propagation path; and deeply integrates topological and semantic information to generate a unified query representation with both semantic understanding and physical perception capabilities, providing coordinates in physical space for the subsequent retrieval process, making the retrieval results more likely to be maintenance data that is physically connected or causally related to the current fault.

[0040] In some embodiments, step S3 includes: Based on the similarity between the hybrid semantic query vector and the maintenance semantic feature vector, the initial maintenance semantic vector set is obtained; the topological connectivity from each associated node to each fault node is calculated based on the random walk algorithm, and the causal logical path is obtained based on the node sequence traversed during the random walk, the causal logical path including nodes directly connected to the current fault node in the ship knowledge graph through preset mechanism relationship edges; the similarity is corrected using the topological connectivity score, and the candidate maintenance semantic vector set is obtained based on the corrected similarity.

[0041] This embodiment calculates the similarity between the hybrid semantic query vector and each maintenance semantic feature vector in the ship maintenance database. For example, cosine similarity is used as the metric, and the similarity scores are sorted from high to low. A preset number (e.g., the first 50) of maintenance text fragments and their corresponding semantic feature vectors are recalled to form an initial maintenance semantic vector set, so as to achieve rapid semantic filtering of massive maintenance data.

[0042] Then, a random walk algorithm is used to calculate the topological connectivity between each associated node and the faulty node in the ship knowledge graph. Specifically, starting from the faulty node, multiple random walks are performed according to the mechanistic connection edges in the graph. The probability frequency of reaching each associated node during the walk is counted, and this frequency is used as the topological connectivity. The higher the topological connectivity score, the stronger the connection between the associated node and the faulty node in terms of physical topology and causal logic. At the same time, the node sequence traversed in each walk is recorded during the random walk. All nodes in the node sequence that are directly connected to the current faulty node through preset mechanistic relationship edges, along with their connection relationships, are merged to form one or more causal logic paths. Each causal logic path represents a complete causal link starting from the fault phenomenon, propagating along the physical mechanism relationship edges, and finally reaching a certain device or cause node.

[0043] The calculated topological connectivity is used to correct the original cosine similarity. For example, a product correction method is used: Corrected similarity = Original cosine similarity × (1 + Topological connectivity). The topological connectivity score is normalized to [0,1]. Higher connectivity results in a greater gain in the corrected similarity; when connectivity is 0, the corrected similarity equals the original cosine similarity. Through this correction, physically related maintenance text fragments receive higher ranking positions, while fragments that are topologically unreachable or have low connectivity are relatively demoted.

[0044] Based on the corrected similarity, all text segments in the initial maintenance semantic vector set are reordered in descending order, and the top-ranked text segments (e.g., the top 5) and their corresponding semantic feature vectors are selected to form a candidate maintenance semantic vector set.

[0045] This embodiment calculates topological connectivity through random walks, injecting physical topological constraints into the sorting process. This allows the system to automatically reduce or eliminate such interfering information, significantly improving the engineering relevance of the recall. Furthermore, the node sequences recorded during the random walk are transformed into causal logical paths. These paths are not only used for similarity correction in this step but also provide structured graph evidence for subsequent graph editing distance calculations and evidence chain tracing, allowing users to intuitively see the complete propagation chain of the fault from phenomenon to cause. In addition, through topological connectivity correction, the maintenance data in the candidate set are highly correlated with the current fault in terms of physical mechanism. The large language model infers based on these high-quality, physically consistent facts, reducing the probability of the model fabricating unreasonable conclusions due to receiving irrelevant information and improving the reliability and interpretability of the diagnostic results.

[0046] In some embodiments, the process of obtaining the reference text fragment in step S4 is as follows: The graph edit distance score and the cross semantic attention score are weighted and fused, and the text segment corresponding to the highest total score after weighted fusion is used as the reference text segment.

[0047] For each maintenance text fragment in the candidate maintenance semantic vector set, based on the associated nodes corresponding to the text fragment, neighboring nodes and relational edges within the K-hop range of the associated node are extracted from the ship knowledge graph to construct a causal path subgraph for the text fragment. Simultaneously, the constructed fault context subgraph is used as the baseline subgraph. A graph edit distance algorithm is employed to calculate the similarity between the causal path subgraph and the fault context subgraph. In this embodiment, the graph edit distance is mapped to a score between 0 and 1 through a normalization transformation (e.g., taking the reciprocal or using exponential decay), referred to as the graph edit distance score. A higher graph edit distance score indicates a more isomorphic topological structure between the two subgraphs, meaning a better match between the equipment relationship network corresponding to the candidate maintenance data and the current fault's topological environment.

[0048] This embodiment employs a pre-trained cross-encoder, which simultaneously inputs the text portion corresponding to the hybrid semantic query vector (i.e., the user fault description text and the fault instruction text) and the candidate maintenance text fragments. The deep semantic relevance between the two is calculated through the cross-attention mechanism of the Transformer, and a score (e.g., between 0 and 1) representing the degree of semantic matching is output, which is called the cross-semantic attention score. This score reflects the degree of semantic fit between the maintenance text fragment and the user fault description.

[0049] For each candidate maintenance text segment, its graph edit distance score and cross-semantic attention score are weighted and fused to obtain the total score. All candidate maintenance text segments are sorted in descending order of total score, and the text segment with the highest total score is selected as the reference text segment. At the same time, the causal logic path corresponding to this text segment is determined as the final causal logic path for use by the large language model for inference.

[0050] Thus, graph edit distance can sensitively capture structural differences between two subgraphs, including missing or redundant nodes and differences in connectivity. In ship fault diagnosis, even similar fault phenomena may have fundamentally different maintenance solutions if they occur in different subsystems or on equipment with different connectivity. Graph edit distance scores can effectively distinguish these subtle but crucial topological differences, avoiding the misselection of maintenance data with structural mismatches as references. Meanwhile, the cross-encoder allows query and document tokens to interact at multiple levels through an attention mechanism, capturing more complex semantic relationships. This is particularly important for accurately understanding the subtle intentions in user fault descriptions, improving the accuracy of semantic matching. Therefore, this embodiment, by weightedly fusing graph edit distance scores and cross-semantic attention scores, ensures that the filtering results simultaneously meet the dual constraints of semantic relevance and physical structural matching. This allows for more accurate location of maintenance data applicable to the current fault scenario, facilitating the generation of concise and accurate diagnostic recommendations.

[0051] In some embodiments, Figure 1 The ship fault diagnosis method shown also includes the following step S6: The system acquires the time interval between fault occurrences, the current operating condition characteristics of the vessel, and the user's confirmation feedback on the fault diagnosis suggestions. Based on the time interval between fault occurrences, the current operating condition characteristics of the vessel, and the confirmation feedback, the system updates the weights of the relationship edges in the vessel knowledge graph. The updated weights are then written back to the vessel knowledge graph for subsequent vessel fault diagnosis.

[0052] The repair confirmation feedback information includes, but is not limited to: The system assigns a confidence score to the successful repair, where the user rates the effectiveness of the diagnostic recommendations using a scale of 1 to 5 (5 representing completely correct and 1 representing completely incorrect). The system then normalizes this score to the [0,1] interval as the confidence score. .

[0053] Fault occurrence time interval: The system automatically records the time when this fault occurred. And compared with the time when this diagnostic path was last adopted in the historical record. By comparing, the time interval is obtained. .

[0054] Ship operating condition characteristic information: When a fault occurs, the system automatically collects the ship's current operating condition parameters and forms an operating condition feature vector. The operating condition feature vector includes, but is not limited to: ship speed, main engine speed, exhaust gas temperature, cooling water temperature, lubricating oil pressure, fuel consumption rate, draft, sea state level, etc.

[0055] Accordingly, the updated weight expression for the relationship edge between fault phenomenon node i and fault cause node j in the ship knowledge graph is: in: Indicates the first The edge weights after the round update Indicates the first Edge weights of the wheel; This represents a pre-defined base weight decay coefficient, used to simulate the natural decay of old experience due to equipment aging. For example... ; This represents the pre-set feedback learning rate coefficient, used to control the strength of the impact of new feedback on the weights. ; This indicates the total number of times the diagnostic path corresponding to this relation edge has been adopted and repaired throughout history; Indicates the first Confidence score for successful repair; Represents the time decay term. This represents the decay constant; the longer the time period, the smaller the empirical contribution. Indicates the current fault diagnosis time. Indicates the first The time of occurrence of each historical fault; This represents the operating condition similarity factor; the more similar the operating conditions, the larger this value. Represents the feature vector of the ship's current operating condition. Indicates the first Ship condition feature vector at the time of each historical failure; This represents the pre-set minimum weight threshold used to ensure the connectivity of the ship knowledge graph.

[0056] This embodiment will use the new weights obtained through evolutionary calculation. Write back to the ship's knowledge graph and replace or update the corresponding relationship edge attributes. When a similar fault diagnosis request is received again, the system will directly read the updated edge weights when calculating the topological connectivity of the random walk in step S3. Relationship edges with higher weights are prioritized in the random walk, thus giving higher priority to high-value, high-matching historical maintenance experience in retrieval and recommendation ranking.

[0057] Through the above mechanism, the diagnostic knowledge in the ship knowledge graph evolves continuously with each maintenance feedback, forming a self-evolving closed loop of diagnosis → feedback → learning → optimized diagnosis, which can continuously improve the system's diagnostic accuracy and operating condition adaptability without human intervention.

[0058] Next, combined Figures 2 to 4 This application provides a detailed description of the ship fault diagnosis process in its embodiments.

[0059] like Figure 2 As shown, the ship fault diagnosis system in this embodiment includes a front-end interaction layer, a business logic processing layer, and a data persistence and knowledge governance layer.

[0060] The front-end interaction layer is responsible for receiving multimodal input from users (voice, image, text) and displaying the diagnostic results in a streaming manner with traceable evidence chains; The business logic processing layer, as the core intelligent hub of this system, is responsible for cross-modal feature fusion, graph topology alignment, controlled retrieval scheduling, and local inference of large models. The data persistence and knowledge governance layer integrates a distributed graph database (Neo4j) and a high-performance vector retrieval library (such as ChromaDB), which not only ensures efficient reading and atomic writing of knowledge, but also supports complex path traversal and graph edit distance calculation based on graph theory.

[0061] The system communicates with each layer through a standardized RESTful API interface. It has a built-in offline computing engine that can complete closed-loop inference on the ship's edge computing terminal (such as an industrial control computer) without relying on an external cloud environment, thus ensuring high confidentiality and high stability.

[0062] like Figure 3As shown, the ship knowledge graph designed in this embodiment establishes dual constraints regarding physics and mechanism. The entity library not only includes equipment nodes, subsystem nodes, fault phenomenon nodes, fault root cause nodes, and maintenance solutions, but also assigns operational attributes and timestamps to edge relationships (such as Caused_By and Has_Component). This modeling approach enables logical relationships across systems and components to possess the spatiotemporal state of physical entities, supporting subsequent GED calculations and multi-hop logical reasoning based on sea state environmental factors.

[0063] like Figure 4 As shown, this ship fault diagnosis system achieves ship fault diagnosis by performing the following steps.

[0064] First, multimodal data acquisition and processing.

[0065] For multimodal input, this system not only utilizes the Whisper model for 16kHz speech transcription but also employs object detection networks such as YOLO to extract bounding boxes and location labels for equipment faults. Specifically, the identified location labels and text entities are precisely mapped to a pre-built ship knowledge graph entity library using an entity chaining algorithm. Using these mapped discrete fault entity nodes as anchors, their K-hop neighbors and edges in the graph are extracted to form a fault context subgraph. Subsequently, a graph neural network is invoked to aggregate the structural features of this fault context subgraph. The output graph representation vector is then tensor-concatenated with the semantic embedding vector of the original text, mapping them to the same high-dimensional space to generate a hybrid semantic query vector that eliminates modal conflicts and carries the physical location attributes of the equipment.

[0066] Then, a secondary reconstruction retrieval constrained by graph connectivity is performed.

[0067] This system proposes a topological constraint-enhanced retrieval method. First, a cosine similarity screening is performed in the ship maintenance database using a hybrid semantic query vector to recall a batch of text fragments. Then, a graph constraint review is triggered: the associated nodes corresponding to each initially screened text fragment are extracted, and a random walk algorithm is used to calculate the topological connectivity between the fragment and the currently faulty node. If the two nodes are isolated in the ship knowledge graph or have an excessively high hop count, the system either does not apply a gain to the cosine similarity of the text fragment or applies a penalty; if a direct logical or physical connection exists, a gain is applied. After a second reconstruction and scoring process, candidate text fragments with physical and logical consistency are recalled to filter out semantically similar but mechanistically unrelated illusory interference.

[0068] Next, GED rearrangement and source generation are performed.

[0069] During the re-ranking phase, the system deploys a customized cross-encoder. This encoder not only calculates the cross-attention score between the query and the document but also extracts the causal path subgraph corresponding to the candidate text fragment and calculates the graph edit distance score between it and the fault context subgraph. The joint graph edit distance score and cross-attention score are then sorted in descending order to select the text fragment with the highest matching degree. This text fragment is then injected as context into a large language model (such as DeepSeek or Llama series). While generating diagnostic suggestions, an attention weight backtracking mechanism is used to attach hyperlinks to graph paths or manual page numbers to each conclusion, achieving traceability of diagnostic conclusions.

[0070] In summary, the ship fault diagnosis solution provided in the above embodiments of this application has at least the following advantages: First, this application anchors the image and text features of the target fault to the relevant fault nodes in the ship knowledge graph, and extracts K-hop neighbors to construct a fault context subgraph, transforming discrete multimodal signals into hybrid semantic query vectors carrying equipment topological location attributes, which greatly improves the accuracy of multimodal feature fusion in highly interfering industrial environments.

[0071] Second, in response to the pain point of conventional RAG technology blindly pursuing semantic similarity while ignoring physical mechanisms, this application introduces graph topological connectivity based on random walks and combines it with graph editing distance for secondary rearrangement. This can filter out semantically similar but physically inaccessible interference information, ensuring that the generated diagnostic suggestions are reasonable in terms of engineering mechanisms and accompanied by a traceable graph evidence chain.

[0072] Third, unlike simple frequency statistics, this application can automatically identify the current sea state / operating parameters, assign higher recommendation weights to historical successful experiences that match the current working conditions, and gradually eliminate old experiences that become ineffective as equipment ages, thus realizing the assetization and dynamic evolution of industrial knowledge.

[0073] Figure 5 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 6At the hardware level, the device includes a processor 510, an internal bus 520, a network interface 530, memory 540, a hardware acceleration device 550, and non-volatile memory 560, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 510 reads the corresponding computer program from the non-volatile memory 560 into the memory 540 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0074] Figure 6 This is a structural block diagram illustrating an exemplary embodiment of a ship fault diagnosis device based on topology constraints. The ship fault diagnosis device can be applied to, for example... Figure 5 The electronic device shown implements the technical solution of this application. The ship fault diagnosis device includes: a fault data acquisition unit 610, a query vector construction unit 620, a topology constraint retrieval unit 630, a text reordering unit 640, and a ship fault diagnosis unit 650, wherein: The fault data acquisition unit 610 is used to acquire multimodal fault data of the ship target fault, the multimodal fault data including fault description text, voice fault commands and images of the fault location; The query vector construction unit 620 is used to map the multimodal fault data to a ship knowledge graph that represents the ship's physical topology and fault causal logic using the entity chain index algorithm, to obtain the fault nodes corresponding to the multimodal data, construct a fault context subgraph based on all fault nodes, and construct a hybrid semantic query vector based on the fault context subgraph and the multimodal fault data. The topology constraint retrieval unit 630 is used to retrieve an initial set of maintenance semantic vectors from a pre-constructed ship maintenance database based on the hybrid semantic query vectors. The ship maintenance database stores text fragments of ship maintenance manuals and technical specifications and their corresponding maintenance semantic feature vectors. The entity chain index algorithm is used to map each text fragment to the ship knowledge graph to obtain the associated nodes corresponding to each text fragment, and the topological connectivity from each associated node to each fault node is calculated. Based on the topological connectivity, a candidate set of maintenance semantic vectors is retrieved from the initial set of maintenance semantic vectors. The text reordering unit 640 is configured to construct a corresponding causal path subgraph based on its associated nodes for each text segment in the candidate maintenance semantic vector set, calculate the graph edit distance score between the causal path subgraph and the fault context subgraph; and calculate the cross semantic attention score between the hybrid semantic query vector and each maintenance semantic feature vector in the candidate maintenance semantic vector set, and based on the graph edit distance score and the cross semantic attention score, select reference text segments from the candidate maintenance semantic vector set and obtain the causal logic path corresponding to the reference text segment. The ship fault diagnosis unit 650 is used to perform fault reasoning on the reference text fragment and its corresponding causal logic path using a large language model to obtain fault diagnosis suggestions.

[0075] In some embodiments, the query vector construction unit 620 is used to perform speech recognition on the voice fault command to obtain fault command text; and to perform target detection on the fault location image to obtain fault location label; and to map the fault description text, the fault command text, and the fault location label to the ship knowledge graph through an entity chain indexing algorithm to obtain at least one fault node.

[0076] In some embodiments, the query vector construction unit 620 is used to extract neighbor nodes and relational edges within a K-hop range of each fault node as anchor points in the ship knowledge graph, and construct a fault context subgraph; K is a preset positive integer.

[0077] In some embodiments, the query vector construction unit 620 is used to perform feature aggregation on the fault context subgraph using a graph neural network to obtain a graph representation vector; calculate the semantic embedding vector corresponding to the multimodal fault data; and concatenate the graph representation vector and the semantic embedding vector using a tensor to obtain the hybrid semantic query vector.

[0078] In some embodiments, the topology constraint retrieval unit 630 is configured to obtain the initial maintenance semantic vector set based on the similarity between the hybrid semantic query vector and the maintenance semantic feature vector; calculate the topological connectivity from each associated node to each fault node based on a random walk algorithm, and obtain a causal logical path based on the node sequence traversed during the random walk, wherein the causal logical path includes nodes directly connected to the current fault node in the ship knowledge graph through a preset mechanism association edge; correct the similarity using the topological connectivity score, and obtain the candidate maintenance semantic vector set based on the corrected similarity.

[0079] In some embodiments, the text reordering unit 640 is used to perform weighted fusion of the graph edit distance score and the cross semantic attention score, and use the text segment corresponding to the highest total score after weighted fusion as the reference text segment.

[0080] In some embodiments, the ship fault diagnosis device further includes: The knowledge graph update unit is used to acquire the fault occurrence time interval, the current operating condition characteristics of the ship, and the user's repair confirmation feedback information on the fault diagnosis suggestions; based on the fault occurrence time interval, the current operating condition characteristics of the ship, and the repair confirmation feedback information, it updates the weights of the relation edges of the ship knowledge graph; and writes the updated weights back to the ship knowledge graph so as to perform subsequent ship fault diagnosis based on the updated ship knowledge graph.

[0081] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0082] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0083] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0084] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0085] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0086] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0087] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0088] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0089] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0090] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0091] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0092] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A ship fault diagnosis method based on topology constraints, characterized in that, The method includes the following steps: Step S1: Obtain multimodal fault data of the target ship fault, including fault description text, voice fault commands, and images of the fault location; Step S2: The entity chain index algorithm is used to map the multimodal fault data to the ship knowledge graph used to characterize the ship's physical topology and fault causal logic, to obtain the fault nodes corresponding to the multimodal data. A fault context subgraph is constructed based on all fault nodes, and a hybrid semantic query vector is constructed based on the fault context subgraph and the multimodal fault data. Step S3: Based on the hybrid semantic query vector, retrieve an initial maintenance semantic vector set from the pre-constructed ship maintenance database. The ship maintenance database stores text fragments of ship maintenance manuals and technical specifications and their corresponding maintenance semantic feature vectors. Use the entity chaining algorithm to map each text fragment to the ship knowledge graph to obtain the associated nodes corresponding to each text fragment, and calculate the topological connectivity from each associated node to each fault node. Based on the topological connectivity, retrieve a candidate maintenance semantic vector set from the initial maintenance semantic vector set. Step S4: For each text segment in the candidate maintenance semantic vector set, construct a corresponding causal path subgraph based on its associated nodes, and calculate the graph edit distance score between the causal path subgraph and the fault context subgraph. Furthermore, the cross-semantic attention score between the hybrid semantic query vector and each maintenance semantic feature vector in the candidate maintenance semantic vector set is calculated. Based on the graph editing distance score and the cross-semantic attention score, a reference text fragment is selected from the candidate maintenance semantic vector set, and the causal logic path corresponding to the reference text fragment is obtained. Step S5: Use a large language model to perform fault reasoning on the reference text fragment and its corresponding causal logic path to obtain fault diagnosis suggestions.

2. The method according to claim 1, characterized in that, The faulty node in step S2 is obtained through the following steps: The voice fault command is subjected to speech recognition to obtain the fault command text; and the image of the fault location is subjected to target detection to obtain the fault location label. The fault description text, the fault instruction text, and the fault location label are mapped to the ship knowledge graph using an entity chain indexing algorithm to obtain at least one fault node.

3. The method according to claim 1, characterized in that, The fault context subgraph in step S2 is obtained through the following steps: Using each fault node as an anchor point, extract its neighbor nodes and relational edges within a range of K hops in the ship knowledge graph to construct a fault context subgraph; K is a preset positive integer.

4. The method according to claim 1, characterized in that, The hybrid semantic query vector in step S2 is obtained through the following steps: The fault context subgraph is aggregated using a graph neural network to obtain a graph representation vector. Calculate the semantic embedding vector corresponding to the multimodal fault data; The graph representation vector and the semantic embedding vector are concatenated using tensors to obtain the hybrid semantic query vector.

5. The method according to claim 1, characterized in that, Step S3 includes: Based on the similarity between the hybrid semantic query vector and the maintenance semantic feature vector, the initial maintenance semantic vector set is obtained; The topological connectivity from each associated node to each fault node is calculated based on the random walk algorithm, and the causal logical path is obtained based on the node sequence traversed during the random walk. The causal logical path includes nodes that are directly connected to the current fault node in the ship knowledge graph through a preset mechanism association edge. The similarity is corrected using the topological connectivity score, and the candidate maintenance semantic vector set is obtained based on the corrected similarity.

6. The method according to claim 1, characterized in that, The process of obtaining the reference text fragment in step S4 is as follows: The graph edit distance score and the cross semantic attention score are weighted and fused, and the text segment corresponding to the highest total score after weighted fusion is used as the reference text segment.

7. The method according to any one of claims 1 to 6, characterized in that, It also includes the following step S6: The system acquires the time interval between failures, the current operating condition characteristics of the vessel, and the user's confirmation feedback on the repair recommendations for the failure diagnosis. Based on the time interval between failures, the current operating condition of the ship, and the repair confirmation feedback information, the weights of the relationship edges in the ship knowledge graph are updated. The updated weights are written back to the ship knowledge graph to enable subsequent ship fault diagnosis based on the updated ship knowledge graph.

8. The method according to claim 7, characterized in that, The updated weight expression for the relationship edge between fault phenomenon node i and fault cause node j in the ship knowledge graph is: in, Indicates the first The edge weights after the round update Indicates the first The edge weights of the wheel, This represents the pre-set base weight decay coefficient. This represents the pre-set minimum weight threshold used to ensure the connectivity of the ship knowledge graph. This represents the pre-set feedback learning rate coefficient. This indicates the total number of times the diagnostic path corresponding to this relation edge has been adopted and performed for maintenance throughout history. Indicates the first Confidence score for successful repair. Represents the time decay term. Represents the attenuation constant. Indicates the current fault diagnosis time. Indicates the first The time of occurrence of each historical fault Indicates the similarity factor of working conditions. This represents the feature vector of the ship's current operating condition. Indicates the first The ship's operating condition feature vector at the time of each historical failure.

9. A ship fault diagnosis device based on topology constraints, characterized in that, The device includes: The fault data acquisition unit is used to acquire multimodal fault data of the target ship fault, including fault description text, voice fault commands, and images of the fault location. The query vector construction unit is used to map the multimodal fault data to a ship knowledge graph that represents the ship's physical topology and fault causal logic using the entity chain index algorithm, to obtain the fault nodes corresponding to the multimodal data, construct a fault context subgraph based on all fault nodes, and construct a hybrid semantic query vector based on the fault context subgraph and the multimodal fault data. A topology constraint retrieval unit is used to retrieve an initial set of maintenance semantic vectors from a pre-constructed ship maintenance database based on the hybrid semantic query vectors. The ship maintenance database stores text fragments of ship maintenance manuals and technical specifications and their corresponding maintenance semantic feature vectors. An entity chain indexing algorithm is used to map each text fragment to the ship knowledge graph to obtain the associated nodes corresponding to each text fragment, and the topological connectivity from each associated node to each fault node is calculated. Based on the topological connectivity, a candidate set of maintenance semantic vectors is retrieved from the initial set of maintenance semantic vectors. The text reordering unit is used to construct a corresponding causal path subgraph based on its associated nodes for each text segment in the candidate maintenance semantic vector set, calculate the graph edit distance score between the causal path subgraph and the fault context subgraph; and calculate the cross semantic attention score between the hybrid semantic query vector and each maintenance semantic feature vector in the candidate maintenance semantic vector set, and based on the graph edit distance score and the cross semantic attention score, select reference text segments from the candidate maintenance semantic vector set and obtain the causal logic path corresponding to the reference text segment. The ship fault diagnosis unit is used to perform fault reasoning on the reference text fragment and its corresponding causal logic path using a large language model to obtain fault diagnosis suggestions.

10. An electronic device, characterized in that, include: processor; as well as A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 8.

Images