Ship design domain knowledge retrieval method and device based on large language model

Abstract

The application provides a ship design domain knowledge retrieval method and device based on a large language model, and belongs to the technical field of big data processing.The method comprises the following steps: performing feature extraction on a multi-modal ship data set to obtain a feature set, and constructing a first multi-modal knowledge graph based on the feature set; mapping data in the first multi-modal knowledge graph to an embedding space to obtain embedding vectors, obtaining semantic association relationships based on a similarity matrix between the embedding vectors of different modal data, integrating the semantic association relationships into the first multi-modal knowledge graph to obtain a second multi-modal knowledge graph; fusing the second multi-modal knowledge graph with a preset large language model to obtain a first large language model; adding specific knowledge in the ship design field to the first large language model to obtain a second large language model; and inputting a query sentence of a user into the second large language model to obtain a retrieval result.The application improves the accuracy of the retrieval result of the knowledge in the ship design field.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and in particular to a method and apparatus for knowledge retrieval in the ship design domain based on a large language model. Background Technology

[0002] With the development of artificial intelligence and big data technologies, knowledge retrieval and intelligent question answering in complex fields have gradually become hot topics in technological research.

[0003] In the field of ship design, the design process involves multiple data types and complex semantic relationships, which places high demands on the accuracy and intelligence of knowledge retrieval. However, traditional knowledge retrieval methods are difficult to meet this demand, leading to many difficulties for engineers when obtaining design-related information.

[0004] Currently, knowledge retrieval in ship design primarily relies on traditional keyword matching and rule-based search methods. The core of traditional retrieval methods is to locate relevant documents or entries in a pre-built database using keywords entered by the user. While simple and easy to use, this approach has significant technical limitations. Ship design involves a wide range of complex data types, and these data often contain implicit semantic relationships. Keyword matching cannot understand the deep semantic relationships within these multimodal data, resulting in low accuracy of search results. Summary of the Invention

[0005] This invention provides a method and apparatus for knowledge retrieval in the ship design domain based on a large language model, addressing the shortcomings of low accuracy in existing ship design domain knowledge retrieval results and thereby improving the accuracy of retrieval results. This invention is applicable to the field of ship design, and also to other fields or scenarios requiring precise knowledge retrieval.

[0006] This invention provides a knowledge retrieval method for the ship design domain based on a large language model, comprising:

[0007] Feature extraction is performed on the multimodal ship dataset to obtain a feature set, and a first multimodal knowledge graph is constructed based on the feature set;

[0008] Data in the first multimodal knowledge graph is mapped to the embedding space to obtain embedding vectors. A similarity matrix between embedding vectors of different modal data is calculated. Semantic associations between cross-modal data are obtained based on the similarity matrix. The semantic associations between cross-modal data are integrated into the first multimodal knowledge graph to obtain a second multimodal knowledge graph.

[0009] The second multimodal knowledge graph is fused with the preset large language model to obtain the first large language model;

[0010] By incorporating proprietary knowledge in the field of ship design into the first language model, a second language model is obtained. This proprietary knowledge includes proprietary terminology, commonly used expressions, and logical rules.

[0011] The user's query is input into the second language model to obtain the search results.

[0012] According to the present invention, a knowledge retrieval method for the ship design domain based on a large language model is provided. The multimodal ship dataset includes multiple types such as text datasets, image datasets, CAD model datasets, and 3D simulation datasets. Feature extraction is performed on the multimodal ship dataset to obtain a feature set, including:

[0013] Each text data in the text dataset is mapped to a semantic feature vector to obtain a text feature set;

[0014] Extract the image feature vector from each image data in the image dataset to obtain the image feature set;

[0015] Extract the geometric feature vector of each CAD model data in the CAD model dataset to obtain the CAD model feature set;

[0016] The simulation feature vector of each 3D simulation data in the 3D simulation dataset is extracted to obtain the simulation feature set.

[0017] According to the present invention, a knowledge retrieval method for the ship design domain based on a large language model is provided, which constructs a first multimodal knowledge graph based on the feature set, including:

[0018] The feature set is mapped to a structured knowledge node set, where each knowledge node represents a concept, component, material property, design standard, or performance index in ship design.

[0019] An adaptive graph evolution algorithm is used to dynamically semantically associate the knowledge nodes to obtain a set of association relationships;

[0020] Based on the set of structured knowledge nodes and the set of relationships, a first-level multimodal knowledge graph is constructed.

[0021] According to the present invention, a knowledge retrieval method for the ship design domain based on a large language model is provided. This method incorporates proprietary knowledge of the ship design domain into a first large language model to obtain a second large language model, including:

[0022] Map the proprietary knowledge into an embedding vector;

[0023] The first language model is updated based on the embedded vectors of the proprietary knowledge to obtain the second language model.

[0024] According to the present invention, a knowledge retrieval method for the ship design domain based on a large language model is provided, in which the user's query statement is input into a second large language model to obtain retrieval results, including:

[0025] The query statement is converted into a query intent vector using the large language model in the second large language model.

[0026] The search results are generated based on the query intent vector using the multimodal knowledge graph in the second language model.

[0027] According to the knowledge retrieval method for the ship design domain based on a large language model provided by the present invention, after inputting the user's query into the second large language model and obtaining the retrieval results, the method further includes:

[0028] The search results are then categorized using the second major language model.

[0029] According to the present invention, a knowledge retrieval method for the ship design domain based on a large language model is provided, which classifies the retrieval results using a second large language model, including:

[0030] The query statement is converted into a query intent vector and a set of key semantic elements using the large language model in the second large language model.

[0031] Calculate the semantic similarity between the query intent vector and the embedding vectors of each knowledge node in the multimodal knowledge graph of the second large language model;

[0032] The credibility score of the set of key semantic elements is based on a text filter.

[0033] Based on the semantic similarity and the credibility score, the classification result of the search results is determined.

[0034] This invention also provides a ship design domain knowledge retrieval based on a large language model, including:

[0035] The first multimodal knowledge graph construction module is used to extract features from the multimodal ship dataset to obtain a feature set, and to construct the first multimodal knowledge graph based on the feature set.

[0036] The second multimodal knowledge graph construction module is used to map the data in the first multimodal knowledge graph to the embedding space to obtain embedding vectors, calculate the similarity matrix between the embedding vectors of different modal data, obtain the semantic association relationship between cross-modal data based on the similarity matrix, and integrate the semantic association relationship between cross-modal data into the first multimodal knowledge graph to obtain the second multimodal knowledge graph.

[0037] The first large language model acquisition module is used to fuse the second multimodal knowledge graph with the preset large language model to obtain the first large language model;

[0038] The second language model acquisition module is used to add proprietary knowledge in the field of ship design to the first language model to obtain the second language model. The proprietary knowledge includes proprietary terminology, common expressions and logical rules.

[0039] The search result acquisition module is used to input the user's query into the second language model to obtain the search results.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the ship design domain knowledge retrieval method based on any of the above-described methods.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the ship design domain knowledge retrieval method based on a large language model as described above.

[0042] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the ship design domain knowledge retrieval method based on a large language model as described above.

[0043] This invention provides a method and apparatus for knowledge retrieval in the ship design domain based on a large language model. It constructs a first multimodal knowledge graph using feature sets from a multimodal ship dataset. A second multimodal knowledge graph is obtained based on the embedding vectors of data in the first multimodal knowledge graph and the first multimodal knowledge graph itself. The multimodal knowledge graph effectively captures deep semantic relationships between different modalities. The second multimodal knowledge graph is then fused with a pre-defined large language model to obtain a first large language model. Specific vocabulary from the ship design domain is added to the first large language model to obtain the second large language model. By introducing semantic embeddings of domain-specific knowledge and combining it with the hierarchical graph structure of domain knowledge, the system can deeply analyze user queries and accurately capture design-related intentions. Finally, the user's query is input into the second large language model to obtain the retrieval results. This invention enhances the model's understanding of ship design-specific terminology and complex semantics by fusing the multimodal knowledge graph with the large language model, thereby improving the accuracy of knowledge retrieval results in the ship design domain. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is one of the flowcharts of the ship design domain knowledge retrieval method based on a large language model provided by the present invention;

[0046] Figure 2 This is the second flowchart of the knowledge retrieval method for ship design domain based on a large language model provided by the present invention;

[0047] Figure 3 This is a schematic diagram of the structure of the ship design domain knowledge retrieval device based on a large language model provided by the present invention;

[0048] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] The following is combined Figure 1 The present invention describes a knowledge retrieval method for the ship design domain based on a large language model, comprising:

[0051] Step 101: Extract features from the multimodal ship dataset to obtain a feature set, and construct a first multimodal knowledge graph based on the feature set;

[0052] Step 102: Map the data in the first multimodal knowledge graph to the embedding space to obtain embedding vectors, calculate the similarity matrix between the embedding vectors of different modal data, obtain the semantic association between cross-modal data based on the similarity matrix, and integrate the semantic association between cross-modal data into the first multimodal knowledge graph to obtain the second multimodal knowledge graph;

[0053] Step 103: The second multimodal knowledge graph is fused with the preset large language model to obtain the first large language model;

[0054] Step 104: Add proprietary knowledge in the field of ship design to the first large language model to obtain the second large language model. The proprietary knowledge includes proprietary terminology, common expressions and logical rules.

[0055] Step 105: Input the user's query into the second large language model to obtain the search results.

[0056] First, a ship dataset is constructed and preprocessed to obtain the target dataset. Then, features are extracted from the target dataset to obtain a feature set.

[0057] The preset large language model is a pre-trained language model specifically for ship design, a domain-enhanced large language model, or other large models with similar functions.

[0058] This embodiment constructs a first multimodal knowledge graph using the feature set of a multimodal ship dataset. Based on the embedding vectors of the data in the first multimodal knowledge graph, a second multimodal knowledge graph is obtained. The multimodal knowledge graph effectively captures deep semantic relationships between different modalities. The second multimodal knowledge graph is then fused with a pre-defined large language model to obtain a first large language model. Specific vocabulary from the ship design field is added to the first large language model to obtain the second large language model. By introducing semantic embeddings of domain-specific knowledge and combining it with the hierarchical graph structure of domain knowledge, the system can deeply analyze user queries and accurately capture design-related intentions. Finally, the user's query is input into the second large language model to obtain search results. This embodiment enhances the model's understanding of ship design-specific terminology and complex semantics by fusing the multimodal knowledge graph with the large language model, thereby improving the accuracy of knowledge retrieval results in the ship design field.

[0059] Based on the above embodiments, the multimodal ship dataset in this embodiment includes multiple types such as text datasets, image datasets, CAD model datasets, and 3D simulation datasets. Feature extraction is performed on the multimodal ship dataset to obtain a feature set, including:

[0060] Each text data in the text dataset is mapped to a semantic feature vector to obtain a text feature set;

[0061] Extract the image feature vector from each image data in the image dataset to obtain the image feature set;

[0062] Extract the geometric feature vector of each CAD model data in the CAD model dataset to obtain the CAD model feature set;

[0063] The simulation feature vector of each 3D simulation data in the 3D simulation dataset is extracted to obtain the simulation feature set.

[0064] Based on the above embodiments, this embodiment constructs a first multimodal knowledge graph based on the feature set, including:

[0065] The feature set is mapped to a structured knowledge node set, where each knowledge node represents a concept, component, material property, design standard, or performance index in ship design.

[0066] An adaptive graph evolution algorithm is used to dynamically semantically associate the knowledge nodes to obtain a set of association relationships;

[0067] Based on the set of structured knowledge nodes and the set of relationships, a first-level multimodal knowledge graph is constructed.

[0068] Based on the above embodiments, the proprietary knowledge mentioned in this embodiment includes proprietary terminology, commonly used expressions, and logical rules. Proprietary knowledge in the field of ship design is added to the first large language model to obtain a second large language model, including:

[0069] Map the proprietary knowledge into an embedding vector;

[0070] The first language model is updated based on the embedded vectors of the proprietary knowledge to obtain the second language model.

[0071] Based on the above embodiments, this embodiment inputs the user's query statement into the second large language model to obtain search results, including:

[0072] The query statement is converted into a query intent vector using the large language model in the second large language model.

[0073] The search results are generated based on the query intent vector using the multimodal knowledge graph in the second language model.

[0074] Based on the above embodiments, this embodiment, after inputting the user's query into the second large language model and obtaining the search results, further includes:

[0075] The search results are then categorized using the second major language model.

[0076] Based on the above embodiments, this embodiment utilizes the second major language model to classify the search results, including:

[0077] The query statement is converted into a query intent vector and a set of key semantic elements using the large language model in the second large language model.

[0078] Calculate the semantic similarity between the query intent vector and the embedding vectors of each knowledge node in the multimodal knowledge graph of the second large language model;

[0079] The credibility score of the set of key semantic elements is based on a text filter.

[0080] Based on the semantic similarity and the credibility score, the classification result of the search results is determined.

[0081] The classification results refer to the domain category labeling of knowledge nodes that match the user's query intent, in order to more accurately limit the professional direction of the query target. The categories are divided according to common knowledge modules in the field of ship design, mainly including but not limited to the following categories: structural design, electrical systems, ship specifications and layout design.

[0082] A classification strategy based on multi-factor fusion is adopted, and the classification result is determined by combining semantic similarity and credibility scores through the following steps:

[0083] (1) Calculate semantic similarity:

[0084] The second largest language model is used to embed vectors into query statements and knowledge nodes, and the query intent vector is denoted as v. q The knowledge node embedding vector is denoted as v. k The semantic similarity is obtained by calculating the cosine similarity between the two, that is:

[0085]

[0086] (2) Extract credibility score:

[0087] Let S be the set of key semantic elements in the query. q The set of knowledge node texts or attribute keywords is denoted as S. k The credibility score (Conf) is calculated based on the element matching ratio. The specific calculation formula is as follows:

[0088]

[0089] (3) Enhanced feature similarity:

[0090] To enhance the model's creativity, a feature representation layer can be introduced to extract feature vectors (such as context encoding, semantic labels, etc.) from the query intent vector and the embedding vectors of knowledge nodes, and to calculate the feature similarity Simfeat, or cosine similarity.

[0091]

[0092] (4) Combined calculation of classification score:

[0093] The three similarity scores can be weighted and fused into a comprehensive classification score, calculated as follows:

[0094]

[0095]

[0096] Where α, β and γ are weighting coefficients.

[0097] (5) Threshold determination to determine the classification result:

[0098] If the score is greater than the set threshold, the knowledge node is determined to be a high match with the query, and the knowledge category to which the knowledge node belongs is used as the final classification result of the query.

[0099] like Figure 2 As shown, the specific steps of the knowledge retrieval method in the field of ship design are as follows:

[0100] Step 1: Intelligent Acquisition and Standardization Processing of Heterogeneous Ship Design Data. This step automatically collects original ship design datasets from multi-source heterogeneous data in the ship design field (including design manuals, 2D / 3D CAD drawings, engineering case documents, simulation results data, etc.). Using intelligent format recognition algorithms and an adaptive standardization mapping engine, it automatically converts heterogeneous data formats to a unified representation, generating a standardized design dataset with semantic tags, thus improving data fusion efficiency and accuracy.

[0101] Step 2: Cross-modal deep semantic parsing and knowledge element construction. Based on the multimodal Transformer model, cross-modal deep semantic parsing is performed on standardized design data to automatically generate atomic-level knowledge elements (the smallest semantic units). Each knowledge element represents concepts, components, material properties, design standards, performance indicators, etc., in ship design. By introducing an adaptive graph evolution algorithm, dynamic semantic association discovery between knowledge elements is achieved. Based on the professional knowledge system in the field of ship design, a cross-modal semantic association network is established to form a multimodal hierarchical knowledge graph with self-evolution capabilities.

[0102] Step 3: Multimodal Collaborative Embedding Representation and Self-Supervised Optimization. A multimodal collaborative embedding mechanism is adopted to uniformly map image features, text features, and graph structure information to a shared semantic space. Self-supervised contrastive learning is introduced to optimize the embedding representation. By designing a task-driven positive and negative sample comparison mechanism, the semantic coupling and feature discrimination capabilities between different modalities are further enhanced, significantly improving the semantic association quality between multimodal data.

[0103] Step 4: Fusion of the Pre-trained Language Model Dedicated to Ship Design and Graph-Driven Semantic Reasoning. This step deeply integrates a hierarchical multimodal knowledge graph into the pre-trained language model dedicated to ship design, constructing a domain-enhanced pre-trained language model for ship design. Based on the pre-training, a terminology system, common expressions, and logical rules specific to the ship design domain are introduced to form a knowledge graph-driven semantic reasoning engine.

[0104] Step 5: Multi-level parsing of user intent and dynamic modeling of context. Based on the hierarchical intent parsing framework, combined with the ability to generate large language models and graph reasoning, the user's natural language query is decomposed into multiple granular semantics. The core design intent, related sub-requirements and contextual historical intent chain are automatically extracted to realize the dynamic modeling and evolution of the intent of continuous design tasks in complex design scenarios.

[0105] Step Six: Cross-modal joint reasoning and knowledge graph adaptive retrieval. A cross-modal joint reasoning algorithm is adopted, which introduces a multi-modal joint similarity measurement mechanism on the basis of the semantic relationship of the knowledge graph. Through the dual mechanism of graph structure reasoning and embedded space semantic matching, the design knowledge related to the user's query intent is accurately located.

[0106] Step Seven: Multi-dimensional View Feedback and Enhanced Knowledge Visualization. The retrieved ship design knowledge is presented in a multi-dimensional view format, including: a structured text summary view; a three-tiered relationship diagram of parameters, standards, and cases; a time-series evolution view (historical evolution); and a design risk warning view. An interactive knowledge visualization interface is also introduced, allowing users to filter, reorganize, and ask further questions about the results from multiple perspectives, enabling proactive knowledge recommendation and intelligent navigation during the design decision-making process.

[0107] To verify the feasibility and effectiveness of this invention in ship design knowledge retrieval, the following detailed explanation of the operational process and comparative results of a ship design knowledge retrieval method based on a large language model in practical application is presented through a specific scenario.

[0108] At a large ship design institute, engineer Zhang is responsible for the preliminary design of the electrical system of a near-shore multipurpose cargo ship. Because this ship type is a non-standardized, custom design, its power distribution system needs to meet specific requirements for crane equipment and cargo hold layout. Zhang needs to quickly search for design schemes, technical specifications, and failure cases of similar ship types to assist in the current design. Traditional retrieval methods involve data scattered across multiple document databases and design drawing systems, requiring manual screening. Complex design logic is difficult to express with keywords, leading to information omissions, incomplete design references, and increased design risks.

[0109] To address the aforementioned issues, this study proposes a ship design knowledge retrieval method based on a large language model. The application process of this method is illustrated below using a real-world work scenario as an example.

[0110] Mr. Zhang entered his search request in natural language into the search interface of the design support system: "Offshore transport vessel, equipped with a heavy-duty crane, rated lifting capacity 150 tons, main cable wiring design reference." The system, based on its built-in large language model (such as a large-scale Transformer pre-trained model), performed semantic parsing on the input statement, automatically identifying and extracting the following key information in a structured manner: Ship type: "Offshore transport vessel"; Functional requirements: "Heavy-duty crane"; "Rated lifting capacity 150 tons"; Search object: "Main cable wiring design".

[0111] The system automatically accesses its internally built ship design knowledge graph. This knowledge graph covers electrical system design documents, structured CAD drawings, equipment parameters, and failure cases from past design projects. Through entity matching and relational reasoning, the system identifies three relevant design cases, each corresponding to different tonnages and wiring strategies, providing a reference for current design requirements.

[0112] The system further utilizes a large language model to intelligently generate summaries from multiple relevant documents, extracting the core design principles and considerations regarding "main cable cabling." An example of the generated summary is as follows: "Referring to Case A, the main cable is laid independently in compartments to avoid interference with the power supply circuit during lifting operations. The peak power requirement of the lifting equipment is 380kW; redundant lines are recommended."

[0113] After obtaining preliminary results, Mr. Zhang can raise new questions as needed. For example, "Were there any electrical faults in these cases?" The system uses the "Design → Accident Cases → Cause Analysis" relationship chain in the knowledge graph to automatically retrieve relevant accident records and generate corresponding summaries to provide feedback to the user.

[0114] Ultimately, based on the design guidelines and risk warnings returned by the system, Zhang developed a main cable wiring plan that met the requirements and was verified, providing reliable support for subsequent design work.

[0115] In subsequent project review meetings, the project team confirmed that the design scheme had sufficient reference materials and a reasonable logic, greatly reducing the time cost of manual data retrieval. Furthermore, by conducting correlation searches of historical accident cases, potential design risks were effectively avoided, significantly improving the safety and reliability of the design.

[0116] To verify the effectiveness of the present invention, a comparative experiment was conducted in a real design scenario. The results of the comparative experiment in knowledge retrieval are shown in Table 1 below.

[0117] Table 1. Comparative Experiment Results of Ship Design Knowledge Retrieval

[0118]

[0119] The experimental results show that the method of this invention significantly outperforms traditional methods in both query comprehension accuracy and relevant result coverage, achieving 95.3% and 93.7% respectively. This effectively addresses the problems of insufficient understanding of complex design requirements and incomplete result coverage in traditional keyword retrieval. Furthermore, the method significantly reduces the interference rate of irrelevant information through semantic understanding and graph reasoning, decreasing it from 41.3% to 8.5%. In terms of retrieval efficiency, the method relies on semantic parsing and structured graph matching to reduce the average retrieval time from 76 seconds in traditional methods to 19 seconds, greatly improving retrieval efficiency. In addition, user feedback ratings show that the method of this invention achieved a user satisfaction score of 4.7 out of 5, compared to 2.8 for traditional methods, demonstrating high acceptance in practical use.

[0120] The experimental results above demonstrate that the present invention significantly improves the efficiency and accuracy of knowledge retrieval in complex ship design scenarios, verifies the system's advantages in multimodal knowledge fusion, professional semantic parsing, and intelligent feedback, provides engineers with efficient and reliable technical support, and greatly improves work efficiency and design quality.

[0121] The following describes the ship design domain knowledge retrieval device based on a large language model provided by the present invention. The ship design domain knowledge retrieval device based on a large language model described below and the ship design domain knowledge retrieval method based on a large language model described above can be referred to and corresponded to each other.

[0122] like Figure 3 As shown, the device includes a first multimodal knowledge graph construction module 301, a second multimodal knowledge graph construction module 302, a first large language model acquisition module 303, a second large language model acquisition module 304, and a retrieval result acquisition module 305, wherein:

[0123] The first multimodal knowledge graph construction module 301 is used to extract features from the multimodal ship dataset to obtain a feature set, and to construct the first multimodal knowledge graph based on the feature set;

[0124] The second multimodal knowledge graph construction module 302 is used to map the data in the first multimodal knowledge graph to the embedding space to obtain embedding vectors, calculate the similarity matrix between the embedding vectors of different modal data, obtain the semantic association relationship between cross-modal data based on the similarity matrix, and integrate the semantic association relationship between cross-modal data into the first multimodal knowledge graph to obtain the second multimodal knowledge graph.

[0125] The first large language model acquisition module 303 is used to fuse the second multimodal knowledge graph with the preset large language model to obtain the first large language model;

[0126] The second language model acquisition module 304 is used to add proprietary knowledge in the field of ship design to the first language model to obtain the second language model. The proprietary knowledge includes proprietary terms, common expressions and logical rules.

[0127] The search result acquisition module 305 is used to input the user's query statement into the second large language model to obtain the search results.

[0128] This embodiment constructs a first multimodal knowledge graph using the feature set of a multimodal ship dataset. Based on the embedding vectors of the data in the first multimodal knowledge graph, a second multimodal knowledge graph is obtained. The multimodal knowledge graph effectively captures deep semantic relationships between different modalities. The second multimodal knowledge graph is then fused with a pre-defined large language model to obtain a first large language model. Specific vocabulary from the ship design field is added to the first large language model to obtain the second large language model. By introducing semantic embeddings of domain-specific knowledge and combining it with the hierarchical graph structure of domain knowledge, the system can deeply analyze user queries and accurately capture design-related intentions. Finally, the user's query is input into the second large language model to obtain search results. This embodiment enhances the model's understanding of ship design-specific terminology and complex semantics by fusing the multimodal knowledge graph with the large language model, thereby improving the accuracy of knowledge retrieval results in the ship design field.

[0129] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a ship design domain knowledge retrieval method based on a large language model. This method includes: extracting features from a multimodal ship dataset to obtain a feature set; constructing a first multimodal knowledge graph based on the feature set; mapping data in the first multimodal knowledge graph to an embedding space to obtain embedding vectors; calculating a similarity matrix between embedding vectors of different modalities; obtaining semantic associations between cross-modal data based on the similarity matrix; integrating the semantic associations between cross-modal data into the first multimodal knowledge graph to obtain a second multimodal knowledge graph; fusing the second multimodal knowledge graph with a preset large language model to obtain a first large language model; adding ship design domain-specific knowledge to the first large language model to obtain a second large language model, wherein the specific knowledge includes specialized terminology, common expressions, and logical rules; and inputting the user's query into the second large language model to obtain retrieval results.

[0130] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0131] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the ship design domain knowledge retrieval method based on a large language model provided by the above methods. The method includes: extracting features from a multimodal ship dataset to obtain a feature set; constructing a first multimodal knowledge graph based on the feature set; mapping data in the first multimodal knowledge graph to an embedding space to obtain embedding vectors; calculating a similarity matrix between embedding vectors of different modal data; obtaining semantic associations between cross-modal data based on the similarity matrix; integrating the semantic associations between cross-modal data into the first multimodal knowledge graph to obtain a second multimodal knowledge graph; fusing the second multimodal knowledge graph with a preset large language model to obtain a first large language model; adding proprietary knowledge in the ship design domain to the first large language model to obtain a second large language model, wherein the proprietary knowledge includes proprietary terminology, common expressions, and logical rules; and inputting the user's query into the second large language model to obtain retrieval results.

[0132] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the ship design domain knowledge retrieval method based on a large language model provided by the above methods. The method includes: extracting features from a multimodal ship dataset to obtain a feature set; constructing a first multimodal knowledge graph based on the feature set; mapping data in the first multimodal knowledge graph to an embedding space to obtain embedding vectors; calculating a similarity matrix between embedding vectors of different modal data; obtaining semantic associations between cross-modal data based on the similarity matrix; integrating the semantic associations between cross-modal data into the first multimodal knowledge graph to obtain a second multimodal knowledge graph; fusing the second multimodal knowledge graph with a preset large language model to obtain a first large language model; adding proprietary knowledge in the ship design domain to the first large language model to obtain a second large language model, wherein the proprietary knowledge includes proprietary terminology, common expressions, and logical rules; and inputting a user's query into the second large language model to obtain retrieval results.

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A knowledge retrieval method for the ship design domain based on a large language model, characterized in that, include: Feature extraction is performed on the multimodal ship dataset to obtain a feature set, and a first multimodal knowledge graph is constructed based on the feature set; Data in the first multimodal knowledge graph is mapped to the embedding space to obtain embedding vectors. A similarity matrix between embedding vectors of different modal data is calculated. Semantic associations between cross-modal data are obtained based on the similarity matrix. The semantic associations between cross-modal data are integrated into the first multimodal knowledge graph to obtain a second multimodal knowledge graph. The second multimodal knowledge graph is fused with the preset large language model to obtain the first large language model; By incorporating proprietary knowledge in the field of ship design into the first language model, a second language model is obtained. This proprietary knowledge includes proprietary terminology, commonly used expressions, and logical rules. The user's query is input into the second language model to obtain the search results; After inputting the user's query into the second largest language model and obtaining the search results, the process also includes: The search results are then categorized using the second major language model. The search results are classified using the second major language model, including: The query statement is converted into a query intent vector and a set of key semantic elements using the large language model in the second large language model. Calculate the semantic similarity between the query intent vector and the embedding vectors of each knowledge node in the multimodal knowledge graph of the second large language model; The credibility score of the set of key semantic elements is based on a text filter. Based on the semantic similarity and the credibility score, the classification result of the search results is determined; The three similarity scores are weighted and fused into a comprehensive classification score, calculated as follows: ; ; Where α, β, and γ are weighting coefficients. To query the semantic similarity between the intent vector and the knowledge node embedding vector, To score credibility, The feature similarity between feature vectors extracted from the query intent vector and the embedding vector of knowledge nodes; If the score is greater than the set threshold, the knowledge node is determined to be a high match with the query, and the knowledge category to which the knowledge node belongs is used as the final classification result of the query.

2. The knowledge retrieval method for ship design domain based on a large language model according to claim 1, characterized in that, The multimodal ship dataset includes various types of datasets such as text datasets, image datasets, CAD model datasets, and 3D simulation datasets. Feature extraction is performed on the multimodal ship dataset to obtain a feature set, including: Each text data in the text dataset is mapped to a semantic feature vector to obtain a text feature set; Extract the image feature vector from each image data in the image dataset to obtain the image feature set; Extract the geometric feature vector of each CAD model data in the CAD model dataset to obtain the CAD model feature set; The simulation feature vector of each 3D simulation data in the 3D simulation dataset is extracted to obtain the simulation feature set.

3. The knowledge retrieval method for ship design domain based on a large language model according to claim 1, characterized in that, Constructing a first multimodal knowledge graph based on the aforementioned feature set includes: The feature set is mapped to a structured knowledge node set, where each knowledge node represents a concept, component, material property, design standard, or performance index in ship design. An adaptive graph evolution algorithm is used to dynamically semantically associate the knowledge nodes to obtain a set of association relationships; Based on the set of structured knowledge nodes and the set of relationships, a first-level multimodal knowledge graph is constructed.

4. The knowledge retrieval method for ship design domain based on a large language model according to claim 1, characterized in that, By incorporating ship design-specific knowledge into the first major language model, a second major language model is obtained, which includes: Map the proprietary knowledge into an embedding vector; The first language model is updated based on the embedded vectors of the proprietary knowledge to obtain the second language model.

5. The knowledge retrieval method for ship design domain based on a large language model according to any one of claims 1-4, characterized in that, The user's query is input into the second largest language model to obtain the search results, including: The query statement is converted into a query intent vector using the large language model in the second large language model. The search results are generated based on the query intent vector using the multimodal knowledge graph in the second language model.

6. A knowledge retrieval device for the ship design domain based on a large language model, characterized in that, The method for knowledge retrieval in the ship design domain based on a large language model, as described in any one of claims 1-5, includes: The first multimodal knowledge graph construction module is used to extract features from the multimodal ship dataset to obtain a feature set, and to construct the first multimodal knowledge graph based on the feature set. The second multimodal knowledge graph construction module is used to map the data in the first multimodal knowledge graph to the embedding space to obtain embedding vectors, calculate the similarity matrix between the embedding vectors of different modal data, obtain the semantic association relationship between cross-modal data based on the similarity matrix, and integrate the semantic association relationship between cross-modal data into the first multimodal knowledge graph to obtain the second multimodal knowledge graph. The first large language model acquisition module is used to fuse the second multimodal knowledge graph with the preset large language model to obtain the first large language model; The second language model acquisition module is used to add proprietary knowledge in the field of ship design to the first language model to obtain the second language model. The proprietary knowledge includes proprietary terminology, common expressions and logical rules. The search result acquisition module is used to input the user's query into the second language model to obtain the search results.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the ship design domain knowledge retrieval method based on a large language model as described in any one of claims 1 to 5.

Images