Construction method of ship owner opinion reply intelligent agent for ship design field

Abstract

The invention belongs to the field of ship design, and provides a construction method of a ship owner opinion reply intelligent agent oriented to the field of ship design. Analyzing the historical project document through an OCR technology and a multi-modal model to obtain an editable text; the ship owner opinion reply knowledge base systematically archives historical project documents to form a traceable ship owner opinion reply knowledge base; and meanwhile, a previous solution for retrieving similar questions in the ship owner opinion reply knowledge base through three indexes is supported, and compared with manual browsing of historical project documents, the retrieval speed and the accuracy of recalled texts are improved. According to the method, text fragments are recalled from a constructed ship owner opinion reply knowledge base through opinions put forward by a ship owner, final cue words are constructed according to the recalled text fragments, the final cue words can effectively guide a large language model to focus attention to the field where questions input by a user are located, and the generalization answer tendency of the model is avoided.

Description

Technical Field

[0001] This invention belongs to the field of ship design technology, and in particular relates to a method for constructing an intelligent agent for responding to shipowner opinions in the field of ship design. Background Technology

[0002] During the ship design process, shipowners often submit personalized design suggestions based on standard ship design schemes. After receiving these suggestions, the design institute distributes them to designers according to their professional categories for evaluation and feedback. Designers, in responding to these suggestions, frequently need to refer to past project feedback documents. These historical feedback documents are mostly scanned copies or PDF files, requiring manual review, resulting in slow searches, low efficiency, and a long response time to shipowner feedback.

[0003] Therefore, when responding to shipowners' comments on standard ship design proposals, designers generally rely on manually reviewing historical designs. However, manually reviewing designs has the following problems:

[0004] 1) Since most historical communication records are scanned documents, it is difficult for manual personnel to perform traditional keyword searches within individual files;

[0005] 2) There are multiple files in the historical communication records, and it is not clear to humans which file contains the answer record for a specific similar question, making it difficult to locate and search among multiple files;

[0006] 3) Some similar questions exist in both Chinese and English, and may be similar questions, but it is quite difficult to manually search for semantically similar questions;

[0007] 4) Manual review is inevitably prone to omissions or errors. Summary of the Invention

[0008] To address the problems in existing technologies where historical communication records are scanned documents, resulting in slow query times, low efficiency, and long response cycles when manually reviewing historical plans to respond to shipowners' opinions, as well as the potential for omissions or errors in manual responses leading to low accuracy and reliability, a method for constructing an intelligent agent for responding to shipowners' opinions in the field of ship design is proposed, thus solving the aforementioned problems.

[0009] A method for constructing a shipowner's opinion response agent in the field of ship design, comprising the following steps:

[0010] S1: Obtain and parse historical project documents: Obtain scanned copies of historical project documents related to ship design; parse and integrate different types of data in the historical project documents to obtain editable text of the historical project documents;

[0011] S2: Semantic segmentation and boundary optimization: Semantically segment the editable text to obtain coarse-grained text blocks; analyze the semantic continuity in each coarse-grained text block and optimize the boundary of semantic segmentation; then segment the coarse-grained text blocks according to the optimized segmentation boundary to obtain text block metadata;

[0012] S3: Constructing a text block index: The text block index includes a first keyword index, a first sparse vector index, and a first dense vector index; the text block metadata is processed based on the BERT model, the term frequency-inverse document frequency algorithm, and the context attention weight to construct the first keyword index; then the text block metadata is vectorized to obtain the first sparse vector index and the first dense vector index;

[0013] S4: Construct a knowledge base for shipowner feedback responses: Construct a knowledge base for shipowner feedback responses by combining editable text, text chunk attributes, text chunk metadata, and text chunk indexes;

[0014] S5: Shipowner's Opinion Response: Construct a shipowner's opinion index corresponding to the text block index; fuse and match the text block metadata in the shipowner's opinion response knowledge base through the shipowner's opinion index to obtain a recalled text fragment; use the recalled text fragment as the context of the shipowner's opinion to obtain shipowner's opinion prompt words; input the shipowner's opinion prompt words into a large language model to obtain a solution and source link for the shipowner's opinion, which serve as the response of the shipowner's opinion response agent.

[0015] Preferably, the method for parsing and integrating different types of data in the historical project documents as described in step S1 is as follows:

[0016] S11: Extract plain text data from scanned historical project documents using optical character recognition (OCR) technology to obtain text information;

[0017] S12 identifies composite data in the scanned historical project documents using a multimodal model to obtain composite information; the text information and composite information are in Markdown format; the composite data includes: formulas, tables, and images; the multimodal model uses the open-source Shusheng Wanxiang model;

[0018] S13: Integrate the text information and the composite information according to the document structure of the scanned historical project document to obtain the editable text of the historical project document.

[0019] Preferably, the method for obtaining text block metadata in step S2 is as follows:

[0020] S21: Semantically segment the editable text according to a specific format to obtain coarse-grained text blocks; the specific format includes, but is not limited to: Markdown headings, question-answer pairs;

[0021] S22: Use the sliding window algorithm or sentence vector clustering algorithm to analyze the semantic continuity in each coarse-grained text block, and determine the boundary for optimizing semantic segmentation based on semantic continuity;

[0022] S23: Segment the coarse-grained text block according to the optimized segmentation boundary to obtain text block metadata.

[0023] Preferably, the method for obtaining the first keyword index is as follows:

[0024] S31: Entity recognition is performed on the text block metadata using the BERT model to obtain entity words;

[0025] S32: Extract keywords from the text block metadata using the BERT model to obtain keywords; the keywords and entity words constitute global keywords; the BERT models in S31 and S32 are both BERT models fine-tuned using the shipbuilding engineering corpus;

[0026] S33: The global keywords are vectorized using the term frequency-inverse document frequency algorithm (TF-IDF algorithm) and contextual attention weights to obtain a weighted keyword vector, which is the first keyword index.

[0027] Preferably, the method for obtaining the weighted keyword vector in step S33 is as follows:

[0028] A1: The global importance score of global keywords in the historical project document set is quantified by the term frequency-inverse document frequency algorithm; the importance score is the initial weight.

[0029] A2: Input a specific context into the pre-trained language model; calculate the semantic importance, i.e., the attention weight, of the global keyword in the specific context of the input through the self-attention mechanism of the pre-trained language model; the specific context is the specific sentence in which the global keyword is located in the text block metadata;

[0030] A3: Multiply the initial weight and the attention weight to obtain the comprehensive weight coefficient of the global keyword;

[0031] A4: Multiply the comprehensive weight coefficient by the global keyword vector to obtain the weighted keyword vector; the global keyword vector is the output after the global keywords are input into the model.

[0032] Preferably, the method for obtaining the sparse vector index and the dense vector index is as follows:

[0033] B1: By using word frequency statistics, the text block metadata is mapped to a high-dimensional sparse vector, i.e., the first sparse vector index.

[0034] B2: By using an embedding model, text chunk metadata is mapped to a low-dimensional dense vector, i.e., the first dense vector index.

[0035] Preferably, the method for responding to the shipowner's comments in step S5 is as follows:

[0036] S51: Constructing an index of shipowner opinions: The index of shipowner opinions includes: a second keyword index, a second sparse vector index, and a second dense vector index; global keywords of shipowner opinions are extracted using the BERT model; the global keywords are vectorized using the TF-IDF algorithm and contextual attention weights to obtain the second keyword index; the second sparse vector index and the second dense vector index of the shipowner opinions are obtained by vectorizing the shipowner opinions.

[0037] S52: Retrieval of the Shipowner's Opinion Response Knowledge Base: By fusing and matching the text block metadata in the Shipowner's Opinion Response Knowledge Base with the index of the shipowner's opinion, the text block metadata recalled from the Shipowner's Opinion Response Knowledge Base and the source link of the text block metadata are obtained.

[0038] S53: Reordering of Recalled Text Fragments: Concatenate the shipowner's comments and the recalled text fragment metadata to obtain a concatenated vector; input the concatenated vector into the Cross-Encoder ranking model to calculate semantic similarity; reorder all recalled text fragment metadata according to semantic similarity;

[0039] S54: Text Compression: Compressed text is obtained by compressing the reordered text chunk metadata using a large language model;

[0040] S55: Generate a response to the shipowner's comments: Use the compressed text as the context of the shipowner's comments to obtain shipowner comment prompts; input the shipowner comment prompts into a large language model to obtain a response to the shipowner's comments, which serves as the response from the shipowner's comment response agent.

[0041] Preferably, the method for retrieving the shipowner's feedback response knowledge base is as follows:

[0042] C1: Calculate the score of each text block metadata in the shipowner's opinion response knowledge base through the shipowner's opinion index; the text block metadata score is calculated by calculating the cosine similarity between the corresponding indexes of the shipowner's opinion index and the text block metadata.

[0043] C2: Standardize the scores of each text block metadata, and then perform a weighted fusion of the standardized scores to obtain the score corresponding to each text block metadata.

[0044] C3: Based on the score corresponding to each text block metadata, retrieve the text block metadata and its source link from the shipowner's feedback response knowledge base that meet the preset score threshold; the source link can jump to the location of the historical project document corresponding to the text block metadata and highlight it; the text content of the source link includes: the page number and project name of the text block metadata in the corresponding historical project document.

[0045] Preferably, before performing the S52 shipowner opinion response knowledge base retrieval, the data in the shipowner opinion response knowledge base is filtered based on the global keywords of the opinions raised by the shipowners to obtain a filtered shipowner opinion response knowledge base; then, the shipowner opinion response knowledge base retrieval is performed in the filtered shipowner opinion response knowledge base.

[0046] Preferably, the languages ​​of the historical project documents include English and Chinese. When the historical project documents are in English, the editable text corresponding to the historical project documents is translated using a large language model to obtain the corresponding editable Chinese text. Then, the Chinese editable text is segmented and a text segment index is constructed. The text segment metadata of the English historical project documents stored in the shipowner's feedback response knowledge base is the text fragment translated into Chinese.

[0047] Beneficial effects:

[0048] This invention proposes a method for constructing an intelligent agent for responding to shipowner feedback in the field of ship design. It constructs a knowledge base for responding to shipowner feedback based on historical project documents. By searching the knowledge base, the efficiency and accuracy of queries are improved. At the same time, the recalled text is concatenated with the shipowner's feedback, and the concatenated result is input into a large language model to quickly obtain the corresponding response, avoiding the problem of omissions that are prone to occur in manual processing, and improving the accuracy and reliability of the response.

[0049] By parsing historical project documents using OCR technology and a multimodal model, the original documents can be converted into editable text, effectively solving the problem of direct retrieval when the documents are scanned. This lays the foundation for subsequent segmentation and indexing of historical project documents. The editable text content is semantically segmented, and corresponding first keyword index, first sparse vector index, and first dense vector index are built for each text block. A shipowner feedback response knowledge base is then established. This segmentation-based knowledge base avoids the slow retrieval efficiency and information interference issues in long texts, while achieving fast and accurate matching. Furthermore, the knowledge base systematically archives historical project documents, forming a traceable database. When searching the knowledge base, the system supports fusion matching of past solutions to similar questions using the three indexes, improving retrieval speed and the accuracy of retrieved text compared to manually reviewing historical project documents.

[0050] By using the recall text and final prompts derived from the shipowner's feedback, the large language model can be effectively guided to focus its attention on the domain of the feedback, avoiding the tendency for generalized responses. Simultaneously, the final prompts ensure that the large language model strictly adheres to the knowledge points provided in the recall text when generating responses, significantly suppressing irrelevant associations and thus improving the accuracy of its responses to shipowner feedback. Furthermore, using the large language model to respond to shipowner feedback effectively improves the efficiency and accuracy of responses, while avoiding the slowness and potential for omissions inherent in manual responses. Attached Figure Description

[0051] Figure 1 This is a flowchart of a method for constructing a shipowner's opinion response agent for the field of ship design.

[0052] Figure 2 A flowchart for building a knowledge base to respond to shipowner feedback.

[0053] Figure 3 A flowchart illustrating the response process of an intelligent agent to a shipowner's comments. Detailed Implementation

[0054] The following detailed description is provided in conjunction with the accompanying drawings and specific embodiments.

[0055] Example 1:

[0056] like Figure 1 As shown, a method for constructing a shipowner's opinion response agent for the field of ship design includes the following steps:

[0057] S1: Obtain and parse historical project documents: Obtain scanned copies of historical project documents related to ship design; parse and integrate different types of data in the historical project documents to obtain editable text of the historical project documents;

[0058] S2: Semantic segmentation and boundary optimization: Semantically segment the editable text to obtain coarse-grained text blocks; analyze the semantic continuity in each coarse-grained text block and optimize the boundary of semantic segmentation; then segment the coarse-grained text blocks according to the optimized segmentation boundary to obtain text block metadata;

[0059] S3: Constructing a text block index: The text block index includes a first keyword index, a first sparse vector index, and a first dense vector index; the text block metadata is processed based on the BERT model, the term frequency-inverse document frequency algorithm, and the context attention weight to construct the first keyword index; then the text block metadata is vectorized to obtain the first sparse vector index and the first dense vector index;

[0060] S4: Construct a knowledge base for shipowner feedback responses: Construct a knowledge base for shipowner feedback responses by combining editable text, text chunk metadata, and text chunk index;

[0061] S5: Shipowner's Opinion Response: Construct a shipowner's opinion index corresponding to the text block index; fuse and match the text block metadata in the shipowner's opinion response knowledge base through the shipowner's opinion index to obtain a recalled text fragment; use the recalled text fragment as the context of the shipowner's opinion to obtain shipowner's opinion prompt words; input the shipowner's opinion prompt words into a large language model to obtain a solution and source link for the shipowner's opinion, which serve as the response of the shipowner's opinion response agent.

[0062] Example 2:

[0063] like Figure 2 As shown, the steps for preprocessing scanned documents from historical projects and building a knowledge base for shipowner feedback responses include:

[0064] 1. Multimodal document parsing

[0065] 1) Receive historical project documents; the types of historical project documents are: scanned copies, PDFs, Word documents, etc., and the historical project documents contain unstructured data such as shipowner opinions and technical feedback.

[0066] 2) Document parsing

[0067] CV Technology Analysis: OCR technology is used to extract plain text data from scanned historical project documents to obtain text information, which retains the original format (such as tables, paragraph headings, etc. in the scanned document).

[0068] Multimodal understanding: If scanned documents of historical projects contain complex charts and formulas, use a multimodal model (such as the Shusheng Wanxiang model) to identify composite data (such as mixed text and image content) and convert the composite data into structured text, i.e., composite information.

[0069] OCR technology excels at extracting printed or handwritten fonts from scanned documents, and is particularly suitable for basic text information (such as headings and paragraphs). However, traditional OCR technology struggles to handle nested tables, images, or complex mathematical formulas. Multimodal models, on the other hand, can compensate for the limitations of OCR technology by recognizing mixed text and image content in documents. Therefore, by combining OCR technology with multimodal models, it is possible to ensure that all content in scanned documents is accurately captured and recognized.

[0070] Specifically, the text information extracted by the OCR technology and the composite information obtained by the multimodal model recognition are output and saved as a Markdown file.

[0071] Markdown standardization: The parsed content is divided into semantic layers (headings / lists / code blocks, etc.) to generate formatted Markdown files while preserving the logical structure of the original document.

[0072] Specifically, the parsed content includes textual information and composite information.

[0073] Specifically, the semantic layering refers to restoring the content according to the arrangement order of plain text information and combined text and image content in the historical project documents (while maintaining the Markdown format).

[0074] Specifically, the languages ​​of historical project documents include English and Chinese. When the historical project document is in English, the corresponding editable text is translated using a large language model to obtain the corresponding editable Chinese text. Subsequently, text chunking and text chunk indexing are performed based on this editable Chinese text. This approach avoids the following situation when matching feedback from shipowners: even if an English text fragment is highly similar to the user's input question in content, the significant difference in their corresponding index positions due to language differences leads to a lower semantic similarity score, ultimately causing the loss of English text fragments that are similar in content but ignored due to language differences.

[0075] 2. Knowledge Embedding and Vectorization

[0076] 2.1 Semantic Segmentation and Boundary Optimization: Markdown content is segmented semantically (e.g., by question-answer pairs or specific formats). In order to avoid information fragmentation during the segmentation process, sliding windows or sentence vector clustering are used to optimize the block boundaries.

[0077] Specifically, the steps for text segmentation are as follows:

[0078] Step 1: Perform preliminary segmentation of the editable text according to a specific format to form coarse-grained text blocks; the specific format is Markdown headings or question-answer pairs, etc.

[0079] Step 2: Use sliding window or sentence vector clustering methods to analyze the semantic continuity in each coarse-grained text block to obtain optimized segmentation boundaries;

[0080] Specifically, the sliding window moves across a coarse-grained text block using a fixed-size window (e.g., 256 tokens), and evaluates the boundary position based on semantic continuity within the window. This semantic continuity is achieved by calculating a score of semantic similarity between texts within the window. This semantic similarity score can be obtained by converting the text within the window into text vectors and then calculating the cosine similarity between the text vectors.

[0081] Specifically, sentence vector clustering first encodes sentences into vectors, and then uses clustering algorithms (such as K-means algorithm) to aggregate semantically similar adjacent sentences to determine boundaries;

[0082] Step 3: Segment the coarse-grained text blocks according to the optimized segmentation boundaries to obtain text block metadata.

[0083] 2.2 Vector Encoding: The text blocks are converted into text vectors (1024 dimensions) through an embedding model (BGE-M3) to support multilingual semantic understanding.

[0084] Specifically, the generated vector is used to construct a knowledge base for responding to shipowners' opinions, and the generated text vector is used to establish a dense vector index in the knowledge base for responding to shipowners' opinions, that is, the generated text vector is used as a dense vector index.

[0085] Meanwhile, when users query questions, they input the opinions raised by ship owners into the embedded model, which then converts the opinions into corresponding text vectors.

[0086] Specifically, the embedding model used in Example 2 is the same, which is the BGE-M3 model (Beijing General Embedding - Multi-Functionality, Multi-Linguality, Multi-Granularity, BGE-M3 model).

[0087] Using a unified embedding model when constructing the shipowners' feedback response knowledge base and vectorizing shipowners' feedback ensures consistency in vector representation, prevents discrepancies in the vectorization of synonymous texts due to differences in embedding models, and thus ensures high accuracy in text retrieval within the shipowners' feedback response knowledge base based on shipowners' feedback.

[0088] 3. Construction of a knowledge base for responding to shipowner feedback

[0089] 3.1 Construction of the Shipowner's Feedback Response Knowledge Base:

[0090] The shipowner's feedback response knowledge base stores text block metadata and text block attributes (document source, time, project number, etc.), and also saves the text block indexes corresponding to the text block metadata: the first keyword index, the first sparse vector index, and the first dense vector index.

[0091] Specifically, each text block metadata has a first keyword index, a first sparse vector index, and a first dense vector index; the text block metadata is obtained through the semantic segmentation and boundary optimization steps in 2.1.

[0092] Specifically, text chunk indexes are obtained by vectorizing text chunk metadata. Vectorization is a process where global keywords are vectorized to obtain keyword vectors, i.e., keyword vector indexes. Similarly, sparse vectors are obtained by applying sparse vectorization techniques to text fragment metadata, resulting in sparse vector indexes, and dense vector indexes are obtained similarly. Vectorization is a technique that maps text characters to a mathematical vector space, allowing comparisons between different semantics within a unified vector space. Therefore, each "text fragment" has its own corresponding "keyword index," and both "sparse vector indexes" and "dense vector indexes," except for the global keyword vector which maps to a few words within the fragment, represent the entire fragment.

[0093] Specifically, the steps to achieve keyword vectorization are as follows:

[0094] 1) Keyword extraction and entity recognition are performed on the text block metadata using a BERT model fine-tuned with a shipbuilding engineering corpus.

[0095] Specifically, the BERT model, fine-tuned by the shipbuilding engineering corpus, is used to identify the text block metadata in real time to obtain the entity words of the text block metadata. The entity words are used to represent and structure specific conceptual objects in the domain, providing a basis for the subsequent generation of weighted keyword vectors. The entity words are specific entity words such as ship names and component names.

[0096] 2) Keyword extraction is performed on the text chunk metadata using a BERT model fine-tuned with a shipbuilding engineering corpus to obtain high-frequency, high-weight terms, i.e., keywords. These keywords may or may not include entity words; a global keyword system is constructed using these keywords and entity words.

[0097] 3) For the identified global keywords (such as hull structural components, propulsion system components, and specification clauses), the steps for generating weighted keyword vectors according to the TF-IDF algorithm (Term Frequency-Inverse Document Frequency) and contextual attention weights, and writing them into the index, are as follows:

[0098] The first step is to quantify the global statistical importance of global keywords in the historical project document collection using the TF-IDF algorithm, and generate initial weights.

[0099] Specifically, the method for generating initial keyword weights using the TF-IDF algorithm is as follows:

[0100] Step 1: Calculate term frequency (TF):

[0101] Calculate the frequency of global keywords in a single historical project document; the formula is:

[0102] (1)

[0103] Step 2: Calculate the inverse document frequency (IDF):

[0104] Inverse document frequency (IVF) represents the global keyword importance; the formula is:

[0105] (2)

[0106] Step 3: Calculate the final TF-IDF value, and use the IF-IDF value as the initial weight of the corresponding global keyword;

[0107] The TF-IDF value is the product of TF and IDF, and the formula is:

[0108] (3)

[0109] The second step is to extract the semantic importance of the global keywords in a specific context, i.e., the attention weights, through the self-attention mechanism of the pre-trained language model (BERT model).

[0110] Specifically, the specific context (such as a sentence or document fragment) in which the global keyword is located is input into the pre-trained language model. The self-attention mechanism in the pre-trained language model processes the specific context of the input, calculates the attention score between each word and other words in the specific context, and uses the attention score between the global keyword and other words as the semantic importance of the global keyword in the specific context, i.e., the attention weight; the attention weight reflects the prominence of the global keyword.

[0111] Specifically, the specific context refers to the specific textual environment in which the global keyword appears, such as a sentence, paragraph, or the entire text chunk metadata content. The specific context provides the semantic background of the global keyword, enabling the model to dynamically evaluate the importance of the keyword based on the context, rather than considering individual words in isolation.

[0112] The third step is to multiply the initial weight and the attention weight to obtain the comprehensive weight coefficient of the keyword;

[0113] Fourth step: Multiply the comprehensive weight coefficient by the global keyword vector to obtain the weighted keyword vector; the weighted keyword vector serves as the first keyword index.

[0114] Specifically, the global keyword vector is a text vector obtained by inputting global keywords into the embedding model. Compared with extracting corresponding vectors from dense vectors obtained through the embedding model, directly inputting keywords into the embedding model ensures that each keyword has an independent low-dimensional distributed representation.

[0115] Specifically, sparse vectors are obtained by mapping text segments into high-dimensional sparse vectors using a method based on word frequency statistics. The specific steps are as follows:

[0116] Step 1: Building the dictionary

[0117] The historical project documents are segmented into words, and all the segmented words are summarized and filtered to obtain a vocabulary list corresponding to the historical documents (duplicate words are merged into one word). A dictionary is constructed using each word in the vocabulary list and its corresponding index.

[0118] Step 2: Text Preprocessing

[0119] The text block metadata is segmented into independent lexical units.

[0120] Step 3: Word frequency statistics, i.e., constructing vectors

[0121] Based on the independent vocabulary units corresponding to the text block metadata obtained in step two and the dictionary constructed in step one, the specific frequency of each word in the dictionary appearing in the current text segment is counted. A high-dimensional sparse vector is constructed using the specific frequency of each word in the dictionary in the current text segment, and this high-dimensional sparse vector serves as the first sparse vector index. The dimension of the high-dimensional sparse vector is the same as the number of words in the dictionary.

[0122] Dense vectors are obtained by using a neural network-based embedding model to map text fragments into low-dimensional dense vectors, and the resulting low-dimensional dense vectors are used as the index of the first dense vector.

[0123] Specifically, in Embodiment 2, the construction methods for the first keyword index and the second keyword index are the same, the construction methods for the first sparse vector index and the second sparse vector index are the same, and the construction methods for the first dense vector index and the second dense vector index are the same.

[0124] 3.2 Dynamic update mechanism: When new project documents are added to the database as historical reference files, the new project documents are parsed and vectorized in real time, triggering the reconstruction of the shipowner's opinion response knowledge base index to ensure that new knowledge is available immediately.

[0125] like Figure 3 The steps for querying and retrieving opinions raised by shipowners and inputting them into a large language model to obtain corresponding answers are as follows:

[0126] 4. Enhanced query and retrieval generation

[0127] 4.1 Semantic retrieval (routing selection):

[0128] Specifically, semantic retrieval involves searching and matching the shipowner's opinions according to the keyword index, sparse vector index, dense vector index, and the text block index corresponding to the metadata of each text block. The score of the text block index is calculated, and then the score of the text block index is standardized to calculate the fusion score. The K items with the highest scores are the most matching search results.

[0129] The specific steps are as follows:

[0130] 1) Extract global keywords and vectorize the opinions raised by the shipowners to obtain the shipowner opinion index (second keyword index, second sparse vector index and second dense vector index).

[0131] 2) For the metadata of each text block in the shipowner's opinion and response knowledge base, retrieval and matching are performed using three indexing methods: keyword indexing, sparse vector indexing, and dense vector indexing. The scores of the text block indexes in the shipowner's opinion and response knowledge base are calculated. After standardizing the scores of the three indexes in the text block indexes, a fusion score is calculated. Based on the fusion score, Top-K similar historical opinions (cosine similarity) are retrieved from the shipowner's opinion and response knowledge base. The output text block metadata is sorted by relevance, and the text block metadata also includes source information (original document page number, project name).

[0132] Specifically, when a shipowner submits feedback, the system first extracts potential shipbuilding engineering keywords or entities from the feedback. The extracted results serve as the explicit search intent, and together with the shipowner's feedback, generate a corresponding index for the feedback (a second keyword index, a second sparse vector index, and a second dense vector index). This index is then fed into the search engine. Within the search engine, the feedback is matched against various text fragments in the shipowner's feedback and response knowledge base using the keyword index, sparse vector index, and dense vector index respectively. Scores for each index are calculated, standardized, and then a fusion score is calculated to determine the fragments to be recalled, achieving a dual-channel recall mechanism of "semantic + precise terminology."

[0133] Specifically, during the search process in the shipowner's feedback response knowledge base, a function is implemented to forcibly filter the content of the knowledge base based on global keywords in the shipowner's feedback: for example, if the question contains "ABS classification society," the search will only retrieve results from the folder "ABS classification society." This avoids calculating all the original text in the shipowner's feedback response knowledge base, reducing the computational load and improving search efficiency.

[0134] Specifically, the process of calculating the fusion score is as follows:

[0135] The three indexes corresponding to the user's original query, namely the keyword index, sparse vector index, and dense vector index, are compared with the keyword index, sparse vector index, and dense vector index corresponding to the text fragments in the shipowner's opinion reply knowledge base. The scores corresponding to the three indexes, namely the cosine similarity, are obtained respectively.

[0136] Specifically, the formula for cosine similarity is:

[0137] (4)

[0138] Where A is the second keyword index for the shipowner's comments, and B is the first keyword index corresponding to the text block metadata in the shipowner's comment response knowledge base; This indicates the calculation of the dot product of A and B; This represents the vector magnitude of A. This represents the vector magnitude of B.

[0139] Specifically, the scores corresponding to the first sparse vector index and the first dense vector index in the text block metadata of the shipowner's opinion response knowledge base are calculated using the cosine similarity formula of formula (4).

[0140] The scores corresponding to the three indices of the metadata of each text block in the shipowner's opinion response knowledge base are standardized; the standardization is to uniformly map the scores corresponding to the three indices to the same comparable numerical range (such as [0,1]), in order to prepare for the fusion weighting in the subsequent calculation of the fusion score.

[0141] The scores corresponding to the three standardized indices are weighted and summed to obtain the fusion score. The formula for the weighted summation is:

[0142] Fusion Score = (Precise Term Weight × Standardized Keyword Vector Index Score) + (Sparse Semantic Weight × Standardized Sparse Vector Index Score) + (Dense Semantic Weight × Standardized Dense Vector Index Score) (5)

[0143] Based on the fusion score, all text chunk metadata in the shipowner's feedback response knowledge base are sorted from high to low, and the top K text fragments are selected as the final recalled fragments.

[0144] 4.2 Generative Response: Similar historical opinions retrieved are used as context and input into a large language model to generate a solution to the current problem (based on historical cases) and a source link (which can jump to the original document scan and highlight it).

[0145] The steps for generating a response are as follows:

[0146] Step 1: Recall and reorder text fragments: Concatenate the shipowner's comments and the recalled text to obtain a concatenated vector; input the concatenated vector into the Cross-Encoder ranking model to calculate semantic similarity; reorder all recalled texts based on semantic similarity.

[0147] Step 2: Text compression: Compress the reordered text fragments using a large language model to obtain compressed text;

[0148] Specifically, this "compression" mainly refers to semantic compression or content condensation, rather than traditional file size compression. Its core purpose is to significantly reduce text length while preserving core information, thereby improving information density and readability. Specifically, it utilizes the summarizing and rewriting capabilities of large language models (such as GPT) to refine potentially lengthy, repetitive, or irrelevant text blocks into a concise, refined statement that directly answers the user's question. For example, it compresses several related sentences or paragraphs into one or a few highly summarized core statements, thus generating a concise final prompt presented to the user.

[0149] Step 3: Generate a response to the shipowner's comments: Use the compressed text as the context of the shipowner's comments to obtain the shipowner's comment prompts; input the prompts into the large language model to obtain a response to the shipowner's comments.

[0150] Specifically, the response to the shipowner's comments involves inputting the retrieved historical opinions along with the user's original input question into the large language model. Therefore, the background knowledge source of the large language model's response is the historical opinions provided by the input. Consequently, the output of the large language model combines similar historical cases (i.e., historical opinions) from the input with the model's own knowledge, enabling the large language model to provide a more accurate response to the input question.

[0151] Specifically, the source link can jump to the location of the original document scan and highlight it. At the same time, the text content of the source link is: the page number of the document fragment in the original document and the project name.

[0152] 5. Traceability Design

[0153] Source Fragment and Original Link: Provides the original file link corresponding to the metadata of the text chunks in the prompt words, and highlights the relevant cited fragments. By supporting tracing back to view the historical archives of the cited content, it provides users with complete and traceable data evidence, effectively avoiding the inability to quickly verify the correctness of the output response content of the large language model due to missing information in the recalled text.

[0154] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail with reference to the accompanying drawings and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention. In short, all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention patent.

Claims

1. A method for constructing an intelligent agent for responding to shipowner feedback in the field of ship design, characterized in that, S1: Obtain and parse historical project documents: Obtain scanned copies of historical project documents related to ship design; parse and integrate different types of data in the historical project documents to obtain editable text of the historical project documents; S2: Semantic segmentation and boundary optimization: Semantically segment the editable text to obtain coarse-grained text blocks; analyze the semantic continuity in each coarse-grained text block and optimize the boundary of semantic segmentation; then segment the coarse-grained text blocks according to the optimized segmentation boundary to obtain text block metadata; S3: Construct a text block index: The text block index includes: a first keyword index, a first sparse vector index, and a first dense vector index; The text block metadata is processed based on the BERT model, the term frequency-inverse document frequency algorithm, and the context attention weight to construct the first keyword index; then the text block metadata is vectorized to obtain the first sparse vector index and the first dense vector index. S4: Construct a knowledge base for shipowner feedback responses: Construct a knowledge base for shipowner feedback responses by combining editable text, text chunk attributes, text chunk metadata, and text chunk indexes; S5: Shipowner's Opinion Response: Construct a shipowner's opinion index corresponding to the text block index; fuse and match the text block metadata in the shipowner's opinion response knowledge base through the shipowner's opinion index to obtain a recalled text fragment; use the recalled text fragment as the context of the shipowner's opinion to obtain shipowner's opinion prompt words; input the shipowner's opinion prompt words into a large language model to obtain a solution and source link for the shipowner's opinion, which serve as the response of the shipowner's opinion response agent.

2. A method for constructing a shipowner's opinion response intelligent agent based on claim 1, characterized in that, The method for parsing and integrating different types of data in the historical project documents as described in step S1 is as follows: S11: Extract plain text data from scanned historical project documents using optical character recognition (OCR) technology to obtain text information; S12 identifies composite data in the scanned historical project documents using a multimodal model to obtain composite information; The text and composite information are in Markdown format; The composite data includes: formulas, tables, and images; the multimodal model adopts the open-source Shusheng Wanxiang model; S13: Integrate the text information and the composite information according to the document structure of the scanned historical project document to obtain the editable text of the historical project document.

3. A method for constructing a shipowner's opinion response intelligence agent based on claim 1, characterized in that, The method for obtaining text block metadata in step S2 is as follows: S21: Semantically segment the editable text according to a specific format to obtain coarse-grained text blocks; the specific format includes, but is not limited to: Markdown headings, question-answer pairs; S22: Use the sliding window algorithm or sentence vector clustering algorithm to analyze the semantic continuity in each coarse-grained text block, and determine the boundary for optimizing semantic segmentation based on semantic continuity; S23: Segment the coarse-grained text block according to the optimized segmentation boundary to obtain text block metadata.

4. A method for constructing a shipowner's opinion response intelligent agent based on claim 1, characterized in that, The method for obtaining the first keyword index is as follows: S31: Entity recognition is performed on the text block metadata using the BERT model to obtain entity words; S32: Extract keywords from the text block metadata using the BERT model to obtain keywords; the keywords and entity words constitute global keywords; the BERT models in S31 and S32 are both BERT models fine-tuned using the shipbuilding engineering corpus; S33: The global keywords are vectorized using the term frequency-inverse document frequency algorithm (TF-IDF algorithm) and contextual attention weights to obtain a weighted keyword vector, which is the first keyword index.

5. A method for constructing a shipowner's opinion response intelligent agent based on claim 4, characterized in that, The method for obtaining the weighted keyword vector in step S33 is as follows: A1: The global importance score of global keywords in the historical project document set is quantified by the term frequency-inverse document frequency algorithm; the importance score is the initial weight. A2: Input a specific context into the pre-trained language model; calculate the semantic importance, i.e., the attention weight, of the global keyword in the specific context of the input through the self-attention mechanism of the pre-trained language model; the specific context is the specific sentence in which the global keyword is located in the text block metadata; A3: Multiply the initial weight and the attention weight to obtain the comprehensive weight coefficient of the global keyword; A4: Multiply the comprehensive weight coefficient by the global keyword vector to obtain the weighted keyword vector; the global keyword vector is the output after the global keywords are input into the model.

6. A method for constructing a shipowner's opinion response intelligent agent based on claim 1, characterized in that, The methods for obtaining the sparse vector index and the dense vector index are as follows: B1: By using word frequency statistics, the text block metadata is mapped to a high-dimensional sparse vector, i.e., the first sparse vector index. B2: By using an embedding model, text chunk metadata is mapped to a low-dimensional dense vector, i.e., the first dense vector index.

7. A method for constructing a shipowner's opinion response intelligent agent based on claim 1, characterized in that, The method for responding to the shipowner's comments in step S5 is as follows: S51: Constructing an index of shipowner opinions: The index of shipowner opinions includes: a second keyword index, a second sparse vector index, and a second dense vector index; global keywords of shipowner opinions are extracted using the BERT model; the global keywords are vectorized using the TF-IDF algorithm and contextual attention weights to obtain the second keyword index; the second sparse vector index and the second dense vector index of the shipowner opinions are obtained by vectorizing the shipowner opinions. S52: Retrieval of the Shipowner's Opinion Response Knowledge Base: By fusing and matching the text block metadata in the Shipowner's Opinion Response Knowledge Base with the index of the shipowner's opinion, the text block metadata recalled from the Shipowner's Opinion Response Knowledge Base and the source link of the text block metadata are obtained. S53: Reordering of Recalled Text Fragments: Concatenate the shipowner's comments and the recalled text fragment metadata to obtain a concatenated vector; input the concatenated vector into the Cross-Encoder ranking model to calculate semantic similarity; reorder all recalled text fragment metadata according to semantic similarity; S54: Text Compression: Compressed text is obtained by compressing the reordered text chunk metadata using a large language model; S55: Generate a response to the shipowner's comments: Use the compressed text as the context of the shipowner's comments to obtain shipowner comment prompts; input the shipowner comment prompts into a large language model to obtain a response to the shipowner's comments, which serves as the response from the shipowner's comment response agent.

8. A method for constructing a shipowner's opinion response intelligent agent based on claim 7, characterized in that, The method for retrieving the shipowner's feedback response knowledge base is as follows: C1: Calculate the score of each text block metadata in the shipowner's opinion response knowledge base through the shipowner's opinion index; the text block metadata score is calculated by calculating the cosine similarity between the corresponding indexes of the shipowner's opinion index and the text block metadata. C2: Standardize the scores of each text block metadata, and then perform a weighted fusion of the standardized scores to obtain the score corresponding to each text block metadata. C3: Retrieve text block metadata and source links that meet the preset score threshold from the shipowner's feedback response knowledge base based on the score corresponding to each text block metadata. The source tracing link can jump to the location of the historical project document corresponding to the text block metadata and highlight it; The text content of the source link includes: the page number and project name of the corresponding historical project document for the text block metadata.

9. A method for constructing a shipowner's opinion response intelligent agent based on claim 7, characterized in that, Before conducting a search of the S52 shipowner feedback response knowledge base, the data in the knowledge base is filtered based on global keywords related to the shipowners' feedback, resulting in a filtered knowledge base. Then, the shipowner feedback response knowledge base is searched.

10. A method for constructing a shipowner's opinion response intelligent agent based on claim 1, characterized in that, The languages ​​of historical project documents include English and Chinese. When the historical project document is in English, the corresponding editable text of the historical project document is translated using a large language model to obtain the corresponding editable Chinese text. Then, the Chinese editable text is segmented and a text segment index is constructed. The text segment metadata of the English historical project document stored in the shipowner's feedback response knowledge base is the text fragment translated into Chinese.

Images