Integrated Method and Device for Intelligent Retrieval and Document Generation of Multimodal Geological Knowledge

Abstract

This application discloses an integrated method and apparatus for intelligent retrieval and document generation of multimodal geological knowledge, relating to the field of geological document generation technology. The method includes: based on acquired geological retrieval requirements, performing semantic retrieval from a constructed multimodal geological knowledge index to obtain a set of structured semantic vectors and a set of associated knowledge; processing the associated knowledge set using a dynamic prompt word assembly algorithm to obtain structured prompt instructions; inputting the structured prompt instructions into a large language model to obtain a preliminary geological document, identifying key content from it, and obtaining a target geological document based on the mapping relationship between the associated knowledge set and the original multimodal geological data; processing the acquired feedback data into an optimization signal; and based on the optimization signal and a preset collaborative optimization objective function, collaboratively optimizing the feature fusion weight parameters and knowledge assembly weight parameters to obtain optimized weight parameters. This application improves retrieval accuracy and the quality of generated documents.

Description

Technical Field

[0001] This application relates to the field of geological document generation technology, and in particular to an integrated method and apparatus for intelligent retrieval and document generation of multimodal geological knowledge. Background Technology

[0002] With the widespread application of artificial intelligence technology in various fields, techniques based on natural language processing and retrieval-enhanced generation are gradually becoming important supports for automated document generation. In fields requiring highly specialized knowledge, effectively combining retrieval and generation capabilities to improve the accuracy, standardization, and traceability of generated content has become a key challenge in current technological development. Especially in specialized fields like geology, which involve multimodal data, complex terminology systems, and stringent compliance requirements, existing general-purpose generation methods often fail to meet the needs of practical applications, hindering the further development of technology towards specialization and reliability.

[0003] In the context of intelligent document generation in the geological field, existing technologies mainly face the following problems: First, the retrieval function and the document generation function are disconnected from each other and lack an integrated coupling mechanism. This causes the generation process to rely on a general knowledge base or scattered input data, making it difficult to obtain accurate multimodal geological knowledge support, thus affecting the accuracy of the generated content.

[0004] Second, the prompt word design lacks targeted adaptation to the geological field and fails to incorporate dynamic knowledge injection capabilities, resulting in non-standard terminology and poor industry compliance in the generated content, making it difficult to meet the requirements for generating professional documents.

[0005] Third, automatically generated geological documents lack an effective traceability mechanism. Key data and conclusions cannot be linked to the original multimodal source data, making it difficult to verify the credibility of the content and lacking the conditions for auditability.

[0006] Fourth, the optimization feedback of the retrieval model and the generation model are isolated from each other, and can only receive feedback based on a single dimension. It is difficult to achieve bidirectional collaborative optimization between the two, resulting in slow improvement of the overall system performance and a lack of continuous evolution capability.

[0007] The above problems limit the professional level and practical application value of intelligent document generation systems in the geological field. It is urgent to achieve deep integration of retrieval and generation, accurate application of professional terminology, reliable traceability of content, and synergistic optimization of system performance through technological improvements. Summary of the Invention

[0008] The purpose of this application is to provide an integrated method and device for intelligent retrieval and document generation of multimodal geological knowledge, which can improve retrieval accuracy and the quality of generated documents.

[0009] To achieve the above objectives, this application provides the following solution: Firstly, this application provides an integrated method for intelligent retrieval and document generation of multimodal geological knowledge, including: Obtain the geological search requirements input by the user; Based on the geological retrieval requirements, semantic retrieval is performed from the constructed multimodal geological knowledge index to obtain a set of structured semantic vectors and a set of associated knowledge related to the geological retrieval requirements. The multimodal geological knowledge index stores structured semantic vectors generated from the acquired multimodal geological raw data after feature extraction and feature fusion, as well as the mapping relationship between the structured semantic vectors and the corresponding multimodal geological raw data. The associated knowledge set is processed using a dynamic prompt word assembly algorithm to obtain structured prompt instructions; The structured prompts are input into a large language model enhanced with geological knowledge to obtain a preliminary geological document; Based on the preliminary geological document, key content in the preliminary geological document is identified, and a source identification is generated for the key content according to the mapping relationship between the associated knowledge set and the multimodal geological raw data, resulting in a target geological document with a source identification. Obtain user feedback data regarding the target geological document, the structured semantic vector set, and the associated knowledge set, and process the feedback data into a unified optimization signal; Based on the optimized signal and the preset collaborative optimization objective function, the feature fusion weight parameters involved in constructing the multimodal geological knowledge index and the knowledge assembly weight parameters involved in the dynamic prompt word assembly algorithm are collaboratively optimized to obtain optimized weight parameters; the optimized weight parameters are used to process subsequent geological retrieval needs.

[0010] Optionally, the feature fusion weight parameters include modal weight coefficients; the method for constructing the multimodal geological knowledge index specifically includes: Acquire multimodal geological raw data; the multimodal geological raw data includes geological numerical data, geological image data, and geological text data; The multimodal geological raw data are subjected to feature extraction using a professional feature extraction algorithm to obtain a set of multimodal feature vectors; Based on the modality weight coefficients, feature fusion is performed on the multimodal feature vector set to obtain a unified structured semantic vector; the structured semantic vector consists of multiple vectors. Based on the structured semantic vectors and the original multimodal geological data, a multimodal geological knowledge index is constructed.

[0011] Optionally, the feature fusion weight parameters further include a similarity threshold; based on the geological retrieval requirements, semantic retrieval is performed from the constructed multimodal geological knowledge index to obtain a set of structured semantic vectors and a set of associated knowledge related to the geological retrieval requirements, specifically including: The geological search requirements are semantically encoded to obtain a requirement feature vector; The cosine similarity matching algorithm is used to calculate the similarity between the demand feature vector and each structured semantic vector in the multimodal geological knowledge index, so as to obtain the similarity corresponding to each structured semantic vector. Based on the similarity and similarity threshold corresponding to each structured semantic vector, a set of structured semantic vectors is obtained by filtering. Based on the mapping relationship between the structured semantic vectors and the corresponding multimodal geological raw data, the multimodal geological raw data corresponding to the set of structured semantic vectors is obtained, and the associated knowledge set is obtained.

[0012] Optionally, the associated knowledge set is processed using a dynamic prompt word assembly algorithm to obtain structured prompt instructions, specifically including: Based on predefined knowledge classification rules, the knowledge content in the associated knowledge set is classified to obtain data-related knowledge, map description-related knowledge, and literature conclusion-related knowledge. The system utilizes a dynamic prompt template library for the geological field; this library includes document structure templates, data citation templates, map description templates, and literature citation templates. Based on the knowledge assembly weight parameters, the data-type knowledge, the map description-type knowledge, and the literature conclusion-type knowledge are embedded into the data citation template, the map description template, and the literature citation template, respectively, and combined with the document structure template to obtain structured prompt instructions.

[0013] Optionally, the key content includes key assertions, core data items, and chart references; based on the preliminary geological document, the key content identified in the preliminary geological document specifically includes: Based on a pre-set geological terminology database, key assertions are identified from the preliminary geological documents by combining keyword matching and semantic similarity matching. Based on preset data format rules, core data items are extracted from the preliminary geological document by combining regular expression matching and data format validation. Based on preset chart identifiers, chart references are identified from the preliminary geological documents.

[0014] Optionally, based on the mapping relationship between the associated knowledge set and the multimodal geological raw data, a source identification identifier is generated for the key content to obtain a target geological document with the source identification identifier, specifically including: Based on the mapping relationship between the associated knowledge set and the multimodal geological raw data, the multimodal geological raw data corresponding to the key content is determined; Using a hash algorithm, the multimodal geological raw data corresponding to the key content is processed to obtain a unique identifier corresponding to the multimodal geological raw data; Using an encryption algorithm, the unique identifier corresponding to the multimodal geological raw data is encrypted and encoded to obtain a verifiable traceability identifier. The traceability identifier is then embedded in the corresponding position in the preliminary geological document to obtain a target geological document with a traceability identifier.

[0015] Optionally, the feedback data includes a score for the relevance of the search results and a score for the quality of the generated document; processing the feedback data into a unified optimization signal specifically involves: Based on the feedback standardization formula The optimized signal is obtained through calculation; in, To optimize the signal; For feedback weights; A score for the relevance of the search results; To generate a score for document quality; To ensure the relevance of search results; To improve the quality of the generated documents.

[0016] Optionally, the collaborative optimization objective function is specifically: ; in, To collaboratively optimize the objective function; To optimize the signal; The first loss weight; This is the second loss weight; This is the similarity loss term for the search results; To generate a document standardization loss term.

[0017] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the integrated method for intelligent retrieval and document generation of multimodal geological knowledge as described above.

[0018] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the integrated method for intelligent retrieval and document generation of multimodal geological knowledge as described above.

[0019] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the integrated method for intelligent retrieval and document generation of multimodal geological knowledge as described above.

[0020] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides an integrated method and apparatus for intelligent retrieval and document generation of multimodal geological knowledge. First, by acquiring user geological retrieval needs and performing semantic retrieval from a multimodal geological knowledge index to obtain a set of related knowledge, the retrieval process is tightly coupled with the subsequent document generation process. This solves the problem of disconnect between retrieval and generation functions and the lack of an integrated mechanism in existing technologies, achieving document generation support based on accurate multimodal geological knowledge and significantly improving the accuracy and professionalism of the generated content. Second, by using a dynamic prompt word assembly algorithm to process the related knowledge set to generate structured prompt instructions, and inputting these instructions into a large language model enhanced with geological domain knowledge, the application addresses the problem of lack of domain adaptability in general prompt words, leading to non-standard and non-compliant terminology in the generated content. This achieves the automatic generation of professional documents that comply with geological industry standards and terminology requirements. Furthermore, by identifying key content based on preliminary geological documents and generating source identification tags for key content according to the mapping relationship between the associated knowledge set and multimodal geological raw data, a verifiable source association mechanism is established for automatically generated documents. This solves the problems of key data and conclusions in generated documents not being linked to original source data, low content credibility, and lack of auditability, achieving end-to-end traceability and credibility verification of generated documents. Finally, by acquiring user feedback data and processing it into a unified optimization signal, and then using this signal and a collaborative optimization objective function to collaboratively optimize feature fusion weight parameters and knowledge assembly weight parameters, a feedback optimization closed loop with bidirectional linkage between retrieval and generation models is constructed. This solves the problems of isolated model optimization and slow system performance improvement in existing technologies, achieving continuous collaborative evolution and self-improvement of the system in terms of retrieval accuracy and generation quality. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is an application environment diagram of an integrated method for intelligent retrieval and document generation of multimodal geological knowledge according to an embodiment of this application; Figure 2 A flowchart illustrating an integrated method for intelligent retrieval and document generation of multimodal geological knowledge, provided as an embodiment of this application; Figure 3 A schematic diagram illustrating the framework of an integrated method for intelligent retrieval and document generation of multimodal geological knowledge, provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] The integrated method for intelligent retrieval and document generation of multimodal geological knowledge provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send geological retrieval requests and multimodal geological raw data to server 104. Based on the geological retrieval requests, server 104 performs semantic retrieval from a constructed multimodal geological knowledge index to obtain a set of structured semantic vectors and a set of associated knowledge. It then processes the set of associated knowledge using a dynamic prompt word assembly algorithm to obtain structured prompt instructions. These structured prompt instructions are input into a large language model to obtain a preliminary geological document, from which key content is identified. Based on the mapping relationship between the associated knowledge set and the multimodal geological raw data, the target geological document is obtained. Server 104 can then feed back the obtained target geological document, the set of structured semantic vectors, and the set of associated knowledge to terminal 102.

[0026] The terminal 102 can be, but is not limited to, various desktop computers, laptops, and IoT devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers, or it can be a cloud server.

[0027] In one exemplary embodiment, such as Figures 2-3 As shown, an integrated method for intelligent retrieval and document generation of multimodal geological knowledge is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 to 207. Wherein: Step 201: Obtain the geological search requirements input by the user.

[0028] Step 202: Based on the geological retrieval requirements, semantic retrieval is performed from the constructed multimodal geological knowledge index to obtain a set of structured semantic vectors and a set of associated knowledge related to the geological retrieval requirements. The multimodal geological knowledge index stores structured semantic vectors generated after feature extraction and feature fusion of the acquired multimodal geological raw data, as well as the mapping relationship between the structured semantic vectors and the corresponding multimodal geological raw data.

[0029] Step 203: Process the associated knowledge set using a dynamic prompt word assembly algorithm to obtain structured prompt instructions.

[0030] Step 204: Input the structured prompts into the large language model enhanced with geological knowledge to obtain a preliminary geological document.

[0031] Step 205: Based on the preliminary geological document, identify the key content in the preliminary geological document, and generate source identification for the key content according to the mapping relationship between the associated knowledge set and the multimodal geological raw data, so as to obtain the target geological document with source identification.

[0032] Step 206: Obtain user feedback data on the target geological document, structured semantic vector set, and associated knowledge set, and process the feedback data into a unified optimization signal.

[0033] Step 207: Based on the optimized signal and the preset collaborative optimization objective function, the feature fusion weight parameters involved in constructing the multimodal geological knowledge index and the knowledge assembly weight parameters involved in the dynamic prompt word assembly algorithm are collaboratively optimized to obtain the optimized weight parameters; the optimized weight parameters are used to process subsequent geological retrieval needs.

[0034] Implementing steps 201 to 207 achieves the following technical effects: First, by acquiring user geological search needs and performing semantic retrieval from a multimodal geological knowledge index based on those needs to obtain a set of related knowledge, the retrieval process is tightly coupled with the subsequent document generation process. This solves the problem of disconnect between retrieval and generation functions and the lack of an integrated mechanism in existing technologies, achieving document generation support based on accurate multimodal geological knowledge and significantly improving the accuracy and professionalism of the generated content. Second, by using a dynamic prompt word assembly algorithm to process the related knowledge set to generate structured prompt instructions, and then inputting these instructions into a large language model enhanced with geological domain knowledge, prompt words are designed and knowledge is dynamically injected specifically for the geological domain. This solves the problem of general prompt words lacking domain adaptability, leading to non-standard and non-compliant terminology in the generated content, and achieves the automatic generation of professional documents that meet geological industry standards and terminology requirements. Furthermore, by identifying key content based on preliminary geological documents and generating source identification tags for key content according to the mapping relationship between the associated knowledge set and multimodal geological raw data, a verifiable source association mechanism is established for automatically generated documents. This solves the problems of key data and conclusions in generated documents not being linked to original source data, low content credibility, and lack of auditability, achieving end-to-end traceability and credibility verification of generated documents. Finally, by acquiring user feedback data and processing it into a unified optimization signal, and then using this signal and a collaborative optimization objective function to collaboratively optimize feature fusion weight parameters and knowledge assembly weight parameters, a feedback optimization closed loop with bidirectional linkage between retrieval and generation models is constructed. This solves the problems of isolated model optimization and slow system performance improvement in existing technologies, achieving continuous collaborative evolution and self-improvement of the system in terms of retrieval accuracy and generation quality.

[0035] The following will combine Figure 3The integrated method flow shown below details the specific implementation steps of the integrated method: Furthermore, in step 202, the feature fusion weight parameters include modal weight coefficients; unstructured / semi-structured multimodal geological raw data are transformed into efficiently searchable structured semantic vectors, establishing a standardized index library to provide a data foundation for subsequent accurate retrieval, thus solving the problems of scattered geological knowledge and low retrieval efficiency. The method for constructing the multimodal geological knowledge index library specifically includes: Step a1: Obtain multimodal geological raw data; multimodal geological raw data includes geological numerical data (such as well logging curve numerical data). ), geological image data (such as seismic profile image data) ) and geological text data (such as geological literature texts) and map description text )wait.

[0036] Step a2 involves using a professional feature extraction algorithm to extract features from the multimodal geological raw data, resulting in a multimodal feature vector set. Specifically, feature vectors for each modality are extracted from the multimodal geological raw data using feature extraction models corresponding to each modality: geological text data is extracted using the BERT model. , The feature vectors of the geological image data are extracted using a CNN model. The feature vectors of the data are extracted from the geological numerical data through standardization. The feature vectors of ) are used to obtain the multimodal feature vector set { , , , }

[0037] Step a3: Based on the modality weight coefficients, a multimodal feature vector set is fused using a multimodal feature fusion algorithm to obtain a unified structured semantic vector. There are multiple structured semantic vectors. The multimodal feature fusion formula is: V = V represents the fused structured semantic vector. Geological literature text eigenvectors; For seismic profile image data eigenvectors; Numerical data of well logging curves eigenvectors; Description text for the drawing eigenvectors; , , , These are modal weighting coefficients (known quantities, pre-defined based on geological knowledge, satisfying...) ,and .

[0038] Step a4: Based on structured semantic vectors and multimodal geological raw data, a multimodal geological knowledge index is constructed, storing the mapping relationship between structured semantic vectors V and corresponding multimodal geological raw data. ( (This is raw data for multimodal geological analysis).

[0039] Furthermore, the feature fusion weight parameters in step 202 also include a similarity threshold; knowledge matching the user's geological retrieval needs is accurately screened from massive amounts of multimodal geological raw data, providing accurate and reliable input for the Large Language Model (LLM) to generate documents, avoiding the generated content from deviating from professional knowledge support. Based on the geological retrieval needs, semantic retrieval is performed from the constructed multimodal geological knowledge index to obtain a set of structured semantic vectors and associated knowledge related to the geological retrieval needs, specifically including: Step b1, using the same text feature extraction algorithm as step a2 (i.e., the BERT model), to address geological retrieval needs. (e.g., "Well logging and seismic response analysis of a shale gas reservoir in a certain block") is semantically encoded to obtain the demand feature vector. .

[0040] Step b2 involves using a cosine similarity matching algorithm to calculate the similarity between the demand feature vector and each structured semantic vector in the multimodal geological knowledge index, thus obtaining the similarity for each structured semantic vector. The similarity calculation formula is as follows: ; The cosine similarity between the demand feature vector and each structured semantic vector in the multimodal geological knowledge index; Semantic encoding vectors for geological retrieval needs; This is the i-th structured semantic vector in the multimodal geological knowledge index. for and The dot product; for The L2 norm; ∥ ∥for The L2 norm.

[0041] Step b3: Based on the similarity and similarity threshold corresponding to each structured semantic vector, a set of structured semantic vectors is obtained; specifically, one or more structured semantic vectors with similarity higher than a preset similarity threshold are selected to form a set of structured semantic vectors. The filtering rules are as follows: ={ | For each i = 1, 2, ..., N, select the Top-K results in descending order of similarity. =Similarity threshold (default is 0.6, which can be adjusted through subsequent optimization); N is the total number of structured semantic vectors in the multimodal geological knowledge index; K is the number of returned results (i.e. the number of vectors in the structured semantic vector set, the system default is K=5 or user-defined). This is the set of structured semantic vectors retrieved.

[0042] Step b4: Based on the mapping relationship between structured semantic vectors and corresponding multimodal geological raw data, obtain the multimodal geological raw data corresponding to the structured semantic vector set, and obtain the associated knowledge set. ( (Including raw data fragments, literature conclusions, and figure descriptions, etc.) For the retrieved set of related knowledge (as...) Sure).

[0043] Furthermore, step 203 transforms the retrieved scattered multimodal related knowledge into structured prompts that conform to geological industry standards, guiding LLM to generate geological documents with accurate terminology and standardized formatting, thus solving the core problem of inaccurate and non-standard content generated by LLM in professional fields. A dynamic prompt word assembly algorithm is used to process the related knowledge set to obtain structured prompts, specifically including: Step c1: Based on predefined knowledge classification rules, classify the related knowledge sets. The knowledge content is categorized to obtain data-related knowledge. Knowledge of graphic description and knowledge of literature conclusions Specifically: (1) Modal classification mapping based on raw geological data: if From (Well logging numerical data) or (Seismic profile image data) is categorized as data-related knowledge. ;if From (Map description text) is categorized as map description knowledge. ;if From (Geological literature texts) are categorized as literature conclusions. (2) Content-based text classification model: If For text with an unknown source, a text classification model (such as the BERT fine-tuning model) can be used to determine its category. If the text contains keywords such as "curve," "data," "profile," or "well depth," it is classified as data-related knowledge. ;like Knowledge containing keywords such as "map", "illustration", "section", and "geological map" is categorized as map description knowledge. ;like Knowledge containing keywords such as "maps," "illustrations," "sections," and "geological maps" is categorized as literature conclusions. (3) Structured matching based on metadata: If Include metadata (such as data type, source document, figure number, etc.), and classify directly by metadata.

[0044] Step c2: Call the dynamic prompt template library in the geological field. Template library This includes document structure templates, data citation templates, figure description templates, and literature citation templates.

[0045] Step c3 involves using a dynamic prompt word assembly algorithm to embed data-related knowledge, map description knowledge, and literature conclusion knowledge into the data citation template, map description template, and literature citation template, respectively, based on the knowledge assembly weight parameters. This is then combined with the document structure template to obtain structured prompt instructions. The formula for the dynamic prompt word assembly algorithm is as follows: ;P is a structured prompt instruction; This is a document structure template (preset to the standard geological report structure). For data reference templates; This is a template for describing drawings; Reference citation template; For data-related knowledge; This refers to knowledge describing the map or graph. Knowledge related to the conclusions of the literature (all from...) ); , , Weights for knowledge types (satisfying) ).

[0046] Further, in step 204, the structured prompt instructions are... Input is fed into a large language model enhanced with geological knowledge to generate preliminary geological documents. This process transforms structured suggestions into complete professional documents, converting precise retrieved knowledge into a usable final document product, completing the core "retrieval-generation" chain; specifically including: Step d1 invokes the Large Language Model (LLM) that has been infused with geological terminology and industry standard knowledge during the pre-training phase.

[0047] Step d2 involves providing the structured prompts as input to the large language model.

[0048] Step d3: Based on structured prompts and its internal geological domain knowledge, the large language model outputs a preliminary geological document with accurate terminology and a structure conforming to industry standards. .

[0049] Furthermore, the key content in step 205 includes key assertions, core data items, and chart references; establishing a traceable link between the generated content and the original multimodal geological data ensures that key information in the document is auditable and verifiable, addresses the issue of insufficient credibility of the generated content, and improves the reliability of the results; based on the preliminary geological document, the key content in the preliminary geological document is identified, specifically including: Step e1: Based on a pre-set geological terminology database (including metallogenic indicators, ore deposit models, and industry standard terms), key assertions are identified from preliminary geological documents using a combination of keyword matching (including logical words such as "indicates," "shows," and "infers," as well as conclusive statements in geological terminology) and semantic similarity matching. .

[0050] Step e2: Based on preset data format rules, core data items are extracted from the preliminary geological document by combining regular expression matching (matching "numerical value + geological unit" format, such as "500m" and "2.5%)) with data format validation (compliance with well logging, reserve, and other data specifications). .

[0051] Step e3: Based on preset chart identifiers (such as those including "Figure X" and "Table X" identifiers, and related geological map / data descriptions), identify chart references from the preliminary geological documents. .

[0052] Furthermore, in step 205, based on the mapping relationship between the associated knowledge set and the multimodal geological raw data, source identification tags are generated for key content, resulting in target geological documents with source identification tags, specifically including: Step e4: Based on the mapping relationship between the associated knowledge set and the multimodal geological raw data, determine the multimodal geological raw data corresponding to the key content.

[0053] Step e5 involves processing the multimodal geological raw data corresponding to the key content using a hash algorithm to obtain a unique identifier for the multimodal geological raw data. This unique identifier is then encrypted using an encryption algorithm to obtain a verifiable traceability identifier. Prior to this, the multimodal geological raw data corresponding to the key content needs to undergo quality verification (Z-score outlier detection algorithm) and access permission verification (AD domain hash authentication algorithm). Only after successful verification is the hash algorithm processing performed. The functions and parameters are defined as follows: The verifiable formula for generating traceability identifiers is: ; H(X) is a verifiable traceability identifier for key content X; X is key content in the preliminary geological document; H(X) is the hash value of X. The encryption code is used to uniquely identify the original multimodal geological data. (This is the index ID for the original multimodal geological data; Enc is the AES encryption algorithm).

[0054] The mapping relationship constraints are: ; For tracing the source mapping function, it connects the knowledge set. The correlation determines the multimodal geological raw data corresponding to key content X. .

[0055] Step e6 involves embedding the traceability identifier into the corresponding location in the preliminary geological document, resulting in the target geological document with the traceability identifier. The hash algorithm ensures data integrity and tamper-proofness, metadata management supports end-to-end tracing of the relationship between raw data and generated content, the access control system ensures compliance of data access and operations, and the dynamic update mechanism maintains the timeliness of data and rules. This constructs an integrated data governance closed loop encompassing "collection-verification-storage-update-traceability."

[0056] Furthermore, the feedback data F in step 206 includes the relevance of the search results. Scoring and generated document quality The scoring; obtaining user feedback data on the target geological document, the structured semantic vector set, and the associated knowledge set; processing the feedback data into a unified optimization signal to provide a unified input for the collaborative optimization of retrieval and generation parameters, specifically including: Step f1: Show the user the target geological document with traceability tags. and search results and .

[0057] Step f2: Obtain feedback data and apply it according to the feedback standardization formula. The optimized signal is calculated; where, To optimize the signal; For feedback weights; A score for the relevance of the search results; To generate a score for document quality; To ensure the relevance of search results; To improve the quality of the generated documents.

[0058] ; ; in, ∈[1,5] represents the relevance score. ∈{0,1} is a missed detection marker; ∈[1,5] represents the satisfaction rating. ∈[0,1] represents the modification range.

[0059] Furthermore, by utilizing a unified optimization signal to drive the co-evolution of retrieval and generation parameters, a virtuous cycle of "improved retrieval accuracy → optimized generation quality → enhanced feedback signal → further model optimization" is achieved, enabling the system to continuously improve itself. The specific objective function for this co-optimization is: ; in, To collaboratively optimize the objective function; To optimize the signal; The first loss weight; This is the second loss weight; This is the similarity loss term for the search results; To generate a document standardization loss term.

[0060] The constraint condition is: α + β + γ + δ = 1. θ∈[0.5,0.9]; Loss term definition: ; ; in, To standardize the matching degree function (calculated using the geological expert rule base, with values ​​[0,1]); It serves as a database of standards for the geological industry.

[0061] Furthermore, the integrated method for intelligent retrieval and document generation of multimodal geological knowledge also includes: Step 208: Complete the delivery of the target geological documents, and support the long-term self-optimization of the system through log recording and parameter iteration, ensuring that service quality continues to improve with usage. Specifically: Step g1: Select the target geological document with the source identification tag. Documents are delivered to the user; Step g2: Record all data from the entire service process (search results, generated documents, user feedback, optimization parameters) to the system log database; Step g3: When a user initiates a new request, the optimized weight parameters are called, and steps 201 to 207 are repeated to achieve continuous service iteration.

[0062] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When executed by the processor, the computer program implements an integrated method for intelligent retrieval and document generation of multimodal geological knowledge.

[0063] Those skilled in the art will understand that Figure 3 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0064] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0065] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0066] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of the relevant data are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

[0067] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0068] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0069] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0070] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An intelligent retrieval and document generation integrated method of multi-modal geological knowledge, characterized in that, The method comprises the following steps: obtaining a user input geological retrieval requirement; based on the geological retrieval requirement, performing semantic retrieval from the constructed multi-modal geological knowledge index library to obtain a set of structured semantic vectors associated with the geological retrieval requirement and a set of associated knowledge; wherein the multi-modal geological knowledge index library stores structured semantic vectors generated after feature extraction and feature fusion of obtained multi-modal geological original data, and a mapping relationship between the structured semantic vectors and the corresponding multi-modal geological original data; processing the set of associated knowledge using a dynamic prompt assembly algorithm to obtain a structured prompt instruction; inputting the structured prompt instruction into a large language model enhanced by geological domain knowledge to obtain a preliminary geological document; based on the preliminary geological document, identifying the key content in the preliminary geological document, and generating a provenance identifier for the key content based on the mapping relationship between the set of associated knowledge and the multi-modal geological original data to obtain a target geological document with a provenance identifier; obtaining feedback data from the user for the target geological document, the set of structured semantic vectors and the set of associated knowledge, and processing the feedback data into a unified optimization signal; based on the optimization signal and a preset collaborative optimization objective function, collaboratively optimizing the feature fusion weight parameters involved in constructing the multi-modal geological knowledge index library and the knowledge assembly weight parameters involved in the dynamic prompt assembly algorithm to obtain optimized weight parameters; the optimized weight parameters are used to process subsequent geological retrieval requirements. 2.The intelligent retrieval and document generation integrated method of multi-modal geology knowledge according to claim 1, characterized in that, The feature fusion weight parameters include a modal weight coefficient; the construction method of the multi-modal geological knowledge index library specifically comprises: obtaining multi-modal geological original data; the multi-modal geological original data includes geological numerical data, geological image data and geological text data; using a professional feature extraction algorithm to extract features from the multi-modal geological original data to obtain a set of multi-modal feature vectors; based on the modal weight coefficient, performing feature fusion on the set of multi-modal feature vectors to obtain a unified structured semantic vector; the structured semantic vector is multiple; based on the structured semantic vector and the multi-modal geological original data, a multi-modal geological knowledge index library is constructed. 3.The intelligent retrieval and document generation integrated method of multi-modal geology knowledge according to claim 1, characterized in that, The feature fusion weight parameters also include a similarity threshold; based on the geological retrieval requirement, performing semantic retrieval from the constructed multi-modal geological knowledge index library to obtain a set of structured semantic vectors associated with the geological retrieval requirement and a set of associated knowledge, specifically comprising: performing semantic encoding on the geological retrieval requirement to obtain a requirement feature vector; using a cosine similarity matching algorithm to calculate the similarity between the requirement feature vector and each structured semantic vector in the multi-modal geological knowledge index library to obtain the similarity corresponding to each structured semantic vector; based on the similarity corresponding to each structured semantic vector and the similarity threshold, a set of structured semantic vectors is selected; According to the mapping relationship between the structured semantic vector and the corresponding multi-modal geological original data, multi-modal geological original data corresponding to the structured semantic vector set is obtained, and an associated knowledge set is obtained. 4.The intelligent retrieval and document generation integrated method of multi-modal geology knowledge according to claim 1, characterized in that, The associated knowledge set is processed by using a dynamic prompt assembly algorithm to obtain a structured prompt instruction, specifically including: According to a pre-defined knowledge classification rule, the knowledge content in the associated knowledge set is classified to obtain data class knowledge, map description class knowledge, and literature conclusion class knowledge; A geological field dynamic prompt template library is called; the template library includes a document structure template, a data reference template, a map description template, and a literature reference template; According to a knowledge assembly weight parameter, the data class knowledge, the map description class knowledge, and the literature conclusion class knowledge are respectively embedded in the data reference template, the map description template, and the literature reference template, and are combined with the document structure template to obtain a structured prompt instruction. 5.The intelligent retrieval and document generation integrated method of multi-modal geology knowledge according to claim 1, characterized in that, The key content includes a key argument sentence, a core data item, and a graph reference item; Based on the preliminary geological document, the key content in the preliminary geological document is identified, specifically including: Based on a pre-set geological professional term library, a key argument sentence is identified from the preliminary geological document by using a combination of keyword matching and semantic similarity matching; Based on a pre-set data format rule, a core data item is extracted from the preliminary geological document by using a combination of regular expression matching and data format checking; Based on a pre-set graph identifier, a graph reference item is identified from the preliminary geological document. 6.The intelligent retrieval and document generation integrated method of multi-modal geology knowledge according to claim 1, characterized in that, According to the mapping relationship between the associated knowledge set and the multi-modal geological original data, a provenance identifier is generated for the key content to obtain a target geological document with a provenance identifier, specifically including: According to the mapping relationship between the associated knowledge set and the multi-modal geological original data, the multi-modal geological original data corresponding to the key content is determined; Using a hash algorithm, the multi-modal geological original data corresponding to the key content is processed to obtain a unique identifier corresponding to the multi-modal geological original data; Using an encryption algorithm, the unique identifier corresponding to the multi-modal geological original data is encrypted to obtain a verifiable provenance identifier, and the provenance identifier is embedded in the corresponding position in the preliminary geological document to obtain a target geological document with a provenance identifier. 7.The intelligent retrieval and document generation integrated method of multi-modal geology knowledge according to claim 1, characterized in that, The feedback data includes a score of the relevance of the search results and a score of the quality of the generated document; processing the feedback data into a unified optimization signal specifically includes: According to the feedback standardization formula The optimized signal is calculated; wherein, to optimize the signal; to weight the response; to score the relevance of the search results; to score the quality of the generated documents; to score the relevance of the search results; to score the quality of the generated documents. 8.The intelligent retrieval and document generation integrated method of multi-modal geology knowledge according to claim 1, characterized in that, The collaborative optimization target function specifically includes: ； wherein, is a synergistic optimization objective function; is an optimization signal; is a first loss weight; is a second loss weight; is a retrieval result similarity loss term; is a generated document specification loss term.

9. A computer device comprising: A memory, a processor, and a computer program stored on the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the multi-modal geological knowledge intelligent retrieval and document generation integrated method of any one of claims 1-8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the multi-modal geological knowledge intelligent retrieval and document generation integrated method of any one of claims 1-8.

Images