Data retrieval method and system based on large model and combined with multiple processing mechanisms

Abstract

The data retrieval method comprises the following steps: performing OCR (Optical Character Recognition) and paragraph splitting on an original document to generate a parent segmentation abstract and a child segmentation abstract which have a hierarchical relationship, and fusing an abstract text with metadata to obtain an abstract file; constructing a structured data abstract directory containing semantic features and attribute features; obtaining user dialogue history and current input, constructing a semantic input matrix, and performing semantic analysis and intention reasoning under role and task constraints by using a large model to generate a standardized search intention; inputting the standardized search intention and the structured data abstract directory into a large model, screening candidate abstracts through a bidirectional association matching mechanism, performing hierarchical mapping based on a knowledge base index rule, and reasoning to generate an original storage index of target data; and carrying out validity verification on the original storage index generated by reasoning, obtaining original data content according to the index passing the verification, and returning the original data content.

Description

Technical Field

[0001] This invention relates to the field of data retrieval technology, and in particular to a data retrieval method and system based on a large model and combined with multiple processing mechanisms. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models have become a core technology for the intelligent transformation of various industries. However, large models are limited by the deadline for pre-training data, making it impossible to access current hot topics, real-time data, and confidential data specific to certain industries. Furthermore, they are prone to "illusion" phenomena when dealing with specialized problems.

[0003] To address these issues, mainstream solutions in the industry employ retrieval enhancement generation technology, which uses an external knowledge base to retrieve relevant knowledge fragments before generating an answer. Traditional RAG or most existing data retrieval solutions typically involve two core steps: first, keyword matching or simple rewriting of the user query; second, converting the query into a vector and retrieving document fragments from the vector database by calculating cosine similarity.

[0004] However, existing data retrieval technologies still have the following significant drawbacks in practical applications: Traditional solutions often rely on vector similarity matching after text slicing. Vector retrieval is essentially a probability-based fuzzy matching method, which struggles to handle queries with complex hierarchical relationships or specific constraints (such as time, region, or scenario). When dealing with massive amounts of data, relying solely on similarity can easily recall a large amount of low-relevance "noise" data, leading to knowledge fragmentation and making it difficult to accurately locate target data.

[0005] Existing technologies often only perform referential elimination or simple keyword extraction for user questions. However, in complex business scenarios, users' true needs are often implicit in the context of multi-turn dialogues, and include explicit constraints (such as specific time periods) and implicit needs (such as a preference for a certain solution). The lack of structured extraction and logical verification of these elements makes it easy for the search direction to deviate and difficult to correct.

[0006] Existing retrieval processes are typically linear (input-retrieval-output), lacking an intermediate "reasoning layer." Unlike human experts who can first read through a data catalog and then make logical judgments and path selections based on intent, these systems also lack a secondary verification mechanism for the validity of search results. This means that when faced with complex queries, the system cannot compensate for the shortcomings of simple semantic matching through logical reasoning. Summary of the Invention

[0007] To address the aforementioned technical problems in the existing technology, this invention proposes a data retrieval method and system based on a large model and combined with multiple processing mechanisms to solve the above-mentioned technical problems.

[0008] According to a first aspect of the present invention, a data retrieval method based on a large model and incorporating multiple processing mechanisms is proposed, comprising: S1: Perform OCR recognition and paragraph segmentation on the original document to generate parent segment summary and child segment summary with hierarchical relationship, and merge the summary text with metadata to construct a structured data summary catalog containing semantic features and attribute features; S2: Obtain user dialogue history and current input, construct semantic input matrix, and use a large model to perform semantic parsing and intent reasoning under role and task constraints to generate standardized search intent containing core demand topics, demand-oriented domains and potential constraints. S3: Input the standardized search intent and structured data summary catalog into the large model, filter candidate summaries through a two-way association matching mechanism, and perform hierarchical mapping based on knowledge base indexing rules to infer and generate the original storage index of the target data; S4: Perform validity checks on the original storage index generated by inference, retrieve the original data content based on the index that passes the validation, and return it.

[0009] In some specific embodiments, the construction of the structured data summary catalog in S1 specifically includes: extracting the core topic keywords of each parent segment and sub-segment; establishing a semantic hierarchical mapping relationship between the sub-segment summary and the parent segment summary, clarifying the position of the sub-segment in the global topic of the parent segment; and fusing the domain attributes, update time metadata information and text semantics of the segment summary to generate a comprehensive feature vector, which serves as a component of the summary catalog.

[0010] In some specific embodiments, the specific process of generating standardized search intent in S2 includes: Information input stage: Organize the time sequence of users' historical questions and user feedback information, construct a structured dialogue history context, and combine it with preset constraint prompts as input; Semantic parsing stage: Use large models to extract action keywords and object keywords from user input, and use attention mechanisms to calculate the semantic relevance between the current input and historical dialogues to uncover implicit requests; Intent reasoning phase: Execute context consistency check. If the initially identified intent conflicts with historical requests, backtrack to correct the reasoning bias. If there is user feedback information, assign higher reasoning weight to the feedback information to calibrate the intent. Output phase: Output standardized intent expressions, which should include at least the core requirement theme, requirement type, target domain, and scenario constraints.

[0011] In some specific embodiments, the bidirectional association matching mechanism in S3 adopts a three-level progressive logic: First-level screening: Extract key semantic tags from standardized search intent, compare them with the core keyword set in the summary catalog, and select a primary candidate summary set containing matching keywords; Secondary screening: Calculate the cosine similarity between the comprehensive feature vector of the standardized search intent and the feature vector of each summary in the primary candidate summary set, and retain the summaries with similarity higher than a preset threshold as the precise candidate set; Three-level screening: Analyze the scenario, domain, or time constraints in the standardized search intent, and use the metadata in the summary catalog to filter the precise candidate set, eliminating summaries that do not meet the constraints.

[0012] In some specific embodiments, the original storage index for generating target data in S3 also includes processing hierarchical index association: using the segment ID and storage path prefix in the summary metadata, the selected summary is converted into a unique corresponding original data index; if the selected summary contains both parent and child segments, a parent-child association index pair is generated; if the selected child segment summary belongs to different parent segments but is related to the intent, the indexes across parent segments are associated and marked with their common associated intent keywords.

[0013] In some specific embodiments, the validity check in S4 includes a dual-check mechanism: Index validity verification; simulate index call logic to verify whether the generated index can locate the physically existing original data segments; Review the intent matching degree; re-verify the semantic matching degree between the summary content corresponding to the generated index and the standardized search intent; if the verification fails, backtrack to step S3 to re-select the summary and generate the index.

[0014] According to a second aspect of the invention, a computer-readable storage medium is provided on which one or more computer programs are stored, which, when executed by a computer processor, implement the method described above.

[0015] According to a third aspect of the present invention, a data retrieval system based on a large model and incorporating multiple processing mechanisms is proposed, comprising: The data preprocessing module is configured to perform OCR recognition and paragraph segmentation on the original document, generate parent segment summaries and child segment summaries with hierarchical relationships, and integrate the summary text with metadata to construct a structured data summary catalog containing semantic features and attribute features. The intent recognition module is configured to acquire user dialogue history and current input, construct a semantic input matrix, and use a large model to perform semantic parsing and intent reasoning under role and task constraints to generate standardized search intents that include core demand topics, demand-oriented domains, and potential constraints. The index reasoning module is configured to input standardized search intent and structured data summary catalog into the large model, filter candidate summaries through a two-way association matching mechanism, and perform hierarchical mapping based on knowledge base indexing rules to infer and generate the original storage index of the target data. The verification and retrieval module is configured to perform validity verification on the original storage index generated by inference, retrieve the original data content based on the index that passes the verification, and return it.

[0016] In some specific embodiments, the intent recognition module specifically includes: The information construction unit is configured to standardize and organize the extracted large model dialogue records to form a structured dialogue history that includes user question time series, large model historical responses and user feedback information, and to construct a semantic input matrix by combining preset role positioning and task target prompt words. The multidimensional parsing unit is configured to use a large model to extract action keywords and object keywords from user input to pinpoint direct needs, and to calculate the semantic relevance between the current input text and the historical dialogue content based on an attention mechanism. The inference calibration unit is configured to verify the consistency between the initially identified intent and the context of the dialogue history, and to assign it a high inference weight to correct inference bias when user feedback is detected. The standardized output unit is configured to format the reasoning results into a standardized expression of intent that includes the core requirement theme, requirement type, requirement domain, and potential constraints.

[0017] In some specific embodiments, the indexing inference module is configured to execute a bidirectional association matching mechanism that includes three levels of progressive logic: Keyword initial screening logic: Extract key semantic tags from standardized search intent, compare them with the core keyword set in the summary catalog, and select a primary candidate summary set containing at least one matching keyword; Similarity screening logic: Calculate the cosine similarity between the comprehensive feature vector of the standardized search intent and the feature vector of each summary in the primary candidate summary set, and retain the summaries with similarity higher than the preset threshold as the precise candidate set; The final screening logic of the constraint is to parse the scenario, domain or time constraints in the standardized search intent, use the metadata information in the summary catalog to filter the precise candidate set, and remove the summaries that do not meet the constraints.

[0018] This invention abandons the retrieval model that relies solely on vector cosine similarity and innovatively proposes an indexing mechanism based on "hierarchical summary directory + large-scale model logical reasoning". By constructing a structured summary directory containing parent / child hierarchical relationships and integrating metadata features, this invention utilizes the logical reasoning capabilities of a large-scale model (rather than just computational capabilities) to "understand and select" within the directory. This enables the system to understand the logical connections between summaries, thereby accurately locating target data indexes in massive amounts of data. This significantly reduces the large amount of low-relevance noise data and fragmented knowledge introduced by semantic fuzzy matching in traditional RAG schemes, greatly improving retrieval accuracy.

[0019] Compared to existing technologies that merely perform referential elimination or simple keyword extraction on user questions, this invention constructs a four-stage intent recognition closed loop: "information input - semantic parsing - intent reasoning - result output." By combining dialogue history context, user feedback signals, and explicit role constraints, the system can accurately extract a structured intent matrix containing core needs, target domains, and potential constraints (such as time and scenario). In particular, the introduction of a context consistency verification mechanism enables the system to identify and correct potential user expression deviations or implicit demands, ensuring that the retrieval direction is highly consistent with the user's actual needs.

[0020] This invention introduces a unique dual verification mechanism (index validity verification + intent matching degree review). After generating the index, the system not only verifies the validity of the index's physical path but also backtracks to verify the semantic matching degree between the search results and the original intent. This self-examination and correction capability enables the system to automatically backtrack and re-retrieve when faced with metadata errors or inference biases, effectively avoiding "illusionary" links or invalid references common in traditional retrieval schemes, and ensuring that the data content ultimately returned to the large model is real, valid, and highly relevant. Attached Figure Description

[0021] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many anticipated advantages of the embodiments will be readily recognized as they become better understood through reference to the following detailed description. Other features, objects, and advantages of this application will become more apparent from reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of a data retrieval method based on a large model and combined with multiple processing mechanisms, according to an embodiment of this application; Figure 2 This is a flowchart of the preliminary data processing of a specific embodiment of this application; Figure 3This is a flowchart illustrating search keyword optimization as a specific embodiment of this application; Figure 4 This is a flowchart illustrating how target data is obtained based on keywords and abstract data, according to a specific embodiment of this application. Figure 5 This is a framework diagram of a data retrieval system based on a large model and combined with multiple processing mechanisms, according to an embodiment of this application. Figure 6 This is a schematic diagram of the structure of a computer system used to implement the electronic device of the present application. Detailed Implementation

[0022] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Figure 1 A flowchart illustrating a data retrieval method based on a large model and incorporating multiple processing mechanisms according to an embodiment of this application is shown. Figure 1 As shown, the method includes the following steps: S1: Perform OCR recognition and paragraph segmentation on the original document to generate parent and child segment summaries with hierarchical relationships. Then, integrate the summary text with metadata to construct a structured data summary catalog containing semantic and attribute features.

[0025] In a specific embodiment, S1, constructing a structured data summary catalog, specifically includes: extracting core topic keywords for each parent and child segment; establishing a semantic hierarchy mapping between the child segment summary and the parent segment summary, clarifying the position of the child segment within the global topic of the parent segment; and fusing the domain attributes, update time metadata information, and text semantics of the segment summary to generate a comprehensive feature vector, which serves as a component of the summary catalog. By establishing a semantic hierarchy mapping between parent and child segments and deeply fusing metadata (attributes, time) with text semantics, the constructed summary catalog possesses both "semantic depth" and "attribute breadth." This structured feature not only preserves the contextual logic of knowledge but also provides multi-dimensional filtering criteria for subsequent reasoning, effectively avoiding information gaps caused by flattened slicing.

[0026] S2: Obtain user dialogue history and current input, construct a semantic input matrix, and use a large model to perform semantic parsing and intent reasoning under role and task constraints to generate standardized search intents that include core demand topics, demand-oriented domains, and potential constraints.

[0027] In a specific embodiment, the process of generating standardized search intent in S2 includes: Information input stage: Organize the time sequence of users' historical questions and user feedback information, construct a structured dialogue history context, and combine it with preset constraint prompts as input; Semantic parsing stage: Use large models to extract action keywords and object keywords from user input, and use attention mechanisms to calculate the semantic relevance between the current input and historical dialogues to uncover implicit requests; Intent reasoning phase: Execute context consistency check. If the initially identified intent conflicts with historical requests, backtrack to correct the reasoning bias. If there is user feedback information, assign higher reasoning weight to the feedback information to calibrate the intent. The output phase outputs standardized intent representations, which include at least the core demand theme, demand type, target domain, and scenario constraints. This step constructs a closed loop for intent recognition, encompassing historical backtracking, semantic parsing, consistency verification, and standardized output. This mechanism accurately captures the implicit demands of users in multi-turn dialogues and dynamically corrects inference biases through user feedback weights, ensuring the system's ability to understand complex and fuzzy queries and maintaining a high degree of consistency between the retrieval direction and the user's true intent.

[0028] S3: Input the standardized search intent and structured data summary catalog into the large model, filter candidate summaries through a two-way association matching mechanism, and perform hierarchical mapping based on knowledge base indexing rules to infer and generate the original storage index of the target data.

[0029] In a specific embodiment, the bidirectional association matching mechanism in S3 adopts a three-level progressive logic: First-level screening: Extract key semantic tags from standardized search intent, compare them with the core keyword set in the summary catalog, and select a primary candidate summary set containing matching keywords; Secondary screening: Calculate the cosine similarity between the comprehensive feature vector of the standardized search intent and the feature vector of each summary in the primary candidate summary set, and retain the summaries with similarity higher than a preset threshold as the precise candidate set; A three-tiered screening process is employed: parsing the scenario, domain, or time constraints in the standardized search intent, and using metadata from the abstract catalog to filter the precise candidate set, eliminating abstracts that do not meet the constraints. This three-tiered progressive logic—"initial keyword screening (breadth) - refined vector screening (depth) - final constraint screening (precision)"—achieves an optimal balance between retrieval efficiency and accuracy. First, hard keywords are used to quickly narrow down the scope; then, soft semantics are used to lock in relevant content; finally, hard metadata constraints are used to eliminate non-compliant data (such as expired data or incorrect scenarios), significantly reducing computational overhead and false positive rate while ensuring high recall.

[0030] In a specific embodiment, the original storage index for generating target data in S3 further includes processing hierarchical index associations: using the segment ID and storage path prefix in the summary metadata, the selected summaries are transformed into uniquely corresponding original data indexes; if the selected summary contains both parent and child segments, parent-child association index pairs are generated; if the selected child segment summaries belong to different parent segments but are all related to intent, the cross-parent segment indexes are associated and marked with their common associated intent keywords. Through parent-child association index pairs and cross-parent segment association markings, the problem of "seeing the trees but not the forest" in traditional retrieval is solved. It can not only locate single data points but also restore the hierarchy and logical relationships between knowledge points, providing structurally complete and logically coherent data support for large models to answer complex questions.

[0031] S4: Perform validity checks on the original storage index generated by inference, retrieve the original data content based on the index that passes the validation, and return it.

[0032] In a specific embodiment, the validity check in S4 includes a dual verification mechanism: Index validity verification; simulate index call logic to verify whether the generated index can locate the physically existing original data segments; The intent matching is reviewed; the semantic matching degree between the generated index's corresponding summary content and the standardized search intent is re-verified; if the verification fails, the process goes back to step S3 to re-filter the summary and generate the index. The introduction of dual verification of "physical validity + semantic matching degree" gives the system the ability to self-examine and correct errors. This not only eliminates the return of invalid indexes (dead links) but also prevents situations where the content, although physically present, deviates from the intended meaning, resulting in "off-topic" answers.

[0033] Figure 2 This is a flowchart of the preliminary data processing of a specific embodiment of this application, as shown below. Figure 2 As shown, the preliminary data processing specifically includes the following steps: Step 201: Document Text Extraction. For the original document file (such as PDF, image, scan, etc.), use technologies such as OCR (Optical Character Recognition) to extract the full text data and remove non-text noise.

[0034] Step 202: Document Paragraph Segmentation. Logically segment the full-text data to generate hierarchical parent and child segment summaries. This step goes beyond simple text slicing; instead, it establishes hierarchical mappings, clarifying the location of specific knowledge points within the parent segment's global theme (e.g., the child segment "GPU selection optimization steps" corresponds to the parent segment "large model hardware deployment cost optimization" as a sub-theme).

[0035] Step 203: Generate paragraph summaries. This step utilizes a large model to perform deep semantic analysis on the segmented paragraphs at each level. Specifically, this includes: Semantic topic extraction: Extracting the core topic and keyword set for each parent / child segment, and extracting the core topic of each parent / child segment (e.g., "Large model inference speed optimization method") through keyword recognition or topic model inference, and labeling the core keyword set; Hierarchical association mapping: Establishing semantic association between the sub-segment summary and the parent segment summary, clarifying the sub-segment's position in the global topic; Metadata semantic fusion: Integrating the metadata information of the segment summary (such as domain attributes, update time) with the text semantics to form a comprehensive feature vector of "semantic features + attribute features" (e.g., "Domain: Artificial intelligence deployment; Semantics: Hardware cost optimization; Update time: 2025"), providing support for subsequent scenario-based constraint matching.

[0036] Step 204: Structured Data Storage. The generated summary information, including hierarchical relationships, core topics, and integrated metadata, is organized into a structured data summary catalog and stored in the database, constructing a knowledge index library that can be used for large-scale model inference.

[0037] Figure 3 This is a flowchart illustrating search keyword optimization as a specific embodiment of this application, such as... Figure 3 As shown, it includes the following steps: Step 301: Structured Dialogue Data. The extracted large-scale model dialogue records are standardized and organized to form a structured dialogue historical context. This context includes user questions, time series, historical responses from the large-scale model, and user feedback on those responses (if any). The time series clarifies logical sequence and avoids semantic misunderstandings; user feedback (such as "That's not what I meant," or "I'm more interested in XX aspect") serves as a key intent correction signal.

[0038] Step 303: Initial generation of search terms. Based on the user's current input, the system initially identifies the direction of the user's needs through keyword recognition. For example, it identifies action keywords ("query", "get", "compare") and object keywords ("XX technology", "XX product").

[0039] Step 302: Form plain text data (semantic input matrix). Combine the structured dialogue history (Step 301) with the initially extracted current needs (Step 303), and add constraint prompts to construct a complete semantic input matrix. The prompts clearly define the role of the large model as a "user search intent recognizer," and the task objective is to "parse the accurate meaning of the user's current input."

[0040] Step 304: Model optimization of search terms (intent reasoning and calibration). A large model is used to perform deep analysis of the semantic input matrix under role and task constraints: Contextualization: Calculates the semantic relevance between the current input and the historical dialogue. For example, if the historical question involved "how to improve inference speed," and the current input is only "what are some optimization methods," the model can identify that "optimization methods" point to "inference speed optimization." Uncovering latent needs: Reasoning about implicit needs by combining context. For example, from the history of "the application of large models in document writing", we can infer that the current question about "compliance" refers to "compliance assurance in document writing".

[0041] Inference calibration: Perform context consistency checks. If the initial identification results conflict with historical requests, backtrack and correct them. If there are user feedback signals (such as "I would like to know more about solutions for SMEs"), give them higher weight and incorporate "SME scenario" into the intent.

[0042] Step 305: Generate search keywords (standardized intent output). The large model outputs standardized search intent after inference calibration. This intent not only includes optimized search keywords, but also contains, in a structured form, "core demand theme, demand type, demand target area, and potential constraints (such as time and location)".

[0043] Figure 4 This is a flowchart illustrating how target data is derived from keywords and abstract data, representing a specific embodiment of this application. Figure 4 As shown, it includes: Step 401: Document Structured Data Retrieval. Load the structured data summary catalog constructed in Step 204 as the knowledge base foundation for inference.

[0044] Step 403: Search Keywords (Standardized Intent Input). Receive the standardized search intent output from step 305, which serves as the logical guide and constraint for reasoning.

[0045] In a specific implementation, the indexing reasoning process of the large model relies on the synergy of two core logical capabilities to ensure accurate conversion from intent to index. This is also the core advantage of this mechanism that distinguishes it from traditional keyword retrieval: Context-aware semantic generalization matching: Large models can generalize and match keywords in the summary based on the contextual semantics of the intent, rather than being limited to literal consistency. For example, although "hardware cost control" in the intent and "hardware expenditure optimization" in the summary are different in wording, the large model can recognize them as synonyms through semantic understanding and include them in the candidate set, thereby improving the recall rate of indexing inference; Logical focus under intent constraints: Under the constraint of the "data index inferencer" role defined by the prompt words, the large model can automatically filter inference paths that are unrelated to index generation (such as avoiding judging the correctness of the summary content and not expanding the interpretation of the original data content), and always focus on the logical chain of "summary-intent-index" to ensure inference efficiency and accuracy. Metadata-driven reasoning completion: When the semantic information of the summary text is insufficient, the large model can use information such as "domain attributes" and "update time" in the metadata to complete the reasoning logic. For example, if a summary text does not specify the scenario, but the metadata is labeled "domain: SME informatization", combined with the "SME" constraint of the intent, it can be determined that the summary is related to the intent, achieving accurate matching.

[0046] In summary, this reasoning mechanism, through standardized input construction, four-level progressive reasoning, and multi-dimensional verification, fully leverages the semantic understanding and logical reasoning capabilities of the large model, achieving accurate and efficient transformation from user search intent to the original data index of the knowledge base, and providing core technical support for subsequent accurate data acquisition based on the index.

[0047] Step 402: Organize the data summary and form text data. The system adapts the structured summary directory with the standardized intent, preparing a contextual environment for large models to read and reason.

[0048] Step 404: Model matching returns data indexes (inference and mapping). This is the core inference step, where the large model executes a bidirectional association matching mechanism: First-level screening: Initial screening is conducted based on the keyword set; key semantic tags in the intent (such as "small and medium-sized enterprises" and "hardware cost optimization") are extracted and compared with the core keywords of the abstract to initially screen out a candidate set containing at least one matching keyword; Secondary screening: Cosine similarity-based fine screening is performed based on the comprehensive feature vector; the cosine similarity between the comprehensive feature vector of the intent and the feature vector of the candidate abstract is calculated. For example, the intent "Hardware cost optimization for small and medium-sized enterprises" has a similarity of 0.85 with the abstract "Cost control of GPU selection for large-scale models in small and medium-sized enterprises" (above the threshold of 0.7), and is retained; while the similarity with "Server deployment for large enterprises" is 0.4, and is rejected.

[0049] Three-level filtering: Logical filtering is performed using scenario / time constraints in the intent, combined with summary metadata. Subsequently, based on knowledge base indexing rules, the system transforms the selected best summaries into raw storage indexes (including parent-child association index pairs). During this process, the system can also perform intent matching review to verify the semantic accuracy of the indexes. For example, if the intent involves "post-2024 solutions," old summaries are excluded through metadata. Then, using the "segment ID" and "storage path prefix" in the summary metadata, accurate candidate summaries are transformed into raw data indexes (e.g., if the metadata of a sub-segment summary is "storage path prefix: / knowledge / base / deployment; parent segment ID: D001; child segment ID: D001-03", then the generated index is " / knowledge / base / deployment-D001-D001-03"). If parent-child segments are involved, association index pairs are generated; if they cross parent segments, association tags are applied.

[0050] Step 405: Return the original data based on the index. Before returning the data, a dual verification mechanism is performed: first, the validity of the index generated by inference is verified, confirming the reachability of its physical path. After the verification passes, the specific segment content in the original document is located and read directly based on the index, and returned to the large model to generate the final answer.

[0051] Finally, standardized output is achieved: the output results adopt a structured format of "index sequence + association description + matching criteria". Among them, the index sequence is sorted by relevance; the association description clearly defines the "parent-child" hierarchy and cross-parent segment association markers; the matching criteria indicate the matching status of the core keywords of the summary and the intent keywords for each index, such as "index: / xxx-D001-D001-03; matching criteria: contains intent keywords 'SMEs' 'GPU selection' 'cost optimization'".

[0052] Figure 5 This application illustrates a framework diagram of a data retrieval system based on a large model and incorporating multiple processing mechanisms, as shown in the following embodiment. Figure 5As shown, the system mainly includes a data preprocessing module 501, an intent recognition module 502, an indexing and reasoning module 503, and a verification and retrieval module 504. The data preprocessing module 501 is configured to perform OCR recognition and paragraph segmentation on the original document, generating hierarchical parent and child segment summaries, and fusing the summary text with metadata to construct a structured data summary directory containing semantic and attribute features. The intent recognition module 502 is configured to acquire user dialogue history and current input, construct a semantic input matrix, and use a large model to perform semantic parsing and intent reasoning under role and task constraints, generating a standardized search intent containing core demand topics, demand-oriented domains, and potential constraints. The indexing and reasoning module 503 is configured to input the standardized search intent and the structured data summary directory into the large model, filter candidate summaries through a bidirectional association matching mechanism, and perform hierarchical mapping based on knowledge base indexing rules to infer and generate the original storage index of the target data. The verification and retrieval module 504 is configured to verify the validity of the inferred original storage index, retrieve the original data content based on the verified index, and return it.

[0053] In a specific embodiment, the intent recognition module 502 specifically includes: an information construction unit, configured to standardize and organize the extracted large model dialogue records to form a structured dialogue history containing user question time series, large model historical responses, and user feedback information, and to construct a semantic input matrix by combining preset role positioning and task target prompt words; a multi-dimensional parsing unit, configured to use the large model to extract user input action keywords and object keywords to lock in direct needs, and to calculate the semantic correlation between the current input text and the historical dialogue content based on an attention mechanism; a reasoning calibration unit, configured to verify the consistency between the initially recognized intent and the dialogue history context, and to assign a high reasoning weight to user feedback information when it is detected to correct reasoning bias; and a standardized output unit, configured to format the reasoning result into a standardized intent expression containing core need topics, need types, need target domains, and potential constraints.

[0054] In a specific embodiment, the index reasoning module is configured 503 to execute a bidirectional association matching mechanism containing three levels of progressive logic: the first level is the keyword initial screening logic: extracting key semantic tags from the standardized search intent, comparing them with the core keyword set in the summary directory, and selecting a primary candidate summary set containing at least one matching keyword; the second level is the similarity fine screening logic: calculating the cosine similarity between the comprehensive feature vector of the standardized search intent and the feature vectors of each summary in the primary candidate summary set, and retaining summaries with similarity higher than a preset threshold as the precise candidate set; the third level is the constraint final screening logic: parsing the scene, domain, or time constraints in the standardized search intent, using the metadata information in the summary directory to filter the precise candidate set, and removing summaries that do not meet the constraints.

[0055] The following is for reference. Figure 6 It shows a schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0056] like Figure 6 As shown, the computer system includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of system 600. CPU 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0057] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a liquid crystal display (LCD) and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card and a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0058] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined in the methods of this application. It should be noted that the computer-readable storage medium of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0059] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0060] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0061] The modules described in the embodiments of this application can be implemented in software or in hardware.

[0062] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: perform OCR recognition and paragraph segmentation on the original document, generating parent and child segment summaries with hierarchical relationships, and fusing the summary text with metadata to construct a structured data summary directory containing semantic and attribute features; acquire user dialogue history and current input, construct a semantic input matrix, and use a large model to perform semantic parsing and intent reasoning under role and task constraints, generating a standardized search intent containing core demand topics, demand-oriented domains, and potential constraints; input the standardized search intent and structured data summary directory into the large model, filter candidate summaries through a bidirectional association matching mechanism, and perform hierarchical mapping based on knowledge base indexing rules to infer and generate the original storage index of the target data; verify the validity of the inferred original storage index, and retrieve and return the original data content based on the verified index.

[0063] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A data retrieval method based on a large model and combined with multiple processing mechanisms, characterized in that, include: S1: Perform OCR recognition and paragraph segmentation on the original document to generate parent segment summary and child segment summary with hierarchical relationship, and merge the summary text with metadata to construct a structured data summary catalog containing semantic features and attribute features; S2: Obtain user dialogue history and current input, construct semantic input matrix, and use a large model to perform semantic parsing and intent reasoning under role and task constraints to generate standardized search intent containing core demand topics, demand-oriented domains and potential constraints. S3: Input the standardized search intent and the structured data summary directory into the large model, filter candidate summaries through a two-way association matching mechanism, and perform hierarchical mapping based on knowledge base indexing rules to infer and generate the original storage index of the target data; S4: Perform validity checks on the original storage index generated by inference, retrieve the original data content based on the index that passes the validation, and return it.

2. The data retrieval method based on a large model and combined with multiple processing mechanisms according to claim 1, characterized in that, The construction of the structured data summary catalog in S1 specifically includes: extracting the core topic keywords of each parent segment and sub-segment; establishing the semantic hierarchy mapping relationship between the sub-segment summary and the parent segment summary, clarifying the position of the sub-segment in the global topic of the parent segment; and fusing the domain attributes, update time metadata information and text semantics of the segment summary to generate a comprehensive feature vector, which serves as a component of the summary catalog.

3. The data retrieval method based on a large model and combined with multiple processing mechanisms according to claim 1, characterized in that, The specific process of generating standardized search intent in S2 includes: Information input stage: Organize the time sequence of users' historical questions and user feedback information, construct a structured dialogue history context, and combine it with preset constraint prompts as input; Semantic parsing stage: Use large models to extract action keywords and object keywords from user input, and use attention mechanisms to calculate the semantic relevance between the current input and historical dialogues to uncover implicit requests; Intent reasoning phase: Execute context consistency check. If the initially identified intent conflicts with historical requests, backtrack to correct the reasoning bias. If there is user feedback information, assign higher reasoning weight to the feedback information to calibrate the intent. Output phase: Output standardized intent expressions, which at least include the core requirement theme, requirement type, target domain, and scenario constraints.

4. The data retrieval method based on a large model and combined with multiple processing mechanisms according to claim 1, characterized in that, The bidirectional association matching mechanism in S3 adopts a three-level progressive logic: First-level screening: Extract key semantic tags from standardized search intent, compare them with the core keyword set in the summary catalog, and select a primary candidate summary set containing matching keywords; Secondary screening: Calculate the cosine similarity between the comprehensive feature vector of the standardized search intent and the feature vector of each summary in the primary candidate summary set, and retain the summaries with similarity higher than a preset threshold as the precise candidate set; Three-level screening: Analyze the scenario, domain, or time constraints in the standardized search intent, and use the metadata in the summary catalog to filter the precise candidate set, eliminating summaries that do not meet the constraints.

5. The data retrieval method based on a large model and combined with multiple processing mechanisms according to claim 1, characterized in that, The original storage index for generating target data in S3 also includes processing hierarchical index association: using the segment ID and storage path prefix in the summary metadata, the selected summary is converted into a unique corresponding original data index; if the selected summary contains both parent and child segments, a parent-child association index pair is generated; if the selected child segment summary belongs to different parent segments but is related to the intent, the indexes across parent segments are associated and marked with their common associated intent keywords.

6. The data retrieval method based on a large model and combined with multiple processing mechanisms according to claim 1, characterized in that, The validity check in S4 includes a dual verification mechanism: Index validity verification; Simulate the index call logic to verify whether the generated index can locate the physically existing original data segments; Review the intent matching degree; re-verify the semantic matching degree between the summary content corresponding to the generated index and the standardized search intent; if the verification fails, backtrack to step S3 to re-select the summary and generate the index.

7. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that, When the one or more computer programs are executed by a computer processor, they perform the method according to any one of claims 1-6.

8. A data retrieval system based on a large model and combining multiple processing mechanisms, characterized in that, include: The data preprocessing module is configured to perform OCR recognition and paragraph segmentation on the original document, generate parent segment summaries and child segment summaries with hierarchical relationships, and integrate the summary text with metadata to construct a structured data summary catalog containing semantic features and attribute features. The intent recognition module is configured to acquire user dialogue history and current input, construct a semantic input matrix, and use a large model to perform semantic parsing and intent reasoning under role and task constraints to generate standardized search intents that include core demand topics, demand-oriented domains, and potential constraints. The index reasoning module is configured to input the standardized search intent and the structured data summary directory into the large model, filter candidate summaries through a bidirectional association matching mechanism, and perform hierarchical mapping based on knowledge base indexing rules to infer and generate the original storage index of the target data. The verification and retrieval module is configured to perform validity verification on the original storage index generated by inference, retrieve the original data content based on the index that passes the verification, and return it.

9. A data retrieval system based on a large model and combined with multiple processing mechanisms according to claim 8, characterized in that, The intent recognition module specifically includes: The information construction unit is configured to standardize and organize the extracted large model dialogue records to form a structured dialogue history that includes user question time series, large model historical responses and user feedback information, and to construct a semantic input matrix by combining preset role positioning and task target prompt words. The multidimensional parsing unit is configured to use a large model to extract action keywords and object keywords from user input to pinpoint direct needs, and to calculate the semantic relevance between the current input text and the historical dialogue content based on an attention mechanism. The inference calibration unit is configured to verify the consistency between the initially identified intent and the context of the dialogue history, and to assign it a high inference weight to correct inference bias when user feedback is detected. The standardized output unit is configured to format the reasoning results into a standardized expression of intent that includes the core requirement theme, requirement type, requirement domain, and potential constraints.

10. A data retrieval system based on a large model and combined with multiple processing mechanisms according to claim 8, characterized in that, The indexing reasoning module is configured to execute a bidirectional association matching mechanism that includes three levels of progressive logic: Keyword initial screening logic: Extract key semantic tags from standardized search intent, compare them with the core keyword set in the summary catalog, and select a primary candidate summary set containing at least one matching keyword; Similarity screening logic: Calculate the cosine similarity between the comprehensive feature vector of the standardized search intent and the feature vector of each summary in the primary candidate summary set, and retain the summaries with similarity higher than the preset threshold as the precise candidate set; The final screening logic of the constraint is to parse the scenario, domain or time constraints in the standardized search intent, use the metadata information in the summary catalog to filter the precise candidate set, and remove the summaries that do not meet the constraints.

Images