Adaptive extended retrieval method and system based on semantic ontology and large model knowledge reasoning

Abstract

The application relates to the field of data retrieval and discloses a self-adaptive extended retrieval method and system based on semantic ontology and large model knowledge reasoning; ontology semantic extraction and cell type identification are performed on unstructured documents and structured data with missing table headers or uncertain table header semantics; a multi-modal composite index library and lightweight index description data are constructed; initial retrieval results are obtained according to user queries; a large model knowledge reasoning unit is input with the user query, the initial retrieval results and candidate lightweight index description records; a directional retrieval instruction is generated when there is an uncovered node; ontology relationship traversal is performed according to the instruction, or type feature matching and row-column context reconstruction are performed, and the obtained supplementary rule facts or supplementary evidence data with source positions are fed back to the same large model knowledge reasoning unit; after ontology mapping, node coverage verification and logical consistency checking form a logical closed loop, a comprehensive evidence chain and retrieval application results are generated.

Description

Technical Field

[0001] This invention relates to the fields of data retrieval and artificial intelligence knowledge reasoning, and particularly to an adaptive extended retrieval method and system based on semantic ontology and large model knowledge reasoning. Background Technology

[0002] With the widespread use of heterogeneous databases, unstructured document libraries, and large language models in retrieval enhancement and generation scenarios, joint retrieval of unstructured rule knowledge and structured business data has become a crucial technical aspect. Existing retrieval methods typically rely on table headers, field names, metadata tags, or pre-defined field dictionaries to identify structured data. This makes it difficult to handle structured data with missing or ambiguous headers, inconsistent header semantics with actual content, or anonymized fields. Consequently, large models can only retrieve explicitly labeled text fragments and cannot locate structured evidence hidden in row and column values ​​and their co-occurrence relationships.

[0003] Directly inputting the entire contents of unstructured documents, knowledge graphs, and structured databases into a large model can easily exceed the model's allowed input length and introduce irrelevant data. Furthermore, when only initial vector retrieval results are provided, the large model cannot know the ontology objects, relation types, structured type features, and source locations corresponding to other indexed records, making it difficult to convert missing items in the current evidence into executable retrieval parameters. This application aims to address the following: providing a lightweight index description that locates the original records to the large model without relying on structured data headers or inputting all raw data into the large model, and performing targeted retrieval and evidence re-injection based on the uncovered nodes identified by the large model. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an adaptive extended retrieval method based on semantic ontology and large-scale model knowledge reasoning. The specific technical solution is as follows:

[0005] Based on unstructured document libraries and structured databases with missing or uncertain header semantics, construct a multimodal composite index library and lightweight index description data;

[0006] The multimodal composite index is retrieved according to the user's query instructions to obtain an initial search result set; this set, along with the user's query instructions and candidate index description records, is input into the large model knowledge reasoning unit to generate node coverage status, and to generate targeted search instructions when there are uncovered nodes.

[0007] Based on the targeted retrieval instructions, perform ontology relation traversal to obtain supplementary rule facts, or perform type feature matching and row and column context reconstruction to obtain supplementary evidence data with source location;

[0008] After obtaining supplementary rule facts or supplementary evidence data, it is injected back into the knowledge reasoning unit of the large model, and the node coverage status is updated through ontology mapping and node coverage verification. Based on the node coverage status and logical consistency verification results, a logical closed loop identifier is generated. If the loop is not closed, the instructions are updated and the retrieval is iterative. If the loop is closed, a comprehensive evidence chain and retrieval application results are generated.

[0009] Furthermore, the multimodal composite index library includes a semantic index area, a type feature index area, a relational evidence index area, and index area registration information; the lightweight index description data is generated based on the index area registration information and the index records in each index area, including index record number, ontology mapping label, relational edge type, type feature summary, source location, and retrieval method identifier; the node text of unstructured documents and the original content of cells in structured databases are stored in the multimodal composite index library and located by index record number.

[0010] Furthermore, the step of generating a multimodal composite index library and lightweight index description data based on the collected unstructured document library and structured databases with missing or uncertain header semantics includes:

[0011] Identify heading tags, paragraph tags, line break tags, numbering tags, and table cell tags in an unstructured document library, and generate document nodes according to the order of these tags in the unstructured document library;

[0012] For each document node, perform ontology semantic extraction to obtain the ontology object, entity information, attribute information, event information, time information and semantic relationship edges corresponding to the document node, and encapsulate the document nodes that meet the semantic integrity conditions into a document semantic node set;

[0013] Read structured data tables with missing or ambiguous headers, map them to an initial mixing matrix according to row and column positions, identify the type of each cell element to obtain cell type number, normalized cell value, and type confidence score; generate a structured type feature set based on the distribution of type numbers, type confidence scores, and semantic relationships between cells in the same row, column, and adjacent cells; map document semantic nodes and structured type features to the same ontology tag set to generate semantic index records and type feature index records respectively; generate relationship evidence index records based on semantic index records and type feature index records with the same ontology mapping tags or preset tag relationships; extract the index record number, ontology mapping tag, feature summary, source location, and retrieval method identifier from the above index records to generate lightweight index description data.

[0014] Furthermore, the step of performing type identification on cell elements to obtain cell type number, normalized cell value, and type confidence score includes:

[0015] The cell element is input into a set of type recognition algorithm plugins, which includes at least two of the following: time recognition plugin, numerical recognition plugin, amount recognition plugin, ratio recognition plugin, entity recognition plugin, event recognition plugin, enumeration state recognition plugin, special identifier recognition plugin, and code verification plugin.

[0016] Each type of recognition algorithm plugin identifies the cell element and outputs the candidate type number, candidate normalized value, and candidate confidence score.

[0017] The target type number is determined from multiple candidate type numbers based on the candidate confidence level, the row and column position of the cell, and the distribution of type numbers of adjacent cells;

[0018] The target type number, the corresponding normalized cell value, the type confidence score, the original row number and the original column number of the cell are bound to generate cell type records. For each row, the number of non-empty cells and the number of cell type records whose target type number belongs to the preset target type number set and whose type confidence score meets the preset confidence threshold are counted. The ratio of the two is used as the row-level type coverage, and rows whose row-level type coverage reaches the preset coverage threshold are retained and encapsulated as structured candidate data.

[0019] Furthermore, the step of generating node coverage status and generating a targeted retrieval instruction when there are uncovered nodes in the node coverage status includes:

[0020] Generate a query vector based on the user's query command, extract the text vector to be searched from the multimodal composite index, calculate the semantic matching value between the query vector and the text vector to be searched, generate a dynamic similarity boundary based on the semantic matching value, and filter to obtain the initial search result set;

[0021] Candidate index description records corresponding to the user query command and the initial search result set are selected from the lightweight index description data. The user query command, the initial search result set, the candidate index description records, and the previous round of supplementary evidence are encapsulated into model input data according to the preset field order. In the first round of search, the previous round of supplementary evidence is an empty set.

[0022] The model input data is fed into the trained large model knowledge reasoning unit, which generates a logic graph to be verified, records covered and uncovered nodes, and encapsulates the covered and uncovered nodes into node coverage states. When there are uncovered nodes, a reasoning breakpoint descriptor containing missing types, entity objects, constraints, and target ontology mapping labels is generated based on the uncovered nodes, and a targeted retrieval instruction containing ontology traversal parameters or type constraints and row and column context reconstruction strategies is output.

[0023] Further, the step of generating a query vector based on the user's query instruction and filtering the initial search result set according to dynamic similarity boundaries includes:

[0024] Obtain the user's query command, perform query normalization processing on the user's query command, and use the processed text as the query text to be vectorized.

[0025] The text vector generation unit used to generate the vector of the text being inspected is invoked to map the query text to be vectorized into a query vector;

[0026] Traverse the semantic index area of ​​the multimodal composite index library, extract the fixed-length semantic numerical vectors stored therein as the text vectors to be checked, and calculate the semantic matching value between the query vector and each text vector to be checked based on cosine similarity.

[0027] Sort all semantic matching values ​​in ascending order to form a similarity sequence. Calculate the upper quartile and median values ​​of the similarity sequence and use the difference between the upper quartile and median values ​​as the discrete span.

[0028] The number of entity words, condition phrases, and effective words in the query text to be vectorized is counted, and the query constraint density is generated based on the number of entity words, condition phrases, and effective words.

[0029] Dynamic similarity boundaries are generated based on query constraint density, upper quartile values, and discrete span. Semantic index records are then filtered based on these boundaries, and the retained node text content, semantic matching values, matching levels, index record numbers, ontology mapping tags, and source locations are encapsulated into an initial search result set.

[0030] Further, the step of performing extended retrieval according to the targeted retrieval instruction, generating supplementary rule facts or supplementary evidence data and injecting it back into the large model knowledge reasoning unit to obtain the logical closed-loop identifier includes:

[0031] The instruction type of the targeted retrieval instruction is parsed; when the instruction type is supplementary rule facts, an extended retrieval is performed in the semantic index area and the relational evidence index area according to the ontology traversal parameters to obtain the supplementary rule facts; when the instruction type is supplementary evidence facts, the type matching degree is calculated according to the type constraints and the structured type feature set to determine the target structured data area.

[0032] After determining the target structured data region, the cell containing the target business element is extracted according to the target structured data region and the row and column context reconstruction strategy. The associated entities, associated attributes, associated events, associated time information and associated identification information are extracted in the row, column and adjacent areas of the target business element. The target business element, associated information and their source location are encapsulated as supplementary evidence data.

[0033] Supplementary rule facts or supplementary evidence data are injected back into the large model knowledge reasoning unit, which then maps them to the corresponding ontology nodes of the logic graph to be verified, performing node coverage verification and logic consistency verification. When all nodes to be verified are covered and there are no logical conflicts, the logic loop closure flag is set to closed; otherwise, it is set to unclosed.

[0034] Furthermore, the step of extracting target business elements and their associated information based on the target structured data region and row / column context reconstruction strategy and encapsulating them into supplementary evidence data includes:

[0035] Based on the cell type number and type confidence in the target structured data area, read the cell elements that meet the type constraints and whose type confidence meets the preset confidence threshold, and use them as target business elements. Record the original row number and original column number of each target business element in the structured database as the source row number and source column number.

[0036] Based on the row and column context reconstruction strategy, the adjacent cells are scanned along the row direction, column direction and preset neighborhood direction of the target business element, with the cell corresponding to the source row number and source column number as the center.

[0037] Based on the type number corresponding to the adjacent cells and the ontology mapping label obtained through ontology mapping, they are extracted into associated entities, associated attributes, associated events, associated time information, or associated identification information, respectively.

[0038] For each target business element, it is bound to the associated entity, associated attribute, associated event, associated time information and associated identifier information extracted from the same row and column context, generating a row and column evidence record containing the source row number and source column number, and all row and column evidence records are encapsulated into supplementary evidence data in the order of source position.

[0039] Furthermore, the step of updating the targeted retrieval instruction or generating a comprehensive evidence chain based on the logical closed-loop identifier and outputting the retrieval application results includes:

[0040] When the logical closed-loop flag indicates that the loop is not closed, the reasoning breakpoint descriptor, the executed directional retrieval instructions, and the supplementary rule facts or supplementary evidence data obtained in this round are input into the large model knowledge reasoning unit. The large model knowledge reasoning unit excludes the retrieval parameters that have been executed and have not generated new evidence, and generates updated directional retrieval instructions. When the logical closed-loop flag indicates that the loop is closed, the confidence level corresponding to the rule evidence is generated based on the initial retrieval result set and the supplementary rule facts and supplementary evidence data that have been obtained. The confidence level is then combined with the preset confidence threshold to generate a comprehensive evidence chain.

[0041] Under the constraints of a pre-set controlled inference instruction set, a multi-step verification inference path is generated based on the user's query instruction and the comprehensive evidence chain. The evidence consistency index between the multi-step verification inference path and the comprehensive evidence chain is calculated. When the evidence consistency index is equal to the absolute consistency threshold, the retrieval application results are output according to the structured template of the response results, which includes the verification conclusion field, the rule basis field, and the factual evidence field.

[0042] Furthermore, the step of outputting retrieval application results under the constraints of a controlled inference instruction set includes:

[0043] A controlled inference instruction set and structured output field constraints are loaded into the large model knowledge inference unit; the large model knowledge inference unit adopts a trained generative language model, and the controlled inference instruction set limits the evidence citation range, targeted retrieval instruction fields, application result fields, sampling temperature parameters, decoding methods, and maximum output length;

[0044] The user query command, the comprehensive evidence chain, and the node coverage relationship between the node and the index record number established during the logical closure are input into the large model knowledge reasoning unit. Under the constraints of the controlled reasoning instruction set, the corresponding relationship verification is performed to generate a multi-step verification reasoning path.

[0045] The multi-step verification reasoning path is sequence-segmented, and the target business elements, related entities, related attributes, related events, related time information, related identification information and index record numbers contained therein are extracted to form a generated feature set;

[0046] The comprehensive evidence chain is segmented according to the same rules as the multi-step verification reasoning path, and the target business elements, node text content, semantic matching values, related entities, related attributes, related events, related time information, related identification information, index record number and source location contained therein are extracted to form a source feature set;

[0047] The evidence consistency index is calculated based on the generated feature set and the source feature set. When the evidence consistency index is equal to the absolute consistency threshold, the structured template of the response result is called to output the retrieval application result. When the evidence consistency index is less than the absolute consistency threshold, the sampling temperature parameter is reduced or a deterministic decoding method is adopted to regenerate the multi-step verification reasoning path.

[0048] The present invention also provides an adaptive extended retrieval system based on semantic ontology and large model knowledge reasoning, which is used to implement the above-mentioned adaptive extended retrieval method based on semantic ontology and large model knowledge reasoning. The system includes a composite index module, a large model knowledge reasoning module, a targeted retrieval module, and an application output module.

[0049] The composite index module is used to generate a multimodal composite index library and lightweight index description data based on an unstructured document library and a structured database with missing or uncertain headers, and to retrieve the multimodal composite index library according to the user's query instructions, generating an initial search result set and candidate lightweight index description records.

[0050] The large model knowledge reasoning module is used to receive user query instructions, initial retrieval result set, candidate lightweight index description records, supplementary rule facts and supplementary evidence data, perform node coverage verification, ontology mapping and logical consistency verification, output node coverage status, reasoning breakpoint descriptor, multi-step verification reasoning path and logical closed loop flag, and output directional retrieval instructions when there are uncovered nodes.

[0051] The targeted retrieval module is used to parse the targeted retrieval instruction, perform semantic extension retrieval based on the ontology traversal parameters to obtain supplementary rule facts, or extract supplementary evidence data with source location based on type constraints and row and column context reconstruction strategies, and inject the supplementary rule facts or supplementary evidence data back into the large model knowledge reasoning module.

[0052] The application output module is used to trigger the directional retrieval instruction update and iterative retrieval when the logical closed-loop flag indicates that the loop is not closed, generate a comprehensive evidence chain when the logical closed-loop flag indicates that the loop is closed, and output the retrieval application results under the constraints of the controlled reasoning instruction set.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0054] This invention encapsulates the index record number, ontology mapping label, type feature summary, source location, and retrieval method from semantic index records, type feature index records, and relational evidence index records into lightweight index description data. This allows large models to know the available index range without receiving all the original content of unstructured documents and structured databases, reducing the length of model input and irrelevant data.

[0055] This invention inputs user query commands, initial search result sets, and lightweight index description data into the knowledge reasoning unit of a large model, enabling the large model to identify rule fact gaps or evidence fact gaps based on node coverage status, and output ontology traversal parameters, type constraints, and row and column context reconstruction strategies that can be parsed by the search engine, thereby realizing targeted extended retrieval driven by knowledge reasoning results.

[0056] This invention injects supplementary rule facts or row and column evidence records with source location back into the knowledge reasoning unit of the large model. It generates a logical closed loop identifier through ontology mapping, node coverage verification and logical consistency verification. When the loop is not closed, it excludes invalid search parameters and continues to iterate, thereby reducing repeated searches and evidence chain interruptions.

[0057] After forming a logical closed loop, this invention uses a comprehensive evidence chain, a controlled reasoning instruction set, and an evidence consistency index to constrain the retrieval of application results, so that the verification conclusions can be mapped to specific rule bases, factual evidence, and source locations, reducing the amount of uncited data entering the output results. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart illustrating the principle of the adaptive extended retrieval method based on semantic ontology and large model knowledge reasoning of the present invention.

[0060] Figure 2 This is a schematic diagram of the data structure of the document semantic node set of the present invention;

[0061] Figure 3 This is a schematic diagram of the data structure of the multimodal composite index library and lightweight index interface of the present invention;

[0062] Figure 4 This is a functional block diagram of the adaptive extended retrieval system based on semantic ontology and large model knowledge reasoning of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Example 1:

[0065] Please see Figure 1 As shown, this embodiment provides an adaptive extended retrieval method based on semantic ontology and large model knowledge reasoning, including:

[0066] Step S1000: Based on the collected unstructured document library and the structured database with missing or uncertain headers, generate a document semantic node set and a structured type feature set, respectively, and generate a multimodal composite index library and lightweight index description data based on the document semantic node set and the structured type feature set.

[0067] Specifically, this step converts the unstructured document library into a set of document semantic nodes containing ontology objects, entity information, attribute information, event information, time information, and semantic relationship edges. It also converts the cell content in the structured database into a set of structured type features containing type numbers, normalized values, type confidence, and row and column context relationships. Subsequently, the document semantic nodes and structured type features are mapped to the same ontology tag set to generate semantic index records, type feature index records, and relationship evidence index records. Index metadata that does not contain complete original content is extracted from each index record to form lightweight index description data for use by the knowledge reasoning unit of the large model.

[0068] Further, step S1000 includes:

[0069] Step S1100: Based on document semantic extraction rules, the unstructured document library is filtered to generate a set of document semantic nodes; based on the row-level type coverage verification mechanism, the structured database with missing or uncertain headers is processed to output structured candidate data.

[0070] In the specific implementation process, this step first establishes communication connections with an unstructured document library and a structured database with missing or ambiguous headers. The unstructured document library refers to a collection of text data not organized in a fixed row and column format, originating from pre-accessed rule files, policy documents, explanatory documents, or audit text files in external business systems; its data content consists of text containing natural language sentences, paragraphs, table cells, or numbered content. The structured database with missing or ambiguous headers refers to structured records where data content is stored in a row and column format, but lacks column names, field descriptions, or schema tags, or has ambiguous headers, inconsistent headers with actual content, or fields that have been anonymized.

[0071] Next, the heterogeneous data separation logic is executed. This logic is used to establish unstructured text processing channels and structured type recognition channels respectively, so that ontology objects, entities, attributes, events and time expressions in unstructured text enter the document semantic extraction process, and structured cell content enters the multi-type recognition and row and column context reconstruction process.

[0072] The specific execution logic of the heterogeneous data separation logic is as follows: On the one hand, for the unstructured document library, node splitting processing based on document format tags is performed to obtain a set of document nodes. Specifically, the node splitting processing based on document format tags includes: identifying heading tags, paragraph tags, line break tags, numbering tags, and table cell tags in the unstructured document library, and generating document nodes according to the order of these tags in the unstructured document library. Each document node includes at least a node level, node sequence number, node text content, ontology object set, entity set, attribute set, event set, time information set, and semantic relationship edge set.

[0073] See Figure 2 , Figure 2 This is a schematic diagram of the data structure of the document semantic node set of the present invention.

[0074] Next, the document node set is traversed, and ontology semantic extraction is performed on each document node to obtain the corresponding ontology object, entity information, attribute information, event information, time information, and semantic relationship edges. The ontology semantic extraction includes: identifying company names, personnel names, project names, location names, product names, equipment names, or business object names through a named entity recognition model; identifying amount, quantity, proportion, status, range, threshold, or enumerated attributes through an attribute extraction model; identifying approval, transaction, delivery, change, exception, confirmation, or conclusive events through an event extraction model; and identifying date, period, deadline, or relative time through a time expression recognition model. Semantic relationship edges are then generated based on the syntactic dependencies or positional adjacency relationships between entities, attributes, events, and time information. When a document node contains at least two of the following: ontology object, entity information, attribute information, event information, time information, or semantic relationship edges, and the corresponding recognition confidence meets a preset semantic integrity threshold, the document node is retained and encapsulated into a document semantic node set.

[0075] On the other hand, for structured databases with missing or ambiguous headers, the stored structured data tables are read and mapped to an initial mixing matrix according to their row and column positions. Each element in the initial mixing matrix corresponds to a cell in the structured data table. Since the cell content in the structured database may be numerical values, amounts, proportions, times, enterprise entities, personnel entities, project entities, equipment entities, event descriptions, enumerated states, special identifiers, or any combination of strings and numbers, each cell element in the initial mixing matrix is ​​traversed, and a set of type recognition algorithm plugins is called to perform type recognition on the cell content.

[0076] To remove invalid rows, de-identified label rows, or completely empty rows remaining in the structured data table, this step establishes a row-level type coverage verification mechanism. Cell elements are input into at least two type recognition algorithm plugins. Each plugin outputs candidate type numbers, candidate normalized values, and candidate confidence scores. The target type number is determined based on the candidate confidence score, the cell's row and column position, and the distribution of type numbers in adjacent cells. The target type number, normalized cell value, type confidence score, original row number, and original column number are bound to a cell type record. For any row, the number of non-empty cells and the number of records whose target type numbers belong to a preset target type number set and whose type confidence scores meet a preset confidence threshold are counted. The ratio of these two is used as the row-level type coverage. The preset target type number set consists of the type numbers corresponding to the type recognition algorithm plugins currently enabled in the structured data source. When the number of non-empty cells is zero, the row-level type coverage is set to zero. If the row-level type coverage reaches a preset coverage threshold, the row is retained; otherwise, it is discarded. Retained rows still use the original row number and original column number from the cell type record.

[0077] In this embodiment, all preset thresholds are calibrated on an independent validation sample set. Specifically, candidate thresholds are iterated in steps of 0.01, with the highest F1 score and lowest false positive and false negative rates for identification, mapping, or matching results used as the determining criteria. For example, the preset semantic integrity threshold can be 0.70, the preset reliability threshold for cell type recognition scenarios can be 0.80, the preset coverage threshold can be 0.60, the preset mapping threshold can be 0.75, the preset grading threshold can be 0.10, the preset matching threshold can be 0.75, and the preset reliability threshold for rule-based evidence matching scenarios can be 0.80. The semantic integrity and coverage thresholds are used to remove data with insufficient information, the mapping, grading, and matching thresholds are used to balance recall and false matches, and the confidence threshold is used to exclude records with insufficient reliability in identification or evidence matching. The above values ​​are a set of implementable examples. In actual deployment, they can be re-determined using the same calibration method based on data noise and false positive / false negative requirements, and remain unchanged during operation of the same index version.

[0078] Step S1200: Statistically analyze the distribution of type numbers, type confidence, normalized values, and semantic relationships in each row, column, and adjacent area of ​​the structured candidate data to obtain column-level type feature descriptors and row-level type feature descriptors, and encapsulate them into a structured type feature set.

[0079] In the specific implementation process, this step first loads the structured candidate data into memory, and then traverses the structured candidate data along the column direction, row direction, and preset neighborhood direction.

[0080] For any column in the structured candidate data, cell type records are extracted from the beginning, middle, and end of the column to form a preset sampling window. When the number of valid cells in the column does not exceed the size of the preset sampling window, all valid cell records are used. The occurrence frequency of each target type number, the mean type confidence, the distribution of normalized cell values, the consistency of special identifier format, the proportion of entity types, the proportion of time expressions, and the proportion of event types in the sampled records are statistically analyzed to form a column-level type feature descriptor.

[0081] Subsequently, for any row in the structured candidate data, the co-occurrence relationship of the target type number, the adjacency relationship between entities and attributes, the adjacency relationship between events and time information, and the adjacency relationship between special identifiers and business objects are statistically analyzed to form a row-level type feature descriptor.

[0082] Finally, all rows and columns of the structured candidate data are traversed, and all column-level and row-level type feature descriptors are merged and encapsulated in order according to their row and column numbers in the structured candidate data, outputting a complete set of structured type features.

[0083] The above statistical results constitute a structured type feature set. When a table header is missing, the header is ambiguous, or the field description is inaccurate, subsequent steps filter the structured data area corresponding to the target business element based on the type characteristics of the cell content, the distribution of normalized values, and the row and column context.

[0084] Step S1300: Generate a multimodal composite index library based on the document semantic node set and the structured type feature set, and extract index metadata from the index records of the multimodal composite index library to generate lightweight index description data.

[0085] In the specific implementation process, this step calls the text vector generation unit, ontology mapping unit, index record generation unit, and lightweight index encapsulation unit. The text vector generation unit uses a pre-trained text embedding model to map node text into fixed-length semantic numerical vectors; the ontology mapping unit sets a unified set of ontology labels, with each ontology label recording the label number, label category, allowed corresponding cell type number, normalization value rule, and adjacency relationship type; the ontology mapping unit maps ontology objects, entities, attributes, events, and times in the document semantic nodes into ontology labels, and determines candidate ontology labels for the structured type features based on the type number distribution, normalization value distribution, and row-column co-occurrence relationship in the structured type features; the index record generation unit establishes each index area, and the lightweight index encapsulation unit selects fields from the index records for model inference.

[0086] Traverse the document semantic node set, and read the node text content, ontology object, entity information, attribute information, event information, time information and semantic relationship edges for each document semantic node; generate a fixed-length semantic numerical vector by the text vector generation unit, and generate ontology mapping labels, entity mapping labels, attribute mapping labels, event mapping labels and time mapping labels by the ontology mapping unit.

[0087] A composite index storage unit is constructed, comprising a semantic index area, a type feature index area, and a relational evidence index area. The semantic index area stores the semantic index records corresponding to the semantic nodes of the document; the type feature index area stores the type feature index records corresponding to the structured type feature sets; and the relational evidence index area stores the relational evidence index records formed between the semantic nodes of the document and the cell type records based on ontology objects, entities, attributes, events, time, and identifiers.

[0088] For each document semantic node, an index record number is assigned to it. The index record number, node text content, fixed-length semantic numerical vector, ontology mapping label, semantic relation edge, and source document location are combined into a semantic index record and written into the semantic index area. The index record number is used to locate the node text content and its source location.

[0089] For each column-level and row-level type feature descriptor in the structured type feature set, its type number distribution, normalized value distribution, and semantic association are input into the ontology mapping unit and matched with the entity, attribute, event, time, and identifier definitions in the ontology label set. For each ontology label, the proportion of consistent type numbers, the proportion of cells satisfying the normalized value rule, and the proportion of consistent adjacency relationship types are calculated and summed according to a preset weight of non-negative and summing to 1 to obtain the label matching value. The ontology label with the highest label matching value that reaches the preset mapping threshold is taken as the ontology mapping label, and the label matching value is taken as the mapping confidence. If the preset mapping threshold is not reached, a label to be confirmed is written. Type record numbers are generated based on row and column numbers. The type record number, row and column number, type number distribution, type confidence distribution, normalized value distribution summary, semantic association, ontology mapping label, mapping confidence, and the source position of the corresponding cell type record are combined into a type feature index record and written into the type feature index area.

[0090] Based on the same ontology mapping tags in the semantic index records and the type feature index records, or the pre-defined tag relationships between entities and attributes, entities and events, events and times, and objects and identifiers in the ontology tag set, determine the correspondence between the two types of index records; assign a relationship record number to each correspondence, and combine the relationship record number, the index record numbers at both ends, the relationship edge type, and the source positions at both ends into a relationship evidence index record and write it into the relationship evidence index area.

[0091] Generate index area registration information. The index area registration information records the storage location, data source, record structure, and retrieval method of each index area, including the index area name, index record fields, record number generation rules, vector similarity retrieval interface, type matching retrieval interface, and relation edge traversal interface.

[0092] The lightweight index encapsulation unit extracts the index record number, ontology mapping label, relation edge type, type number distribution summary, type confidence summary, normalized value distribution summary, source location, and retrieval method identifier from each index record to generate a basic lightweight index description record; a null value identifier is written when a certain type of index record does not have a corresponding field. The basic lightweight index description record stores the positioning pointer to the original index record, without copying the complete text of the node or the original content of the cell.

[0093] All basic lightweight index description records are partitioned and registered according to semantic index area, type feature index area, and relational evidence index area to form lightweight index description data, and a version identifier and index record number correspondence relationship is established with the multimodal composite index library; the type feature index records with the pending confirmation label also retain the type number distribution summary, normalized value distribution summary, and source location, so that the large model knowledge reasoning unit can generate type constraints based on the uncovered nodes; when the index records in the multimodal composite index library are updated, the corresponding basic lightweight index description records are updated synchronously.

[0094] See Figure 3 , Figure 3 The data structure relationship between the multimodal composite index library and the lightweight index interface is illustrated. The lightweight index interface provides candidate index description records and location pointers to the large model knowledge reasoning unit; the retrieval server reads the original index records from the multimodal composite index library based on the location pointers and performs the actual retrieval.

[0095] The multimodal composite index library uniformly stores unstructured semantic indexes, structured type feature indexes, and relational evidence indexes; the lightweight index description record only stores the index capability description and positioning fields. The large model knowledge reasoning unit selects the retrieval path based on this record, and the retrieval server then reads the original record according to the index record number.

[0096] Step S2000: Retrieve the multimodal composite index library according to the user query command and generate an initial search result set; filter candidate index description records from the lightweight index description data, input the user query command, the initial search result set and the candidate index description records into the large model knowledge reasoning unit, generate the node coverage status, and output the reasoning breakpoint descriptor and the directional search command when there are uncovered nodes.

[0097] Specifically, this step first uses text vector similarity and dynamic similarity boundaries to obtain an initial retrieval result set. Then, based on the index record number and ontology mapping label in the initial retrieval result set, candidate index description records related to the current query are selected from the lightweight index description data. The large model knowledge reasoning unit receives the task objective, initial evidence, candidate index description records, and supplementary evidence from the previous round, identifies nodes not yet covered by the current evidence, and converts these nodes into targeted retrieval instructions that the retrieval server can execute.

[0098] Further, step S2000 includes:

[0099] Step S2100, according to the user's query command Generate query vectors Extracting multimodal composite index library The detected text vector Calculate the query vector and the detected text vector semantic matching value And based on semantic matching values Generate dynamic similarity boundaries The initial search result set was obtained by filtering. .

[0100] In the specific implementation process, this step first obtains the user query command passed from the external user access terminal. The user query command Natural language text containing the specific verification intent and entities. Regarding the user query instruction. The query normalization process involves organizing non-business expressions, numerical unit expressions, and entity condition expressions in the query text, and then using the processed text as the query text to be vectorized. Organizing non-business expressions, numerical unit expressions, and entity condition expressions in the query text includes: deleting additional statements that do not affect business judgment; converting similar numerical values, units, or time formats into a consistent format; and maintaining the correspondence between the verification objects and their constraints.

[0101] To ensure that the user query command and the semantic index records from the multimodal composite index library in step S1300 can be similarly calculated using the same vector representation, the same text vector generation unit as in step S1300 is invoked to map the user query command into a multidimensional continuous floating-point vector, generating a query vector. The dimension of the query vector must be consistent with the dimension of the fixed-length semantic numerical vector from step S1300.

[0102] Next, the semantic index area of ​​the multimodal composite index library is traversed to extract all fixed-length semantic numerical vectors stored therein, which are used as the searched text vectors. For each searched text vector, the vector similarity calculation unit is invoked to calculate its semantic matching value with the query vector. The semantic matching value is calculated based on cosine similarity, and its calculation logic is as follows: first, the inner product between the query vector and the searched text vector is calculated; then, the norm 2 modulus of the query vector and the searched text vector are calculated respectively; next, the product of the norm 2 modulus of the query vector and the norm 2 modulus of the searched text vector is calculated; finally, the inner product is divided by the product to obtain the semantic matching value between the user query command and the searched text vector in the vector representation space. The semantic matching value represents the degree of semantic relevance between the node text content corresponding to the searched text vector and the user query command.

[0103] To mitigate the impact of varying query lengths, query constraint densities, and differences in index paragraph distribution when filtering search results solely based on fixed thresholds, this step establishes a dynamic similarity boundary generation mechanism. This mechanism takes the semantic index region as input and automatically generates the dynamic similarity boundary corresponding to the current query based on the overall distribution of semantic matching values ​​obtained between the query vector and all retrieved text vectors.

[0104] Sort all semantic matching values ​​in ascending order to form a similarity sequence; calculate the upper quartile and median values ​​of the similarity sequence, and use the difference between the upper quartile and the median value as the discrete span.

[0105] The number of entity words, conditional phrases, and effective tokens in the query text to be vectorized are counted and denoted as follows: , and Entity terms are terms representing the objects being checked; conditional phrases are phrases representing time, amount, location, status, scope, or comparative relationships; and valid tokens are tokens retained after query normalization. The query constraint density is calculated based on the above quantities. .

[0106]

[0107] The dynamic similarity boundary is calculated based on the upper quartile value, discrete span, and query constraint density. The calculation formula is as follows:

[0108]

[0109] in, The upper quartile value. For discrete span, .by The denominator of the query constraint density is used to avoid undefined results when the number of effective tokens is zero; 1 is used as the upper limit of the dynamic similarity boundary to ensure that it does not exceed the upper limit of cosine similarity.

[0110] The semantic index records are sorted from highest to lowest semantic matching value, and records with semantic matching values ​​greater than or equal to the dynamic similarity boundary are retained. When the number of retained records is less than the preset minimum number of candidates, the records with higher rankings are added from the sorting results until the preset minimum number of candidates is reached. When the total number of semantic index records is insufficient, all records are retained.

[0111] The preset minimum and maximum number of candidates are determined based on the number of input terms allowed by the large model knowledge reasoning unit and the average number of terms in the indexed records. When the number of candidates to be retained exceeds the preset maximum number, records are truncated according to semantic matching values, and duplicate records with sequentially adjacent nodes or hierarchical relationships in the same document are deduplicated, ultimately retaining no more than the preset maximum number of candidates.

[0112] Matching levels are generated based on the difference between the semantic matching value and the dynamic similarity boundary: records with a difference reaching a preset grading threshold are marked as core matching records; records with a difference less than the preset grading threshold but not less than zero are marked as general matching records; and records added to meet a preset minimum number of candidates are marked as supplementary retrieval records. Supplementary retrieval records are used to identify uncovered nodes and select subsequent retrieval paths, but are not used as the sole factual basis for verification conclusions.

[0113] The index record number, node text content, semantic matching value, matching level, ontology mapping tag, and source location of the retained records are encapsulated into an initial search result set.

[0114] Step S2200: Select candidate index description records from the lightweight index description data, encapsulate the user query command, initial retrieval result set, candidate index description records and previous round of supplementary evidence into model input data, generate node coverage status by the large model knowledge reasoning unit, and generate reasoning breakpoint descriptors when there are uncovered nodes.

[0115] This step deploys a trained large-scale model knowledge reasoning unit on the inference server. The large-scale model knowledge reasoning unit employs a generative language model with structured text understanding and generation capabilities; its input is configured with fixed field templates, and its output is configured with a field integrity validator. The fixed field templates include a task target area, an initial evidence area, a lightweight index area, a supplementary evidence area, and a retrieval history area.

[0116] The task target area contains the user's query command; the initial evidence area contains the index record number, node text content, semantic matching value, matching level, ontology mapping label, and source location of the initial search result set; the lightweight index area contains candidate index description records that have the same ontology mapping label or adjacent relationship edges as the initial search result set; the supplementary evidence area contains an empty set during the first round of reasoning, and supplementary rule facts or row and column evidence records obtained in the previous round in subsequent rounds; the search history area contains the executed targeted search commands and whether new evidence has been obtained.

[0117] The five data areas are serialized according to a fixed field order to generate model input data, which is then input into the large model knowledge reasoning unit. The candidate index description records input into the large model only contain the index record number, ontology mapping label, relation edge type, type feature summary, source location, and retrieval method identifier. When the large model needs raw text or cell values, it outputs a targeted retrieval instruction, which is then read by the retrieval server based on the index record number.

[0118] The large model knowledge reasoning unit first identifies the objects to be verified, judgment conditions and their logical dependencies from the task target area, and generates a logic graph to be verified; then it maps the rule facts and evidence facts in the initial evidence area and the supplementary evidence area to the corresponding nodes of the logic graph to be verified, and reads the available retrieval methods corresponding to each node in the lightweight index area.

[0119] The logic graph to be verified includes business entity nodes, condition nodes, and logical dependency edges between nodes. Business entity nodes represent objects to be verified, condition nodes represent thresholds, time, status, range, or existence conditions, and logical dependency edges represent the value, attribution, timing, comparison, or negation relationship between the objects to be verified and the judgment conditions; each node also records the corresponding ontology mapping label and evidence citation record number.

[0120] The chain-of-thought knowledge reasoning in this embodiment refers to the knowledge reasoning process in which the large-scale model knowledge reasoning unit gradually generates retrieval decisions based on task nodes and their evidence coverage. It is represented by an outputtable multi-step knowledge reasoning state sequence. This state sequence sequentially records task nodes, covered nodes, uncovered nodes, evidence record numbers, and retrieval decisions. The large-scale model knowledge reasoning unit generates the state sequence according to structured output fields. The field integrity checker only submits the index record number, relation edge type, and type constraint conditions that correspond to the candidate index description record to the retrieval server for execution.

[0121] The large model knowledge reasoning unit generates a set of known fact nodes based on the initial evidence area and the supplementary evidence area. Each known fact node includes the fact source, fact type, ontology mapping label, entity attribute, state value, constraint range, evidence record number, and source location. The fact source is used to distinguish between the initial retrieval results, supplementary rule facts, and row and column evidence records. The evidence record number adopts the index record number of the corresponding semantic index record, type feature index record, or relational evidence index record.

[0122] For each business entity node and condition node in the logic graph to be verified, the large model knowledge reasoning unit compares its ontology mapping label, entity attributes and constraint scope with the set of known fact nodes; if there is a known fact node that can simultaneously satisfy the ontology mapping label and constraint scope, the corresponding node is marked as a covered node and bound with the evidence reference record number; otherwise, it is marked as an uncovered node.

[0123] After all nodes have been verified, the correspondence between covered nodes, uncovered nodes, and their evidence reference records is encapsulated as a node coverage state. If there are no uncovered nodes, the logical loop closure flag is set to closed and the process proceeds to the closed loop branch of step S4100; if there are uncovered nodes, it is further determined whether the node lacks rule facts or structured evidence facts used to verify the rule conditions.

[0124] When an uncovered node corresponds to a business rule, judgment threshold, applicable condition, or conclusion triggering condition, the missing type is determined to be "rule fact not covered"; when an uncovered node has rule conditions but lacks target business elements, status values, event records, or time records, the missing type is determined to be "evidence fact not covered".

[0125] An inference breakpoint descriptor is generated based on the uncovered nodes. The inference breakpoint descriptor includes the missing type, entity object, constraint condition, target ontology mapping label, relationship type, cited evidence record number, and available retrieval method identifier. After the field integrity validator confirms that the required fields are complete, it is output to step S2300.

[0126] In step S2300, the large model knowledge reasoning unit generates a targeted retrieval instruction based on the reasoning breakpoint descriptor and the candidate index description record. The targeted retrieval instruction includes ontology traversal parameters, or includes type constraints and row and column context reconstruction strategies.

[0127] In the specific implementation process, the large model knowledge reasoning unit reads the missing type, entity object, constraint condition, target ontology mapping label and available retrieval method identifier from the reasoning breakpoint descriptor, and reads candidate index description records that are consistent with the target ontology mapping label or connected by preset relation edges from the lightweight index area.

[0128] Based on the missing type field and the retrieval method identifier corresponding to the candidate index description record, the large model knowledge reasoning unit selects either the rule fact supplementation branch or the evidence fact supplementation branch, and generates corresponding targeted retrieval instructions under the constraints of the structured output fields.

[0129] When the rule facts are not covered, the system enters the rule fact supplementation branch. The large model knowledge reasoning unit selects the semantic index record number or the relation evidence index record number from the candidate index description records as the starting record for traversal, and determines the allowed relation edge types based on the ontology mapping label and association type corresponding to the uncovered node.

[0130] The entity objects, constraints, ontology mapping labels, and allowed traversal relation edge types are combined in a fixed order to form the extended query text; the extended query text is used to describe the facts of the rules to be supplemented and does not contain factual conclusions generated by the large model itself.

[0131] The same text vector generation unit as in step S1300 is invoked to map the extended query text into a multidimensional extended word vector, wherein the multidimensional extended word vector has the same dimension as the text vector being examined in the semantic index area.

[0132] Set a similarity decay coefficient less than 1 and greater than 0, and multiply the dynamic similarity boundary obtained in step S2100 with the similarity decay coefficient to obtain the extended similarity boundary; the similarity decay coefficient is determined by the retrieval verification samples labeled with relevant and irrelevant results, with the selection condition being a low false detection rate when the preset recall rate is achieved.

[0133] The traversal start record number, allowed relation edge types, preset traversal depth, multidimensional extended word vectors, and extended similarity boundaries are combined into ontology traversal parameters, and the instruction type is marked as rule fact supplement.

[0134] When the evidence facts are not covered, the system enters the evidence fact supplementation branch. The large model knowledge reasoning unit selects the type feature index record number and the relation evidence index record number from the candidate index description records based on the entity objects, constraints, and target ontology mapping labels in the reasoning breakpoint descriptor.

[0135] Based on the type number distribution summary, normalized value distribution summary, and relation edge type in the lightweight index area, the large model knowledge reasoning unit generates candidate type numbers, candidate normalized value rules, candidate ontology tags, candidate attribute types, candidate event types, and candidate time constraints for the business elements to be verified.

[0136] The field integrity validator checks whether the candidate type number, candidate ontology label, and source index record number exist, and checks whether each candidate field belongs to the preset type number set and ontology label set. If the verification fails, it returns the missing field or out-of-bounds field to the large model knowledge reasoning unit, so that it can regenerate the candidate type feature results.

[0137] After the verification is passed, the candidate type number, candidate normalization value rule, candidate ontology label, candidate attribute type, candidate event type and candidate time constraint are combined into type constraint conditions; the fields of the type constraint conditions correspond to the structured type feature set in step S1200.

[0138] Select the type recognition algorithm plugin and the relationship template based on the relationship type in the inference breakpoint descriptor, and set the in-row extraction range, in-column extraction range and neighborhood extraction range. Combine the selected type recognition algorithm plugin, relationship template and extraction range into a row and column context reconstruction strategy.

[0139] The type feature index record number, relation evidence index record number, type constraint conditions, and row and column context reconstruction strategy are combined into an evidence fact supplementation instruction. Then, the field integrity verifier checks the validity of the required fields and record numbers of the ontology traversal parameters or evidence fact supplementation instruction. After the verification is passed, it is output to step S3000 as a targeted retrieval instruction via the data bus.

[0140] The retrieval control fields output by the large model knowledge reasoning unit are limited to index record number, relation edge type, type constraint conditions, and row and column extraction range. After verifying the above fields, the retrieval server reads the corresponding original records and performs matching. The free text generated by the large model is not written into the comprehensive evidence chain as supplementary rule facts or supplementary evidence data.

[0141] In step S3000, when there are uncovered nodes in the node coverage state, the directional retrieval instruction is parsed and ontology relation traversal or structured type feature retrieval is performed to generate supplementary rule facts or supplementary evidence data with source location; the retrieval results are injected back into the large model knowledge reasoning unit to update the node coverage state and obtain the logical closed loop identifier.

[0142] Specifically, the targeted retrieval module first checks the instruction type and index record number in the targeted retrieval instructions; for rule fact supplementation instructions, it performs ontology traversal and vector similarity retrieval in the semantic index area and relational evidence index area; for evidence fact supplementation instructions, it performs type matching in the type feature index area and performs row and column context reconstruction in the structured candidate data; then, it feeds the retrieval results with source location back into the large model knowledge reasoning unit to complete node coverage verification and logical consistency verification.

[0143] Further, step S3000 includes:

[0144] Step S3100: Perform semantic ontology extended retrieval or type feature matching according to the instruction type of the targeted retrieval instruction to obtain supplementary rule facts or target structured data regions.

[0145] First, the instruction type, index record number, and retrieval parameters in the targeted retrieval instruction are parsed, and the existence of the index record number in the current multimodal composite index library version is verified. If the index record number does not exist or the version is inconsistent, an instruction failure flag is returned, and the targeted retrieval instruction is regenerated in step S2300.

[0146] When the instruction type is supplementary rule facts, starting from the traversal start record number in the ontology traversal parameters, the system traverses along the allowed traversal relation edge types within the relation evidence index area to a preset depth; it reads the traversed semantic index records and calculates the similarity between their fixed-length semantic numerical vectors and multi-dimensional extended word vectors; it retains the node text content, index record number, ontology mapping label, and source location of nodes whose similarity satisfies the extended similarity boundary, thus forming supplementary rule facts.

[0147] When the instruction type is supplementary evidence facts, the candidate type number, candidate normalization value rule, candidate ontology label, candidate attribute type, candidate event type and candidate time constraint are read from the type constraints. The structured type feature set is traversed, and the type matching degree is calculated according to the consistency of type number, type confidence, normalization value similarity, ontology mapping consistency and row and column adjacency relationship.

[0148] The candidate data regions involved in the calculation are sorted from high to low according to their type matching degree to obtain the type matching degree ranking result, and the type matching degree corresponding to the first ranked candidate data region is read as the highest matching degree.

[0149] The highest matching degree is compared with the preset matching degree threshold; when the highest matching degree is greater than or equal to the preset matching degree threshold, the row and column range of the corresponding candidate data region in the structured candidate data is extracted.

[0150] The extracted row and column ranges and their corresponding cell type records are encapsulated into the target structured data area; when the highest matching degree is less than the preset matching degree threshold, a no-match flag is returned, and the no-match flag is written into the search history area as the result of this round of search execution.

[0151] The targeted retrieval module selects either the ontology relation traversal path or the headerless structured evidence location path according to the instructions output by the large model and verified for field integrity. The retrieval scope is limited by the index record number, relation edge type, type constraints, and row and column extraction range in the instructions. Each retrieval result retains the index record number and source location.

[0152] Step S3200: When the target structured data area is obtained, the target business elements and their associated entities, associated attributes, associated events, associated time information and associated identification information are extracted according to the target structured data area and row and column context reconstruction strategy, and they are bound as supplementary evidence data with source location.

[0153] In the specific implementation process, this step first establishes a row-column evidence reconstruction mechanism to generate row-column evidence records. The specific execution logic of the row-column evidence reconstruction mechanism is as follows:

[0154] First, based on the target structured data area output in step S3100, read the cell elements that meet the type constraints and whose type confidence meets the preset confidence threshold, and use them as target business elements; read the original row number and original column number saved in the cell type record, and record them as the source row number and source column number of the target business element.

[0155] Next, the row and column context reconstruction strategy output in step S2300 is invoked. Based on the in-row extraction range, in-column extraction range, and neighborhood extraction range included in the row and column context reconstruction strategy, adjacent cells are scanned in the data area corresponding to the previously recorded row and column numbers, along the row direction, column direction, and preset neighborhood direction of the target business element.

[0156] Simultaneously, the type recognition algorithm plugin set included in the row and column context reconstruction strategy is invoked to determine the type of the data content within the adjacent cells. If the data content of the adjacent cells corresponds to an entity type number, it is extracted as an associated entity; if the data content of the adjacent cells corresponds to an attribute type number, it is extracted as an associated attribute; if the data content of the adjacent cells corresponds to an event type number, it is extracted as an associated event; if the data content of the adjacent cells corresponds to a time type number, it is extracted as associated time information; and if the data content of the adjacent cells corresponds to a special identifier type number, it is extracted as associated identifier information.

[0157] The associated entities, associated attributes, associated events, associated time information, and associated identifier information all include a type number, ontology mapping label, content value, normalized value, type confidence score, source row number, and source column number. The type number is determined by the matched type recognition algorithm plugin; the content value is the original content of the adjacent cell; the normalized value is the content standardized by the corresponding plugin; and the source row number and source column number are the original row and column numbers of this information in the structured database. The ontology mapping label is determined based on the type feature index record and relationship evidence index record corresponding to the adjacent cell.

[0158] For each target business element in the target structured data region, it is bound to related entities, related attributes, related events, related time information, and related identification information extracted within the same row, column, or neighborhood, generating row and column evidence records. These records include the target business element, the source row number of the target business element, the source column number of the target business element, related information, and the source row and column numbers of each related piece of information.

[0159] The generation logic of the row and column evidence description text is as follows: when associated time information exists, the normalized value of the associated time information is placed at the beginning of the sentence; when associated entity exists, the content value of the associated entity is used as an entity record; when associated event exists, the content value of the associated event is used as an event record; then the target business element and its associated attributes are written at the end of the sentence, and the row and column evidence description text is generated according to the syntactic structure of "associated time information + associated entity + associated event + target business element + associated attribute".

[0160] If no associated time information is extracted from the context of a target business element, but an associated entity or event is extracted, then row and column evidence description text is generated according to the syntactic structure of "associated entity or event + target business element". If no associated entity, associated event or associated time information is extracted from the context of a target business element, then the target business element and its source row and column numbers are retained, and row and column evidence description text containing the source location is generated so that the target business element can be traced in subsequent verification.

[0161] Finally, all row and column evidence records are sorted according to the source position of the target business element, and the corresponding type feature index record number and relationship evidence index record number are written into the row and column evidence records, and the output is packaged as supplementary evidence data.

[0162] Each target business element and its associated information in the supplementary evidence data retains the original row and column positions, type feature index record numbers, and relational evidence index record numbers in the structured database; the data of the back-injected large model knowledge reasoning unit is read from the corresponding record by the retrieval server, and fact fields without associated source record numbers are not accepted.

[0163] In step S3300, the supplementary rule facts or supplementary evidence data are injected back into the large model knowledge reasoning unit. The large model knowledge reasoning unit performs ontology mapping, node coverage verification, and logical consistency verification to generate a logical closed-loop identifier and a multi-step knowledge reasoning state sequence.

[0164] Read the initial search result set output in step S2100, the supplementary rule facts output in step S3100, and the supplementary evidence data output in step S3200, and write their index record number, source location, node text content, or row and column evidence records into the supplementary evidence area in step S2200, while retaining the correspondence between the current round of targeted search instructions and search results in the search history area.

[0165] Based on the ontology mapping tags in the supplementary rule facts or row and column evidence records, the large model knowledge reasoning unit maps the new facts to the corresponding business entity nodes or condition nodes in the logic graph to be verified; when the ontology mapping tags are consistent and the constraint ranges match, the index record number of the new facts is bound to the corresponding node.

[0166] After completing the ontology mapping, the large model knowledge reasoning unit updates the multi-step knowledge reasoning state sequence. The updated state sequence includes the current reasoning round, the coverage status of each node, the evidence record number referenced by each covered node, nodes that are still not covered, the executed targeted retrieval instructions, and the number of newly added evidence in this round.

[0167] For nodes that are still not covered, perform node coverage verification again; when the new facts can satisfy the ontology mapping label, entity attributes and constraint scope of the node, update the corresponding node to a covered node; otherwise, keep it in an uncovered state.

[0168] After node coverage verification, logical consistency verification is performed, including determining whether rule facts and evidence facts can form a corresponding relationship through the same ontology mapping label, determining whether the target business element and its associated information meet the threshold range, time range, object range, state range or event conditions defined by the rule facts, and determining whether there are conflicts between multiple state values ​​corresponding to the same business entity.

[0169] If all business entity nodes and condition nodes are covered and there are no logical conflicts, the logical loop closure flag is set to 1, and the correspondence between nodes and evidence record numbers is output; if there is at least one uncovered node or a logical conflict, the logical loop closure flag is set to 0, and the uncovered node and conflicting node are output.

[0170] After supplementing evidence, uncovered nodes, covered nodes, and their evidence record numbers are written into the same multi-step knowledge reasoning state sequence. The next round of targeted retrieval instructions targets the uncovered nodes in this state sequence and retains the instructions executed in the previous round and the number of newly added evidence, thus avoiding repeated execution of retrieval parameters for which no new evidence has been obtained.

[0171] Step S4000: Iterative control is executed based on the logical closed-loop identifier. When the loop is not closed, the large model knowledge reasoning unit updates the directional retrieval instructions. When the loop is closed, a comprehensive evidence chain is generated, and the retrieval application results are output under the constraints of the controlled reasoning instruction set.

[0172] Specifically, this step reads the logical closed-loop identifier, multi-step knowledge reasoning state sequence, and correspondence between nodes and evidence record numbers output by step S3300; when the loop is not closed, it generates the next round of targeted retrieval instructions around the still uncovered nodes; when the loop is closed, it integrates the rule basis and factual evidence according to the evidence citation relationship.

[0173] Further, step S4000 includes:

[0174] In step S4100, when the logical closed-loop identifier is 0, the directional retrieval instruction is updated and the iteration continues. When the logical closed-loop identifier is 1, a comprehensive evidence chain is generated based on the correspondence between the node and the evidence record number.

[0175] This step establishes a branch control mechanism based on the logical closed-loop identifier, the retrieval round, and the number of newly added evidence. After reading the logical closed-loop identifier, it executes either the unclosed branch or the closed branch.

[0176] In the unclosed-loop branch, read the uncovered nodes, conflicting nodes, executed directional retrieval instructions, and the number of newly added evidence in this round from the multi-step knowledge reasoning state sequence; write the above fields into the retrieval history area and supplementary evidence area of ​​the large model knowledge reasoning unit.

[0177] The large model knowledge reasoning unit regenerates reasoning breakpoint descriptors for uncovered nodes and excludes index record numbers, relation edge types, or type constraints that have been executed but have not generated new evidence. If conflicting nodes exist, the conflicting evidence record numbers and conflicting fields are written into the constraints, so that the next round of targeted retrieval prioritizes the retrieval of independent source records that can confirm the conflicting fields.

[0178] The field integrity validator verifies the updated targeted search command. If the verification passes, it sends back to step S3100 to execute the next round of search. The system has a preset maximum number of search rounds. If there are still uncovered nodes after reaching the maximum number of search rounds, or if no new evidence is obtained in two consecutive rounds, the iteration stops and an insufficient evidence flag and uncovered nodes are output. No affirmative verification conclusion is generated.

[0179] In the closed-loop branch, stop updating the targeted retrieval instructions, and read the rule basis data and factual verification data according to the evidence record number bound to each covered node.

[0180] The data on which the rules are based includes the node text content, semantic matching value, index record number, and source document location in the supplementary rule facts or initial search result set; the data on which the facts are corroborated includes the target business element, related entity, related attribute, related event, related time information, related identification information, source row and column position, and corresponding index record number in the supplementary evidence data.

[0181] The candidate rule-based data is deduplicated. When multiple node text contents originate from the same unstructured document and their node sequence numbers are adjacent, the node text content with the higher semantic matching value is retained. When there is a hierarchical relationship between adjacent nodes, both the text content of the parent node and the text content of the child node with the higher semantic matching value are retained to obtain the rule-based data.

[0182] To filter out redundant data generated during multiple rounds of iterative expansion retrieval, a correspondence filtering process is performed between the rule-based data and the candidate fact-confirming data. Specifically, the same text vector generation unit as in step S1300 is invoked to map the node text content in the rule-based data to rule-based semantic vectors; and the row and column evidence description text in the candidate fact-confirming data is mapped to fact-evidence semantic vectors. The rule-based semantic vector represents the rule semantics corresponding to the node text content, and the fact-evidence semantic vector represents the fact semantics corresponding to the row and column evidence records.

[0183] Subsequently, the rules are calculated based on semantic vectors. semantic vector of factual evidence The confidence level of the rule-based evidence between them The calculation formula is as follows:

[0184]

[0185] in, This represents the cosine similarity between the node text content and the row and column evidence description text. The time matching coefficient, ranging from 0 to 1, represents the degree of matching between the time information in the row and column evidence records and the time range specified by the user's query command or node text content. If no specific time range exists, it will be... Set to 1; The balance coefficient is greater than 0 and less than 1. Its value is determined by verifying the rule evidence samples with labeled correct and incorrect correspondences, with a higher correct correspondence rate as the selection criterion.

[0186] The rule evidence correspondence confidence scores of all row and column evidence records are sorted in descending order, and then filtered according to a preset confidence threshold. If the rule evidence correspondence confidence score between a certain row and column evidence record and at least one rule-based data is greater than or equal to the preset confidence threshold, the record is retained; otherwise, it is discarded as a redundant record. The preset confidence threshold is determined through the rule evidence verification samples, with the selection criterion being that a larger number of records are retained while meeting the preset correct correspondence rate.

[0187] A comprehensive chain of evidence is generated according to the rule-based data, followed by factual evidence. The rule-based field of the comprehensive chain of evidence includes the index record number, node text content, semantic matching value, matching level, and source document location. The factual evidence field includes the type feature index record number, relationship evidence index record number, target business element, source row and column numbers of the target business element, associated entity, associated attribute, associated event, associated time information, associated identifier information, source row and column numbers of each associated piece of information, and row and column evidence description text.

[0188] In step S4200, under the constraints of the controlled inference instruction set, the user query instruction, the comprehensive evidence chain, and the node coverage relationship are input into the large model knowledge inference unit to generate a multi-step verification inference path. The evidence consistency index is calculated based on the multi-step verification inference path, and the retrieval application result is output when the evidence consistency index is equal to the absolute consistency threshold.

[0189] This step invokes the same large model knowledge reasoning unit as step S2200. This unit, used in steps S2200 to S4100 to identify uncovered nodes and generate targeted retrieval instructions, is used in this step to generate application output based on the comprehensive evidence chain that has formed a logical closed loop. The two stages share ontology label definitions and evidence record numbers, and do not use mutually independent and uncorrelated reasoning contexts.

[0190] Load a controlled inference instruction set into the knowledge inference unit of the large model. The controlled inference instruction set includes the scope of evidence citation, output field constraints, prohibition of rewriting factual content, sampling temperature parameters, decoding method, and maximum output length; the scope of evidence citation limits the large model to only cite user query commands, integrates the evidence chain and node coverage relationship, and the output field constraints limit the results to include three fields: verification conclusion, rule basis, and factual evidence.

[0191] The user query command, the comprehensive evidence chain, and the node coverage relationship are input into the large model knowledge reasoning unit in a fixed field order. The large model knowledge reasoning unit first reads the verification object and constraints in the user query command, and then reads the evidence record numbers supporting each business entity node and condition node along the node coverage relationship.

[0192] Correspondence verification includes: determining whether the object being checked matches the ontology object or entity attribute in the rule-based data; determining whether the associated entities, associated attributes, associated events, associated time information, and associated identifier information in the factual verification data meet the restrictions in the user's query instruction; and determining whether the target business element falls within the threshold, time, status, or event conditions limited by the rule-based data.

[0193] After completing the correspondence verification, a multi-step verification reasoning path is generated. The multi-step verification reasoning path sequentially records the ontology node corresponding to the verification object, the record number of the rule basis referenced, the record number of the factual evidence referenced, the comparison result of the rule conditions and the target business element, the logical consistency verification result, and the verification conclusion; each step retains the source record number and does not output facts that are not supported by the comprehensive evidence chain.

[0194] Before output, evidence consistency verification is performed. Following a unified field name, numerical unit, and time format, the multi-step verification reasoning path is segmented into features. This extracts target business elements, entity attributes, state values, constraint ranges, associated entities, associated attributes, associated events, associated time information, associated identifier information, and index record numbers, forming a generated feature set. The comprehensive chain of evidence is segmented according to the same rules to form a set of source features. .

[0195] Based on the generated feature set Source feature set Calculate the consistency index of evidence between the generated feature set and the source feature set. The calculation formula is:

[0196]

[0197] in, This represents the number of elements in the intersection of the generated feature set and the source feature set. This indicates the number of elements in the generated feature set; when the generated feature set is empty, the evidence consistency index is 0.

[0198] In this embodiment, the absolute consistency threshold is set to 1. When the evidence consistency index is less than 1, it indicates that there are factual features in the multi-step verification reasoning path that cannot be mapped to the comprehensive evidence chain. At this time, the multi-step verification reasoning path is cleared, the sampling temperature parameter is reduced, or a deterministic decoding method is used to regenerate. If the number of consecutive regenerations reaches the preset upper limit and still fails to reach the absolute consistency threshold, a generation failure flag is output.

[0199] When the evidence consistency index equals the absolute consistency threshold, it is determined that all factual features in the multi-step verification reasoning path can be found in the comprehensive evidence chain, and a preset response result structure template is invoked; the response result structure template includes verification conclusion field, rule basis field, and factual evidence field.

[0200] Based on the multi-step verification reasoning path, determine whether the object to be verified meets the judgment conditions corresponding to the user's query command, and fill the determination result into the verification conclusion field; read the rule basis record number supporting the conclusion along the multi-step verification reasoning path, and fill the corresponding node text content and source document position into the rule basis field; read the fact evidence record number, and fill the target business element, related information, and source row and column position into the fact evidence field.

[0201] The large-scale model knowledge reasoning unit performs statement concatenation on structured fields without changing the node text content, target business elements, source location, and evidence record number in the comprehensive evidence chain, generates retrieval application results, and outputs them through the application programming interface. When an insufficient evidence flag is received, only the uncovered nodes and the retrieved scope are output, without outputting a positive verification conclusion.

[0202] The same ontology label and evidence record number are used in the processes of lightweight index reading, uncovered node identification, targeted retrieval instruction generation, supplementary evidence verification, and application result output; the rule basis field and fact evidence field in the retrieval application results are thus located to the source location of the unstructured document or structured database.

[0203] Example 2:

[0204] This embodiment, based on Embodiment 1, provides an adaptive extended retrieval system based on semantic ontology and large-scale model knowledge reasoning, such as... Figure 4 As shown, the system includes a composite index module, a large model knowledge reasoning module, a targeted retrieval module, and an application output module;

[0205] The composite index module is used to generate a multimodal composite index library and lightweight index description data based on an unstructured document library and a structured database with missing or uncertain headers, and to retrieve the multimodal composite index library according to the user's query instructions, generating an initial search result set and candidate lightweight index description records.

[0206] The large model knowledge reasoning module is used to receive user query instructions, initial retrieval result set, candidate lightweight index description records, supplementary rule facts and supplementary evidence data, perform node coverage verification, ontology mapping and logical consistency verification, output node coverage status, reasoning breakpoint descriptor, multi-step verification reasoning path and logical closed loop flag, and output directional retrieval instructions when there are uncovered nodes.

[0207] The targeted retrieval module is used to parse the targeted retrieval instruction, perform semantic extension retrieval based on the ontology traversal parameters to obtain supplementary rule facts, or extract supplementary evidence data with source location based on type constraints and row and column context reconstruction strategies, and inject the supplementary rule facts or supplementary evidence data back into the large model knowledge reasoning module.

[0208] The application output module is used to trigger the directional retrieval instruction update and iterative retrieval when the logical closed-loop flag indicates that the loop is not closed, generate a comprehensive evidence chain when the logical closed-loop flag indicates that the loop is closed, and output the retrieval application results under the constraints of the controlled reasoning instruction set.

[0209] The system can be deployed in a retrieval server and an inference server with a processor, memory and communication interface; the memory stores index records, lightweight index description data and program instructions that can be executed by the processor. When the processor executes the program instructions, it completes index construction, model invocation, targeted retrieval, evidence reinjection and application result output.

[0210] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An adaptive extended retrieval method based on semantic ontology and large-scale model knowledge reasoning, characterized in that, include: Based on unstructured document libraries and structured databases with missing or uncertain header semantics, construct a multimodal composite index library and lightweight index description data; The multimodal composite index is retrieved according to the user's query instructions to obtain an initial search result set; this set, along with the user's query instructions and candidate index description records, is input into the large model knowledge reasoning unit to generate node coverage status, and to generate targeted search instructions when there are uncovered nodes. Based on the targeted retrieval instructions, perform ontology relation traversal to obtain supplementary rule facts, or perform type feature matching and row and column context reconstruction to obtain supplementary evidence data with source location; After obtaining supplementary rule facts or supplementary evidence data, it is injected back into the knowledge reasoning unit of the large model, and the node coverage status is updated through ontology mapping and node coverage verification. Based on the node coverage status and logical consistency verification results, a logical closed loop identifier is generated. If the loop is not closed, the instructions are updated and the retrieval is iterative. If the loop is closed, a comprehensive evidence chain and retrieval application results are generated.

2. The adaptive extended retrieval method based on semantic ontology and large-scale model knowledge reasoning according to claim 1, characterized in that, The multimodal composite index library includes a semantic index area, a type feature index area, a relational evidence index area, and index area registration information. The lightweight index description data is generated based on the index area registration information and the index records in each index area, including the index record number, ontology mapping label, relational edge type, type feature summary, source location, and retrieval method identifier. The node text of unstructured documents and the original content of cells in structured databases are stored in the multimodal composite index library and located by the index record number.

3. The adaptive extended retrieval method based on semantic ontology and large-model knowledge reasoning according to claim 1, characterized in that, The construction of the multimodal composite index library and lightweight index description data includes: Identify headings, paragraphs, line breaks, numbering, and table cell markers in unstructured documents and generate document nodes. Perform ontology semantic extraction on the document nodes to obtain ontology objects, entities, attributes, events, time, and semantic relationship edges, forming a set of document semantic nodes. Structured data with missing or ambiguous headers are mapped to an initial mixing matrix according to row and column positions. Cell types are identified, and a structured type feature set is generated based on the type distribution and semantic association of rows, columns, and adjacent cells. Document semantic nodes and structured type features are mapped to the same ontology tag set to generate semantic index records, type feature index records, and relation evidence index records. The index record number, ontology mapping tag, feature summary, source location, and retrieval method identifier are extracted to form lightweight index description data.

4. The adaptive extended retrieval method based on semantic ontology and large-model knowledge reasoning according to claim 3, characterized in that, The cell type identification includes: Input cell elements into at least two type recognition algorithm plugins, and each plugin outputs candidate type numbers, candidate normalized values, and candidate confidence scores; determine the target type number based on the candidate confidence scores, cell row and column positions, and the distribution of type numbers in adjacent cells; The target type number, normalized cell value, type confidence, and original row and column number are bound to form a cell type record. For each row, the ratio of the number of records that meet the target type number set and the type confidence reaches the preset threshold to the number of non-empty cells is used as the row-level type coverage. Rows with row-level type coverage reaching the preset coverage threshold are retained to form structured candidate data.

5. The adaptive extended retrieval method based on semantic ontology and large-model knowledge reasoning according to claim 1, characterized in that, The generated node coverage status and targeted retrieval instructions include: The initial retrieval result set is filtered based on the semantic matching value between the user query command and the semantic index record. Candidate index description records are determined from the lightweight index description data. The user query command, the initial retrieval result set, the candidate index description records, and the supplementary evidence from the previous round are encapsulated as model input data and input into the large model knowledge reasoning unit. The large model knowledge reasoning unit generates a logic graph to be verified and distinguishes between covered and uncovered nodes. When there are uncovered nodes, it generates a reasoning breakpoint descriptor containing missing types, entity objects, constraints, and target ontology mapping labels, and outputs ontology traversal parameters, or outputs type constraints and row and column context reconstruction strategies.

6. The adaptive extended retrieval method based on semantic ontology and large-model knowledge reasoning according to claim 5, characterized in that, The initial search result set filtering includes: The user query command is normalized and a query vector is generated. The text vector to be checked is extracted from the semantic index area. The semantic matching value is obtained based on the cosine similarity. The semantic matching values ​​are sorted and the upper quartile value and median value are calculated. The difference between the two is used as the discrete span. The query constraint density is generated based on the number of entity words, conditional phrases, and effective tokens in the query text. A dynamic similarity boundary is generated based on the query constraint density, the upper quartile value, and the discrete span. Semantic index records that reach the dynamic similarity boundary are retained, and the node text content, semantic matching value, index record number, ontology mapping label, and source location are encapsulated into an initial search result set.

7. The adaptive extended retrieval method based on semantic ontology and large-model knowledge reasoning according to claim 1, characterized in that, Executing the targeted retrieval instruction includes: The instruction type of the targeted retrieval instruction is parsed; when it is a supplement to rule facts, an extended retrieval is performed in the semantic index area and the relational evidence index area according to the ontology traversal parameters to obtain the supplementary rule facts; when it is a supplement to evidence facts, the target structured data area is determined according to the type constraints and structured type feature set, and the target business elements and related information are extracted according to the row and column context reconstruction strategy to form supplementary evidence data with source location. The supplementary rule facts or supplementary evidence data are injected back into the knowledge reasoning unit of the large model, and ontology mapping, node coverage verification and logical consistency verification are performed. When all nodes to be verified are covered and there is no logical conflict, the logical closed loop flag is set to closed loop; otherwise, it is set to unclosed loop.

8. The adaptive extended retrieval method based on semantic ontology and large-model knowledge reasoning according to claim 7, characterized in that, The row and column context reconstruction includes: Read cells that meet the type constraints and have a type confidence level of a preset threshold from the target structured data area as target business elements, and record their original row number and original column number; scan adjacent cells along the row, column and preset neighborhood of the cell as the center; Extract associated entities, associated attributes, associated events, associated time information, or associated identifier information based on the type number and ontology mapping label of adjacent cells; bind the target business element with the associated information in the same row and column context to form a row and column evidence record containing the source row number and source column number, and encapsulate it into supplementary evidence data in the order of the source position.

9. The adaptive extended retrieval method based on semantic ontology and large-scale model knowledge reasoning according to claim 1, characterized in that, The retrieval iteration and application output include: When the logical closed loop flag indicates that the loop is not closed, the reasoning breakpoint descriptor, the executed directional retrieval instructions, and the supplementary rule facts or supplementary evidence data of this round are input into the large model knowledge reasoning unit to exclude retrieval parameters that have not generated new evidence and generate updated directional retrieval instructions. When the logical closed-loop flag indicates that the loop is closed, a comprehensive evidence chain is generated based on the initial retrieval result set, supplementary rule facts, and supplementary evidence data. Under the constraints of the controlled inference instruction set, a multi-step verification inference path is generated based on the user query instruction and the comprehensive evidence chain, and the retrieval application results are output based on the evidence consistency index between the multi-step verification inference path and the comprehensive evidence chain.

10. The adaptive extended retrieval method based on semantic ontology and large-model knowledge reasoning according to claim 9, characterized in that, The output of retrieval application results based on the evidence consistency index of the multi-step verification reasoning path and the comprehensive evidence chain includes: Load the controlled reasoning instruction set and structured output field constraints into the large model knowledge reasoning unit, and input the user query instruction, the comprehensive evidence chain, and the node coverage relationship between the node and the index record number into the large model knowledge reasoning unit to generate a multi-step verification reasoning path; The multi-step verification reasoning path and the comprehensive evidence chain are segmented into features to form a generated feature set and a source feature set. The evidence consistency index is calculated based on the two feature sets. When the evidence consistency index reaches the absolute consistency threshold, the structured template of the response result is called to output the retrieval application result. Otherwise, the sampling temperature parameter is reduced or the multi-step verification reasoning path is regenerated using a deterministic decoding method.

11. An adaptive extended retrieval system based on semantic ontology and large-model knowledge reasoning, characterized in that, It includes a composite index module, a large model knowledge reasoning module, a targeted retrieval module, and an application output module; The composite index module is used to generate a multimodal composite index library and lightweight index description data based on an unstructured document library and a structured database with missing or uncertain headers, and to retrieve the multimodal composite index library according to the user's query instructions, generating an initial search result set and candidate lightweight index description records. The large model knowledge reasoning module is used to receive user query instructions, initial retrieval result set, candidate lightweight index description records, supplementary rule facts and supplementary evidence data, perform node coverage verification, ontology mapping and logical consistency verification, output node coverage status, reasoning breakpoint descriptor, multi-step verification reasoning path and logical closed loop flag, and output directional retrieval instructions when there are uncovered nodes. The targeted retrieval module is used to parse the targeted retrieval instruction, perform semantic extension retrieval based on the ontology traversal parameters to obtain supplementary rule facts, or extract supplementary evidence data with source location based on type constraints and row and column context reconstruction strategies, and inject the supplementary rule facts or supplementary evidence data back into the large model knowledge reasoning module. The application output module is used to trigger the directional retrieval instruction update and iterative retrieval when the logical closed-loop flag indicates that the loop is not closed, generate a comprehensive evidence chain when the logical closed-loop flag indicates that the loop is closed, and output the retrieval application results under the constraints of the controlled reasoning instruction set.

Images