A method for intelligent information matching and retrieval based on document tree

Abstract

This invention proposes a document tree-based intelligent information matching and retrieval method. It combines document structure with feature information to generate a document tree carrying document feature information, enabling hierarchical matching and rapidly converging the number of target documents. This reduces computational load, increases retrieval speed, and improves matching accuracy. Applying this technical solution enables fast, efficient, timely, and accurate matching and searching of massive amounts of document data, improving knowledge production efficiency. This invention effectively addresses the cost and accuracy issues in knowledge production and information retrieval. The document tree designed in this invention intelligently and objectively captures document keyword information and carries the document's theme, structure, and content information, eliminating the need for manual annotation and extraction. This effectively reduces labor costs and solves the problems of inaccurate and incomplete keyword annotation, significantly improving the accuracy of matching and retrieval.

Description

Technical Field

[0001] This invention relates to the fields of document search, data matching, and natural language processing, and in particular to a method for intelligent information matching and retrieval based on document trees. Background Technology

[0002] Current information retrieval, especially in document retrieval, largely relies on keyword matching. However, keyword matching struggles to meet users' search requirements. First, tagging information with keywords requires significant manual labor. Second, manual tagging cannot guarantee a comprehensive and accurate representation of the document's content, especially for interdisciplinary academic documents. A limited number of keywords is insufficient to express the knowledge structure of the various disciplines covered in the document, resulting in issues such as incomplete or inaccurate keyword searches, low search quality, and an excessive number of matching results.

[0003] On the other hand, while traditional string matching algorithms and semantic similarity algorithms, or even the latest natural language processing techniques, can achieve high matching accuracy, they also face challenges. With the explosion of knowledge, especially the exponential growth in the volume of unstructured and semi-structured documents, these information retrieval methods typically take a considerable amount of time to complete a single search, hindering efficient and rapid document matching and retrieval. This severely impacts productivity and the speed of knowledge re-creation and reprocessing.

[0004] Therefore, the importance of automatically, efficiently, and promptly matching and searching for expected documents in large amounts of data or even massive amounts of information documents, and discovering information documents with high similarity in terms of themes, viewpoints, content, and other elements, is extremely prominent. Summary of the Invention

[0005] This invention proposes a method for intelligent information matching and retrieval based on document trees, which can achieve fast, efficient, timely and accurate matching and searching of massive document data, thereby improving people's knowledge production efficiency.

[0006] The present invention adopts the following technical solution.

[0007] A method for intelligent information matching and retrieval based on a document tree, wherein the method combines document structure with feature information to generate a document tree that can carry document feature information, and reduces the amount of computation by quickly converging the number of target documents through hierarchical matching; the method includes the following steps;

[0008] Step S1: For the existing document set Data information extraction and document feature preprocessing are performed to generate corresponding document tree sets. It also records the index correspondence between documents and the document tree, where D0 represents the set of all document data. Let i be a document in D0, that is, element i in set D0; T0 represents the document tree set corresponding to all document data. Let i be the document tree in T0, that is, the element i of set T0; and It is a one-to-one correspondence;

[0009] Step S2: Extract data information and process document features in real time from the input document to generate the corresponding input document tree;

[0010] Step S3: Load The topic relevance of all document tree topic feature nodes in the input document tree to the topic feature nodes in the input document tree is calculated, and a subset of document trees with topic matching values ​​greater than a certain threshold is obtained. T1∈T0, where T1 represents a subset of document trees in T0 that match the topic of the input document tree. Let i be the document tree in T1, that is, the element i in set T1;

[0011] Step S4: Based on the document identifier corresponding to T1, obtain the subset of documents in document set D0 that match the topic of the input document. D1∈D0, where D1 represents a subset of documents in D0 that match the topic of the input document. Document i in D1 is an element of set D1;

[0012] Step S5: Loading Structural similarity is calculated between the structural feature nodes of all document trees and the structural feature nodes of the input document tree to obtain a subset of document trees whose structural matching value is greater than a certain threshold. T2∈T1∈T0, where T2 represents a subset of document trees in T1 that matches the input document tree structure. Let i be the document tree in T2, that is, the element i of set T2;

[0013] Step S6: Based on the document identifier corresponding to T2, obtain the document subset in document subset D1 that matches the structural hierarchy of the input document. D2∈D1∈D0, where D2 represents the subset of documents in D1 that match the structure of the input document. Document i in set D2, i.e., element i in set D2;

[0014] Step S7: Load Content similarity is calculated between the content feature nodes of all document trees in the input document tree and the content feature nodes of the input document tree. A subset of document trees with a content matching value greater than a certain threshold is obtained. T3∈T2∈T1∈T0, where T3 represents a subset of document trees in T2 that match the content of the input document tree. Let i be the document tree in T3, that is, element i of set T3;

[0015] Step S8: Based on the document identifier corresponding to T3, obtain the document subset in document subset D2 that matches the content of the input document. D3∈D2∈D1∈D0, where D3 represents the subset of documents in D2 that match the content of the input document. Document i in set D3, i.e., element i in set D3;

[0016] Step S9: Based on the scenario intent of the matching search, select document subsets D3, D2, and D1 from high to low matching degree as the output results of the matching search.

[0017] The document data information extraction and document feature processing in steps S1 and S2 include the following steps:

[0018] Step S101: Use an encoder-decoder network model, or a pre-trained BERT or GPT model, to generate the original tree-like hierarchical structure of the document data, i.e., the original document tree;

[0019] Step S102: Use the natural language processing module to perform keyword extraction, entity recognition, relation extraction, and event extraction operations on the document fragments under each node of the original document tree to generate feature objects in vector or matrix form;

[0020] Step S103: Create or update corresponding feature objects of different types or reference relationships of feature objects for nodes at different positions in the original document tree;

[0021] Step S104: Based on the original document tree, a simplified, compressed, and paradigmatic tree-like hierarchical structure, namely the document summary tree, is generated by transforming nodes, local moving, pruning algorithms, and insertion or merging methods.

[0022] Step S105: Accordingly, update the feature objects or reference relationships of each node in the document summary tree.

[0023] Step S3 includes the following steps:

[0024] Step S301: From Extract the root node of all document trees and load the topic feature set Vs that it points to;

[0025] Step S302: Extract and load the root node of the input document tree and the topic features it points to;

[0026] Step S303: Load the network model or calculation module based on word vector relevance calculation, and calculate the relevance between each element in Vs and vs.

[0027] Step S304: Compare the calculated relevance value with the preset threshold ts. If it is found that the relevance between a certain element in Vs and vs exceeds ts, it is determined that the corresponding document has a match in one or more aspects of subject direction, content theme, overall viewpoint, and overall sentiment, and the document tree identifier corresponding to the element is recorded.

[0028] Step S305: Finally, obtain a subset of document trees that meet the matching requirements.

[0029] Step S5 includes the following steps:

[0030] Step S501: From Extract the tree structure of all document trees, excluding leaf nodes, and load the set of structural features Va that it points to;

[0031] Step S502: Extract and load the tree structure of the input document tree, excluding leaf nodes and their pointed structural features va;

[0032] Step S503: Load the network model or calculation module based on tree similarity calculation, and calculate the tree similarity between each element in Va and va;

[0033] Step S504: Compare the calculated tree similarity value with the preset threshold ta. If it is found that the similarity between a certain element in Va and the va tree exceeds ta, it is determined that there is a match that meets expectations in one or more aspects of the document chapter level, the transfer of viewpoints and ideas, and the overall writing style, and the document tree identifier corresponding to the element is recorded.

[0034] Step S505: Finally, obtain a subset of document trees that meet the matching requirements.

[0035] Step S7 includes the following steps:

[0036] Step S701: From Extract all leaf nodes of the document tree and load the content feature set Vc that they point to;

[0037] Step S702: Extract and load the leaf nodes of the input document tree and the content features (vc) they point to;

[0038] Step S703: Load the network model or calculation module based on string matching and semantic similarity calculation, and calculate the similarity between each element in Vc and the content of vc;

[0039] Step S704: Compare the calculated value of content similarity with the preset threshold tc. If it is found that the content similarity between some leaf nodes of a certain element in Vc and some leaf nodes of vc exceeds tc, it is determined that there is a matching of the text description similarity or text repetition rate of the local content, i.e., the document fragment, and the document tree identifier and node position corresponding to the element are recorded.

[0040] Step S705: Calculate the average similarity between all leaf nodes of a certain element in Vc and vc. When it exceeds a certain preset threshold tc', it is determined that the full text content, i.e. all document fragments, has a high similarity, and the document tree identifier corresponding to the element is recorded.

[0041] Step S706: Finally, obtain a subset of document trees that meet the matching requirements.

[0042] In the process of document data extraction and feature processing, the method categorizes the extracted features into three types: thematic features, structural features, and content features. Thematic features include the document's theme, subject classification, keywords and word frequencies, and a digital representation of the overall sentiment. Structural features include the contextual relationships at the chapter level, chapter-level summaries, and a digital representation of keywords and word frequencies at the chapter level. Content features include summaries of document fragments at the corresponding level, keywords and word frequencies of those fragments, and a digital representation of the original textual information of those fragments.

[0043] The nodes of the document tree are divided into three categories: root node, non-leaf nodes, and leaf nodes. The root node is used to represent the theme features of the document, the other non-leaf nodes are used to represent the structural features of the document, and the leaf nodes are used to represent the content features of the document.

[0044] Based on the characteristics of the storage and computing resources of the actual system, each node of the document tree corresponds to a matrix, a vector, or a group of vectors, or other data structure forms that can express the characteristics of the document. The matrix includes tree arrays or variations of tree arrays; the vectors and vector groups include KD trees or variations of KD trees.

[0045] In the method, the document tree information generated by the preprocessing of the document library is stored in a permanent or semi-permanent form, and loaded into memory or cache as needed during the matching and retrieval calculation process to improve the speed of matching and retrieval; the method stores the overall tree structure of the document tree in a graph database according to the data structure form of the final selected document tree node, and stores the topic features, structural features, and content features of the node in a suitable vector database to improve the speed of data access, reading and writing operations.

[0046] The method configures matching strategies with different rules and matching thresholds with different precision to meet different application scenarios and matching retrieval intentions. During the matching retrieval calculation process, different network models or calculation modules are dynamically loaded according to different matching strategies, and result sets of different interval sizes are output according to the matching threshold.

[0047] The document tree includes a first-level original document tree and a second-level document summary tree. Generating the original document tree is a prerequisite for generating the document summary tree.

[0048] The original document tree is generated using an encoder-decoder network model or a pre-trained BERT or GPT model, producing the original tree-like hierarchical structure of the document data. The tree has a greater depth or more levels, more complex paths, and a larger amount of data, making it unsuitable for direct matching and retrieval operations.

[0049] The document summary tree is a simplified, compressed, and paradigmatic tree-like hierarchical structure generated from the original document tree through node transformation, local movement, pruning algorithms, insertion, or merging. The relevance or similarity calculations of the information intelligent matching and retrieval method based on the document tree are all performed on the document summary tree, so as to achieve the goal of low computational load but almost no loss of any key information of the document.

[0050] The nodes of the document tree are divided into topic feature nodes, structural feature nodes, and content feature nodes;

[0051] The topic feature node, which is the root node of the document tree, is used to represent the topic features of the document. The feature component it points to can include the digital expression of the document's topic, subject classification, keywords and word frequencies, and sentiment information.

[0052] Structural feature nodes, which are non-leaf nodes other than the root node, are used to represent the structural features of a document. The feature components they point to can include the contextual relationships at the chapter level, the summary at the chapter level, the keywords at the chapter level, and the digital representation of word frequency information.

[0053] Content feature nodes, also known as leaf nodes, are used to represent the content features of a document. The feature components they point to include the summary of each document fragment at the corresponding level, the keywords and word frequencies of the document fragment, and the digital representation of the original text information of the document fragment.

[0054] The aforementioned documents refer to files or data stored on computer systems or websites that are primarily text-based, including but not limited to electronic files in various formats such as doc / docx, pdf, wps, txt, xml, and caj, such as professional materials, academic literature, patent documents, conference papers, dissertations, legal documents, application documents, and journal articles.

[0055] The document tree-based intelligent information matching and retrieval method employs a document tree-based intelligent information matching and retrieval system, which includes:

[0056] The document tree generation module mainly extracts data information from documents and processes document features to generate document trees.

[0057] The document tree storage module mainly stores the document tree data after preprocessing of the document library, and generally uses a suitable graph database, vector database, or vector database.

[0058] The model library / algorithm library mainly stores various related algorithms, network models, and NLP modules for use by the document generation module and the matching and retrieval module.

[0059] The matching strategy configuration module is mainly used to configure various matching strategies, algorithms, and precision in different scenarios.

[0060] The matching and retrieval module mainly performs calculations and provides services for document tree topic relevance, document tree structure similarity, and document tree content similarity.

[0061] The results output module outputs matching results according to the scene intent.

[0062] This invention combines document structure with feature information to generate a document tree that carries document feature information, thereby enabling hierarchical matching and rapidly converging the number of target documents. This reduces computational load, increases retrieval speed, and improves matching accuracy. Applying this technical solution enables fast, efficient, timely, and accurate matching and searching of massive amounts of document data, improving people's knowledge production efficiency.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] (1) This invention can effectively solve the cost and accuracy problems in knowledge production and information retrieval processes. The document tree designed in this invention can intelligently and objectively capture document keyword information and carry the document's theme, structure, and content information without manual annotation or extraction. This can effectively reduce labor costs and solve the problems of inaccurate and incomplete keywords from manual annotation, thus greatly improving the accuracy of matching retrieval.

[0065] (2) This invention can effectively solve the problems of resource consumption and speed limitations in knowledge production and information retrieval processes. This invention achieves step-by-step calculation and filtering through three progressive steps: "topic matching - structure matching - content matching," which can quickly converge the number of target documents, thereby reducing computational load, increasing retrieval speed, and improving people's knowledge production efficiency. At the same time, due to the reduction in computational load, it can save on various hardware resources such as computing and memory, and reduce the energy consumption of the computing module.

[0066] This invention has a wide range of applications and significant potential for widespread adoption. It can be applied to traditional information retrieval and internet search, as well as various scenarios such as knowledge recommendation and plagiarism detection. By pre-setting matching strategies with different rules, it loads different algorithms in the three stages of topic matching, structure matching, and content matching, adapting to thresholds of varying precision to output results for different matching elements, thereby meeting diverse needs such as information retrieval, plagiarism detection, and recommendation. Attached Figure Description

[0067] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0068] Appendix Figure 1 This is a document tree structure diagram of a preferred embodiment of the present invention;

[0069] Appendix Figure 2 This is a flowchart of a preferred embodiment of the present invention;

[0070] Appendix Figure 3 This is a system block diagram of a preferred embodiment of the present invention. Detailed Implementation

[0071] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0072] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0073] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0074] like Figures 1 to 3A method for intelligent information matching and retrieval based on document trees, wherein the method combines document structure with feature information to generate a document tree that can carry document feature information, and reduces the amount of computation by quickly converging the number of target documents through step-by-step matching; including the following steps;

[0075] Step S1: For the existing document set Data information extraction and document feature preprocessing are performed to generate corresponding document tree sets. It also records the index correspondence between documents and the document tree, where D0 represents the set of all document data. Let i be a document in D0, that is, element i in set D0; T0 represents the document tree set corresponding to all document data. Let i be the document tree in T0, that is, the element i of set T0; and It is a one-to-one correspondence;

[0076] The following is an example of this:

[0077]

[0078] Step S2: Extract data information and process document features in real time from the input document to generate the corresponding input document tree;

[0079] Step S3: Load The topic relevance of all document tree topic feature nodes in the input document tree to the topic feature nodes in the input document tree is calculated, and a subset of document trees with topic matching values ​​greater than a certain threshold is obtained. T1∈T0, where T1 represents a subset of document trees in T0 that match the topic of the input document tree. Let i be the document tree in T1, that is, the element i in set T1;

[0080] Step S4: Based on the document identifier corresponding to T1, obtain the subset of documents in document set D0 that match the topic of the input document. D1∈D0, where D1 represents a subset of documents in D0 that match the topic of the input document. Document i in D1 is an element of set D1;

[0081] Step S5: Loading Structural similarity is calculated between the structural feature nodes of all document trees and the structural feature nodes of the input document tree to obtain a subset of document trees whose structural matching value is greater than a certain threshold. T2∈T1∈T0, where T2 represents a subset of document trees in T1 that matches the input document tree structure. Let i be the document tree in T2, that is, the element i of set T2;

[0082] Step S6: Based on the document identifier corresponding to T2, obtain the document subset in document subset D1 that matches the structural hierarchy of the input document. D2∈D1∈D0, where D2 represents the subset of documents in D1 that match the structure of the input document. Document i in set D2, i.e., element i in set D2;

[0083] Step S7: Load Content similarity is calculated between the content feature nodes of all document trees in the input document tree and the content feature nodes of the input document tree. A subset of document trees with a content matching value greater than a certain threshold is obtained. T3∈T2∈T1∈T0, where T3 represents a subset of document trees in T2 that match the content of the input document tree. Let i be the document tree in T3, that is, element i of set T3;

[0084] Step S8: Based on the document identifier corresponding to T3, obtain the document subset in document subset D2 that matches the content of the input document. D3∈D2∈D1∈D0, where D3 represents the subset of documents in D2 that match the content of the input document. Document i in set D3, i.e., element i in set D3;

[0085] Step S9: Based on the scenario intent of the matching search, select document subsets D3, D2, and D1 from high to low matching degree as the output results of the matching search.

[0086] The document data information extraction and document feature processing in steps S1 and S2 include the following steps:

[0087] Step S101: Use an encoder-decoder network model, or a pre-trained BERT or GPT model, to generate the original tree-like hierarchical structure of the document data, i.e., the original document tree;

[0088] Step S102: Use the natural language processing module to perform keyword extraction, entity recognition, relation extraction, and event extraction operations on the document fragments under each node of the original document tree to generate feature objects in vector or matrix form;

[0089] In this example, node feature components can include ordinary eigenvalues, eigenvectors, feature matrices, etc. In actual programming, all feature components of each node can be encapsulated into a feature object corresponding to that node. The feature object is described as follows:

[0090]

[0091] Step S103: Create or update corresponding feature objects of different types or reference relationships of feature objects for nodes at different positions in the original document tree;

[0092] In this embodiment, the feature objects obtained in step S102 are added to or updated in the data content of the corresponding document tree node. Alternatively, the relationship between nodes and node feature objects can be established through pointers or address references, such as:

[0093]

[0094] Step S104: Based on the original document tree, a simplified, compressed, and paradigmatic tree-like hierarchical structure, namely the document summary tree, is generated by transforming nodes, local moving, pruning algorithms, and insertion or merging methods.

[0095] Step S105: Accordingly, update the feature objects or reference relationships of each node in the document summary tree.

[0096] Step S3 includes the following steps:

[0097] Step S301: From Extract the root node of all document trees and load the topic feature set Vs that it points to;

[0098] Step S302: Extract and load the root node of the input document tree and the topic features it points to;

[0099] Step S303: Load the network model or calculation module based on word vector relevance calculation, and calculate the relevance between each element in Vs and vs.

[0100] Step S304: Compare the calculated relevance value with the preset threshold ts. If it is found that the relevance between a certain element in Vs and vs exceeds ts, it is determined that the corresponding document has a match in one or more aspects of subject direction, content theme, overall viewpoint, and overall sentiment, and the document tree identifier corresponding to the element is recorded.

[0101] Step S305: Finally, obtain a subset of document trees that meet the matching requirements.

[0102] Step S5 includes the following steps:

[0103] Step S501: From Extract the tree structure of all document trees, excluding leaf nodes, and load the set of structural features Va that it points to;

[0104] Step S502: Extract and load the tree structure of the input document tree, excluding leaf nodes and their pointed structural features va;

[0105] Step S503: Load the network model or calculation module based on tree similarity calculation, and calculate the tree similarity between each element in Va and va;

[0106] Step S504: Compare the calculated tree similarity value with the preset threshold ta. If it is found that the similarity between a certain element in Va and the va tree exceeds ta, it is determined that there is a match that meets expectations in one or more aspects of the document chapter level, the transfer of viewpoints and ideas, and the overall writing style, and the document tree identifier corresponding to the element is recorded.

[0107] Step S505: Finally, obtain a subset of document trees that meet the matching requirements.

[0108] Step S7 includes the following steps:

[0109] Step S701: From Extract all leaf nodes of the document tree and load the content feature set Vc that they point to;

[0110] Step S702: Extract and load the leaf nodes of the input document tree and the content features (vc) they point to;

[0111] Step S703: Load the network model or calculation module based on string matching and semantic similarity calculation, and calculate the similarity between each element in Vc and the content of vc;

[0112] Step S704: Compare the calculated value of content similarity with the preset threshold tc. If it is found that the content similarity between some leaf nodes of a certain element in Vc and some leaf nodes of vc exceeds tc, it is determined that there is a matching of the text description similarity or text repetition rate of the local content, i.e., the document fragment, and the document tree identifier and node position corresponding to the element are recorded.

[0113] Step S705: Calculate the average similarity between all leaf nodes of a certain element in Vc and vc. When it exceeds a certain preset threshold tc', it is determined that the full text content, i.e. all document fragments, has a high similarity, and the document tree identifier corresponding to the element is recorded.

[0114] Step S706: Finally, obtain a subset of document trees that meet the matching requirements.

[0115] In the process of document data extraction and feature processing, the method categorizes the extracted features into three types: thematic features, structural features, and content features. Thematic features include the document's theme, subject classification, keywords and word frequencies, and a digital representation of the overall sentiment. Structural features include the contextual relationships at the chapter level, chapter-level summaries, and a digital representation of keywords and word frequencies at the chapter level. Content features include summaries of document fragments at the corresponding level, keywords and word frequencies of those fragments, and a digital representation of the original textual information of those fragments.

[0116] The nodes of the document tree are divided into three categories: root node, non-leaf nodes, and leaf nodes. The root node is used to represent the theme features of the document, the other non-leaf nodes are used to represent the structural features of the document, and the leaf nodes are used to represent the content features of the document.

[0117] Based on the characteristics of the storage and computing resources of the actual system, each node of the document tree corresponds to a matrix, a vector, or a group of vectors, or other data structure forms that can express the characteristics of the document. The matrix includes tree arrays or variations of tree arrays; the vectors and vector groups include KD trees or variations of KD trees.

[0118] In the method, the document tree information generated by the preprocessing of the document library is stored in a permanent or semi-permanent form, and loaded into memory or cache as needed during the matching and retrieval calculation process to improve the speed of matching and retrieval; the method stores the overall tree structure of the document tree in a graph database according to the data structure form of the final selected document tree node, and stores the topic features, structural features, and content features of the node in a suitable vector database to improve the speed of data access, reading and writing operations.

[0119] The method configures matching strategies with different rules and matching thresholds with different precision to meet different application scenarios and matching retrieval intentions. During the matching retrieval calculation process, different network models or calculation modules are dynamically loaded according to different matching strategies, and result sets of different interval sizes are output according to the matching threshold.

[0120] The document tree includes a first-level original document tree and a second-level document summary tree. Generating the original document tree is a prerequisite for generating the document summary tree.

[0121] The original document tree is generated using an encoder-decoder network model or a pre-trained BERT or GPT model, producing the original tree-like hierarchical structure of the document data. The tree has a greater depth or more levels, more complex paths, and a larger amount of data, making it unsuitable for direct matching and retrieval operations.

[0122] The document summary tree is a simplified, compressed, and paradigmatic tree-like hierarchical structure generated from the original document tree through node transformation, local movement, pruning algorithms, insertion, or merging. The relevance or similarity calculations of the information intelligent matching and retrieval method based on the document tree are all performed on the document summary tree, so as to achieve the goal of low computational load but almost no loss of any key information of the document.

[0123] The nodes of the document tree are divided into topic feature nodes, structural feature nodes, and content feature nodes;

[0124] The topic feature node, which is the root node of the document tree, is used to represent the topic features of the document. The feature component it points to can include the digital expression of the document's topic, subject classification, keywords and word frequencies, and sentiment information.

[0125] Structural feature nodes, which are non-leaf nodes other than the root node, are used to represent the structural features of a document. The feature components they point to can include the contextual relationships at the chapter level, the summary at the chapter level, the keywords at the chapter level, and the digital representation of word frequency information.

[0126] Content feature nodes, also known as leaf nodes, are used to represent the content features of a document. The feature components they point to include the summary of each document fragment at the corresponding level, the keywords and word frequencies of the document fragment, and the digital representation of the original text information of the document fragment.

[0127] The aforementioned documents refer to files or data stored on computer systems or websites that are primarily text-based, including but not limited to electronic files in various formats such as doc / docx, pdf, wps, txt, xml, and caj, such as professional materials, academic literature, patent documents, conference papers, dissertations, legal documents, application documents, and journal articles.

[0128] The document tree-based intelligent information matching and retrieval method employs a document tree-based intelligent information matching and retrieval system, which includes:

[0129] The document tree generation module mainly extracts data information from documents and processes document features to generate document trees.

[0130] The document tree storage module mainly stores the document tree data after preprocessing of the document library, and generally uses a suitable graph database, vector database, or vector database.

[0131] The model library / algorithm library mainly stores various related algorithms, network models, and NLP modules for use by the document generation module and the matching and retrieval module.

[0132] The matching strategy configuration module is mainly used to configure various matching strategies, algorithms, and precision in different scenarios.

[0133] The matching and retrieval module mainly performs calculations and provides services for document tree topic relevance, document tree structure similarity, and document tree content similarity.

[0134] The results output module outputs matching results according to the scene intent.

[0135] The flowchart of this method is as follows: Figure 2 As shown, this flowchart mainly describes the process of obtaining a subset of documents that meet the expected criteria by progressively matching and filtering through three dimensions: topic, structure, and content. The flowchart includes the following:

[0136] (1) Preprocess the existing document library to generate a document tree set;

[0137] (2) Real-time processing of input documents to generate input document trees;

[0138] (3) From document tree set Retrieve and load the topic feature set V s ;

[0139] (4) Extract and load topic features v from the input document tree t. s ;

[0140] (5) From V s and v s Perform topic relevance calculations to obtain a subset of document trees whose topic matching values ​​are greater than a certain threshold.

[0141] (6) Based on the document identifier corresponding to T1, obtain the subset of documents that match the topic of the input document.

[0142] (7) From document tree set Retrieve and load the structural feature set V a ;

[0143] (8) Extract and load structural features v from the input document tree t. a ;

[0144] (9) From V a and v a Perform structural similarity calculations to obtain a subset of document trees whose structural matching values ​​are greater than a certain threshold.

[0145] (10) Based on the document identifier corresponding to T2, obtain the document subset that matches the structural hierarchy of the input document.

[0146] (11) From document tree set Retrieve and load the content feature set V c ;

[0147] (12) Extract and load content features v from the input document tree t. c ;

[0148] (13) From V c and v c Perform content similarity calculations to obtain a subset of documents whose content matching values ​​are greater than a certain threshold. Based on the document identifier corresponding to T3, obtain the subset of documents that match the content of the input document.

Claims

1. A method for intelligent information matching and retrieval based on document trees, characterized in that: The method combines document structure with feature information to generate a document tree that can carry document feature information, and reduces the amount of computation by quickly converging the number of target documents through hierarchical matching. Includes the following steps; Step S1: For the existing document set Data information extraction and document feature preprocessing are performed to generate corresponding document tree sets. It also records the index correspondence between documents and the document tree, where D0 represents the set of all document data. Let i be a document in D0, that is, element i in set D0; T0 represents the document tree set corresponding to all document data. Let i be the document tree in T0, that is, the element i of set T0; and It is a one-to-one correspondence; Step S2: Extract data information and process document features in real time from the input document to generate the corresponding input document tree; Step S3: Load The topic relevance of all document tree topic feature nodes in the input document tree to the topic feature nodes in the input document tree is calculated, and a subset of document trees with topic matching values ​​greater than a certain threshold is obtained. T1∈T0, where T1 represents a subset of document trees in T0 that match the topic of the input document tree. Let i be the document tree in T1, that is, the element i in set T1; Step S4: Based on the document identifier corresponding to T1, obtain the subset of documents in document set D0 that match the topic of the input document. D1∈D0, where D1 represents a subset of documents in D0 that match the topic of the input document. Document i in D1 is an element of set D1; Step S5: Loading Structural similarity is calculated between the structural feature nodes of all document trees and the structural feature nodes of the input document tree to obtain a subset of document trees whose structural matching value is greater than a certain threshold. T2∈T1∈T0, where T2 represents a subset of document trees in T1 that matches the input document tree structure. Let i be the document tree in T2, that is, the element i of set T2; Step S6: Based on the document identifier corresponding to T2, obtain the document subset in document subset D1 that matches the structural hierarchy of the input document. D2∈D1∈D0, where D2 represents the subset of documents in D1 that match the structure of the input document. Document i in set D2, i.e., element i in set D2; Step S7: Load Content similarity is calculated between the content feature nodes of all document trees in the input document tree and the content feature nodes of the input document tree. A subset of document trees with a content matching value greater than a certain threshold is obtained. T3∈T2∈T1∈T0, where T3 represents a subset of document trees in T2 that match the content of the input document tree. Let i be the document tree in T3, that is, element i of set T3; Step S8: Based on the document identifier corresponding to T3, obtain the document subset in document subset D2 that matches the content of the input document. D3∈D2∈D1∈D0, where D3 represents the subset of documents in D2 that match the content of the input document. Document i in set D3, i.e., element i in set D3; Step S9: Based on the scenario intent of the matching search, select document subsets D3, D2, and D1 from high to low matching degree as the output results of the matching search.

2. The method for intelligent information matching and retrieval based on document tree according to claim 1, characterized in that: The document data information extraction and document feature processing in steps S1 and S2 include the following steps: Step S101: Use an encoder-decoder network model, or a pre-trained BERT or GPT model, to generate the original tree-like hierarchical structure of the document data, i.e., the original document tree; Step S102: Use the natural language processing module to perform keyword extraction, entity recognition, relation extraction, and event extraction operations on the document fragments under each node of the original document tree to generate feature objects in vector or matrix form; Step S103: Create or update corresponding feature objects of different types or reference relationships of feature objects for nodes at different positions in the original document tree; Step S104: Based on the original document tree, a simplified, compressed, and paradigmatic tree-like hierarchical structure, namely the document summary tree, is generated by transforming nodes, local moving, pruning algorithms, and insertion or merging methods. Step S105: Accordingly, update the feature objects or reference relationships of each node in the document summary tree.

3. The method for intelligent information matching and retrieval based on document tree according to claim 1, characterized in that: Step S3 includes the following steps: Step S301: From Extract the root node of all document trees and load the topic feature set Vs that it points to; Step S302: Extract and load the root node of the input document tree and the topic features it points to; Step S303: Load the network model or calculation module based on word vector relevance calculation, and calculate the relevance between each element in Vs and vs; Step S304: Compare the calculated relevance value with the preset threshold ts. If it is found that the relevance between a certain element in Vs and vs exceeds ts, it is determined that the corresponding document has a match in one or more aspects of subject direction, content theme, overall viewpoint, and overall sentiment, and the document tree identifier corresponding to the element is recorded. Step S305: Finally, obtain a subset of document trees that meet the matching requirements.

4. The method for intelligent information matching and retrieval based on document tree according to claim 1, characterized in that: Step S5 includes the following steps: Step S501: From Extract the tree structure of all document trees, excluding leaf nodes, and load the set of structural features Va that it points to; Step S502: Extract and load the tree structure of the input document tree, excluding leaf nodes and their pointed structural features va; Step S503: Load the network model or calculation module based on tree similarity calculation, and calculate the tree similarity between each element in Va and va; Step S504: Compare the calculated tree similarity value with the preset threshold ta. If it is found that the similarity between a certain element in Va and the va tree exceeds ta, it is determined that there is a match that meets expectations in one or more aspects of the document chapter level, the transfer of viewpoints and ideas, and the overall writing style, and the document tree identifier corresponding to the element is recorded. Step S505: Finally, obtain a subset of document trees that meet the matching requirements.

5. The method for intelligent information matching and retrieval based on document tree according to claim 1, characterized in that: Step S7 includes the following steps: Step S701: From Extract all leaf nodes of the document tree and load the content feature set Vc that they point to; Step S702: Extract and load the leaf nodes of the input document tree and the content features (vc) they point to; Step S703: Load the network model or calculation module based on string matching and semantic similarity calculation, and calculate the similarity between each element in Vc and the content of vc; Step S704: Compare the calculated value of content similarity with the preset threshold tc. If it is found that the content similarity between some leaf nodes of a certain element in Vc and some leaf nodes of vc exceeds tc, it is determined that there is a matching of the text description similarity or text repetition rate of the local content, i.e., the document fragment, and the document tree identifier and node position corresponding to the element are recorded. Step S705: Calculate the average similarity between all leaf nodes of a certain element in Vc and vc. When it exceeds a certain preset threshold tc', it is determined that the full text content, i.e. all document fragments, has a high similarity, and the document tree identifier corresponding to the element is recorded. Step S706: Finally, obtain a subset of document trees that meet the matching requirements.

6. The method for intelligent information matching and retrieval based on document tree according to claim 2, characterized in that: In the process of document data extraction and feature processing, the method categorizes the extracted features into three types: thematic features, structural features, and content features. Thematic features include the document's theme, subject classification, keywords and word frequencies, and a digital representation of the overall sentiment. Structural features include the contextual relationships at the chapter level, chapter-level summaries, and a digital representation of keywords and word frequencies at the chapter level. Content features include summaries of document fragments at the corresponding level, keywords and word frequencies of those fragments, and a digital representation of the original textual information of those fragments. The nodes of the document tree are divided into three categories: root node, non-leaf nodes, and leaf nodes. The root node is used to represent the theme features of the document, the other non-leaf nodes are used to represent the structural features of the document, and the leaf nodes are used to represent the content features of the document. Based on the characteristics of the storage and computing resources of the actual system, each node of the document tree corresponds to a matrix, a vector, or a group of vectors, or other data structure forms that can express the characteristics of the document. The matrix includes tree arrays or variations of tree arrays; the vectors and vector groups include KD trees or variations of KD trees.

7. The method for intelligent information matching and retrieval based on document tree according to claim 1, characterized in that: In the method, the document tree information generated by the document library preprocessing is stored in a permanent or semi-permanent form, and loaded into memory or cache as needed during the matching and retrieval calculation process to improve the speed of matching and retrieval. The method stores the overall tree structure of the document tree in a graph database based on the final selected data structure form of the document tree nodes, and stores the topic features, structural features, and content features of the nodes in a suitable vector database or vector database to improve the speed of data access, reading and writing operations.

8. The method for intelligent information matching and retrieval based on document tree according to claim 1, characterized in that: The method configures matching strategies with different rules and matching thresholds with different precision to meet different application scenarios and matching retrieval intentions. During the matching retrieval calculation process, different network models or calculation modules are dynamically loaded according to different matching strategies, and result sets of different interval sizes are output according to the matching threshold.

9. The method for intelligent information matching and retrieval based on document tree according to claim 1, characterized in that: The document tree includes a first-level original document tree and a second-level document summary tree. Generating the original document tree is a prerequisite for generating the document summary tree. The original document tree is generated using an encoder-decoder network model or a pre-trained BERT or GPT model, producing the original tree-like hierarchical structure of the document data. The tree has a greater depth or more levels, more complex paths, and a larger amount of data, making it unsuitable for direct matching and retrieval operations. The document summary tree is a simplified, compressed, and paradigmatic tree-like hierarchical structure generated from the original document tree through node transformation, local movement, pruning algorithms, insertion, or merging. The relevance or similarity calculations of the information intelligent matching and retrieval method based on the document tree are all performed on the document summary tree, so as to achieve the goal of low computational load but almost no loss of any key information of the document. The nodes of the document tree are divided into topic feature nodes, structural feature nodes, and content feature nodes; The topic feature node, which is the root node of the document tree, is used to represent the topic features of the document. The feature component it points to can include the digital expression of the document's topic, subject classification, keywords and word frequencies, and sentiment information. Structural feature nodes, which are non-leaf nodes other than the root node, are used to represent the structural features of a document. The feature components they point to can include the contextual relationships at the chapter level, the summary at the chapter level, the keywords at the chapter level, and the digital representation of word frequency information. Content feature nodes, also known as leaf nodes, are used to represent the content features of a document. The feature components they point to include the summary of each document fragment at the corresponding level, the keywords and word frequencies of the document fragment, and the digital representation of the original text information of the document fragment.

10. The method for intelligent information matching and retrieval based on document tree according to claim 1, characterized in that: The aforementioned documents refer to files or data stored on computer systems or websites that are primarily text-based, including but not limited to electronic files in various formats such as doc / docx, pdf, wps, txt, xml, and caj, such as professional materials, academic literature, patent documents, conference papers, dissertations, legal documents, application documents, and journal articles. The document tree-based intelligent information matching and retrieval method employs a document tree-based intelligent information matching and retrieval system, which includes: The document tree generation module mainly extracts data information from documents and processes document features to generate document trees; the document tree storage module mainly stores the preprocessed document tree data from the document library, generally using a suitable graph database, vector database, or vector database. The model library / algorithm library mainly stores various related algorithms, network models, and NLP modules for use by the document generation module and the matching and retrieval module. The matching strategy configuration module is mainly used to configure various matching strategies, algorithms, and precision in different scenarios; the matching retrieval operation module mainly completes operations and services including document tree topic relevance, document tree structure similarity, document tree content similarity, and document tree service capabilities. The results output module outputs matching results according to the scene intent.

Images