A Fast Document Feature Extraction System Based on Pre-trained Large Models

Abstract

This invention provides a rapid document element extraction system based on a pre-trained large model, belonging to the field of computer software application technology. The system includes: a parameter domain adaptation module for textualizing documents and constructing an industry-standard corpus based on the textualization results, then adjusting a pre-defined language model using the industry-standard corpus; a dynamic document segmentation module for semantically segmenting industry-standard documents to obtain several text blocks; an entity alignment module for extracting entities and relations from the text blocks and performing entity alignment using uniform manifold approximation and projection methods; and a relational reasoning and knowledge graph completion module for completing a preliminary knowledge graph and storing the completion results. This invention eliminates the need for pre-defined rule templates or data annotation, directly improving element extraction efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer software application technology, and more specifically, to a system for rapid extraction of document elements based on a pre-trained large model. Background Technology

[0002] The core of building a knowledge graph lies in the extraction of indicator elements such as entities and relationships. Furthermore, in-depth extraction of indicator elements from documents facilitates the application expansion and value mining of standard document data knowledge. However, current indicator element extraction techniques are mostly based on rules and supervised / semi-supervised methods. These methods, due to the significant upfront investment required, cannot achieve rapid knowledge extraction. Specific problems include:

[0003] 1) High labor costs: Rule-based extraction methods require domain experts to spend time constructing fine-grained extraction rule expressions; supervised / semi-supervised models rely on large-scale labeled corpora.

[0004] 2) Insufficient semantic coverage and lack of flexibility: The rules have difficulty handling semantic variations in unstructured text (such as the equivalent expressions of "moisture content ≤ 5%" and "moisture content not exceeding 5%). When the document contains expressions not covered by the rules, this part of the knowledge will be missed.

[0005] 3) Weak cross-domain adaptability: Documents in different domains have unique semantic structures and professional terminology systems. Traditional template-based extraction technology requires redesigning the rule system when migrating across domains, making it difficult to be applicable to multiple domains.

[0006] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0007] In view of this, the present invention provides a document feature extraction system based on a pre-trained large model to solve the aforementioned problems.

[0008] To solve the above problems, the specific technical solution adopted by the present invention is as follows:

[0009] A fast document feature extraction system based on a pre-trained large model, the system comprising:

[0010] The parameter domain adaptation module is used to process documents into text and build an industry standard corpus based on the text processing results. The preset language model is then adjusted using the industry standard corpus to obtain the adjusted language model.

[0011] The dynamic document chunking module is used for context-aware dynamic semantic chunking technology to perform semantic segmentation on industry standard documents and obtain several text chunks.

[0012] The entity alignment module is used to extract entities and relations from text blocks based on the Prompt prompting project and the adjusted language model, combined with entity recognition and relation extraction techniques, and to perform entity alignment by combining uniform manifold approximation and projection methods to obtain entity alignment results.

[0013] The relational reasoning and knowledge graph completion module is used to construct a preliminary knowledge graph based on the entity alignment results, complete the preliminary knowledge graph through logical relational reasoning, and store the completion results.

[0014] Preferably, the step of text-ifying the document and constructing an industry-standard corpus based on the text-ification results, and then adjusting the preset language model using the industry-standard corpus to obtain the adjusted language model includes:

[0015] Acquire document data from different industries and build a text processing library for document textification.

[0016] Based on the document's format attributes, a text processing library is used to perform text conversion processing on the document data to obtain document data in standard text format;

[0017] Based on document data in standard text format, construct an industry-standard corpus for domain adaptation;

[0018] The pre-set language model is adjusted based on an industry standard corpus to obtain the adjusted language model.

[0019] Preferably, the context-aware dynamic semantic segmentation technology performs semantic segmentation on industry standard documents to obtain several text blocks, including:

[0020] Obtain industry standard documents and use sentence segmentation tools to parse and process them to obtain a list of sentences that retain their original order;

[0021] Using a pre-trained sentence embedding model, each sentence in the sentence list is encoded into a vector representation, resulting in the first vector matrix;

[0022] Based on the first vector matrix, a boundary detection algorithm based on cosine similarity mutation is used, combined with a dynamic threshold strategy, to perform semantic segmentation on industry standard documents, resulting in several text blocks.

[0023] Preferably, the boundary detection algorithm based on the first vector matrix, utilizing cosine similarity mutation, and combined with a dynamic threshold strategy, performs semantic segmentation on the industry standard document to obtain several text blocks, including:

[0024] Using the cosine similarity calculation formula, cosine similarity is calculated for all adjacent sentence pairs in the first vector matrix;

[0025] The cosine similarity results of all sentence pairs are sorted in descending order, and a dynamic thresholding strategy is used to determine the boundary detection threshold.

[0026] The cosine similarity of each adjacent sentence pair is compared with the boundary detection threshold. If the cosine similarity is less than or equal to the threshold, it is determined that there is a semantic boundary between the two adjacent sentence pairs and the semantic boundary point is obtained. Otherwise, it is determined that there is no semantic boundary between the two adjacent sentence pairs.

[0027] Based on the determined semantic boundary points, semantic segmentation is performed on industry standard documents to obtain several text blocks.

[0028] Preferably, the entity alignment results obtained by combining the Prompt prompting engineering and the adjusted language model with entity recognition and relation extraction techniques to extract entities and relations from text blocks, and combining uniform manifold approximation and projection methods for entity alignment, include:

[0029] For each text block, based on the Prompt prompting project and the adjusted language model, entity extraction is performed using entity recognition and relation extraction techniques to obtain entity strings;

[0030] Using a pre-trained natural language processing model, each entity string is encoded into a semantic vector of a preset dimension to obtain an entity embedding vector;

[0031] The entity embedding vector is nonlinearly reduced in dimension by uniform manifold approximation and projection algorithms to obtain the second vector matrix.

[0032] Based on the hierarchical density clustering method, the second vector matrix is ​​clustered to obtain several synonymous entity clusters, thereby achieving entity alignment and obtaining entity alignment results.

[0033] Preferably, the nonlinear dimensionality reduction of the entity embedding vector using the uniform manifold approximation and projection algorithm to obtain the second vector matrix includes:

[0034] The entity embedding vector is used as the input vector to the uniform manifold approximation and projection algorithm;

[0035] Based on the fitting and transformation method in the uniform manifold approximation and projection algorithm, a graph representation of entity embedding vectors is constructed in the preset first-dimensional space;

[0036] In the pre-defined second-dimensional space, the layout of the graph representation of the entity embedding vector is optimized to obtain the second vector matrix.

[0037] Preferably, the step of clustering the second vector matrix based on hierarchical density clustering to obtain several synonymous entity clusters to achieve entity alignment includes:

[0038] Based on the hierarchical density clustering method, noise is identified in the second vector matrix, and the identified noise points are marked.

[0039] Assign cluster labels to the non-noise points in the second vector matrix to form an initial synonymous entity cluster;

[0040] Each initial synonymous entity cluster is subjected to paradigm rotation and normalization to obtain the final synonymous entity cluster, thereby achieving entity alignment.

[0041] Preferably, the step of constructing a preliminary knowledge graph based on the entity alignment results, completing the preliminary knowledge graph through logical relationship reasoning, and storing the completion results includes:

[0042] Based on the entities and entity relationships in the entity alignment results, a preliminary knowledge graph is constructed;

[0043] Using the knowledge gap identification method, knowledge gaps are identified in the preliminary knowledge graph, generating a list of entities with insufficient connections.

[0044] For each entity in the entity target list, generate rich contextual hints for relational reasoning;

[0045] Based on rich contextual hints for relational reasoning, a modified language model is used, combined with triple verification detection, to generate verified relational triples.

[0046] The verified relation triples are stored, and the preliminary knowledge graph is completed to fill in the gaps in the knowledge graph.

[0047] Preferably, the step of using the knowledge gap identification method to identify knowledge gaps in the preliminary knowledge graph and generate a list of entities with insufficient connections includes:

[0048] Based on a pre-configured query graph database, a query operation is performed on each entity node in the preliminary knowledge graph, and the node degree of each entity node is calculated based on the query operation results.

[0049] The node degree of each entity node is compared with a preset node degree threshold. If the node degree of each entity node is less than the preset node degree threshold, the entity node is considered to be an entity with insufficient connections, and the entity with insufficient connections is regarded as a knowledge gap.

[0050] Collect all entities identified as knowledge gaps and generate a target list of entities with insufficient connectivity.

[0051] Preferably, the rich contextual hints based on relational reasoning, utilizing an adjusted language model and combined with triple verification detection, generate verified relational triples including:

[0052] Based on the adjusted language model, we perform contextual association analysis on rich contextual cues for relational reasoning to identify missing relations;

[0053] The identified missing relations are generated into triplet strings in the form of meta-triples;

[0054] The triplet validation strategy is used to validate triplet strings, retaining the valid triplet strings and deleting the invalid triplet strings, thus generating validated relation triplet strings.

[0055] The beneficial effects of this invention are as follows:

[0056] 1. This invention does not require pre-defined rule templates or data annotation, and can directly improve the efficiency of feature extraction. In addition, for feature extraction tasks in different fields, only the Prompt content needs to be adjusted to achieve adaptation, demonstrating strong scalability.

[0057] 2. The large language model used in this invention can extract elements based on the semantic environment and in combination with preset prompts, and is especially suitable for the task of extracting indicator elements in complex semantic relationship scenarios.

[0058] 3. This invention mainly uses a large oracle model for knowledge extraction because the pre-trained large model has a general semantic understanding foundation and can be quickly adapted to new domains through lightweight domain fine-tuning (such as incremental training with only industry corpus).

[0059] 4. This invention realizes intelligent text segmentation, automated extraction of indicator elements, entity alignment and relation reasoning, which significantly reduces the need for manual intervention and improves processing speed and efficiency, and is especially suitable for large-scale indicator element extraction scenarios.

[0060] 5. In entity alignment, this invention automatically integrates equivalent entities extracted across blocks through similarity clustering, reducing manual intervention; the relation reasoning module combines large-scale model semantic understanding and graph algorithms to automatically complete the relationships not established in the knowledge graph, improving the efficiency and quality of knowledge graph construction. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0062] Figure 1This is a flowchart illustrating the principle of a document element fast extraction system based on a pre-trained large model according to an embodiment of the present invention.

[0063] Figure 2 This is a block diagram illustrating the principle of a fast document element extraction system based on a pre-trained large model according to an embodiment of the present invention.

[0064] Figure 3 This is a technical flowchart of a document element fast extraction system based on a pre-trained large model according to an embodiment of the present invention.

[0065] In the picture:

[0066] 1. Parameter domain adaptation module; 2. Dynamic document segmentation module; 3. Entity alignment module; 4. Relational reasoning and knowledge graph completion module. Detailed Implementation

[0067] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0068] According to an embodiment of the present invention, a system for rapid extraction of document elements based on a pre-trained large model is provided.

[0069] Specifically, the essence of this invention lies in its inherent virtuous cycle of knowledge refinement. Relational reasoning is not only the end point of the process but also the starting point for a new round of optimization. By analyzing the structural defects of the constructed knowledge graph (such as missing relations), intelligent reasoning using a large language model is used to fill these knowledge gaps. These newly added knowledge triples, after reasoning and verification, in turn greatly enrich and strengthen the knowledge base. When processing subsequent tasks, this more complete knowledge base can provide the large language model with more accurate contextual "anchors," thereby guiding the large language model to extract elements more accurately. This self-improving mechanism of "extraction-construction-reasoning-feedback" enables the system to have the ability to continuously learn and self-evolve, which is a fundamental transcendence of existing static extraction tools.

[0070] The innovation of this architecture lies in its synergistic amplification effect. The output quality of each module directly and significantly improves the input quality and processing efficiency of subsequent modules, thus forming a chain of technological advantages. For example, the semantically coherent text blocks produced by dynamic document segmentation provide a large language model with inputs of extremely high signal-to-noise ratio, directly improving the accuracy of initial entity extraction. The higher-quality entity set then simplifies the clustering task of the entity alignment module, producing more accurate standardized entities. Finally, a high-fidelity initial knowledge graph enables the relational reasoning module to make inferences based on more reliable evidence, thereby constructing a more complete and accurate knowledge system. This interconnected and layered design constitutes the non-obvious substantive features and significant progress of this invention compared to existing technologies.

[0071] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figures 1-3 As shown, according to an embodiment of the present invention, a fast document feature extraction system based on a pre-trained large model includes:

[0072] The parameter domain adaptation module 1 is used to process documents into text and, based on the text processing results, construct an industry standard corpus. The preset language model is then adjusted using the industry standard corpus to obtain the adjusted language model.

[0073] In a preferred embodiment, the document is digitized, and an industry-standard corpus is constructed based on the digitization results. The preset language model is then adjusted using the industry-standard corpus to obtain the adjusted language model, which includes:

[0074] Acquire document data from different industries and build a text processing library for document textification.

[0075] Based on the document's format attributes, a text processing library is used to perform text conversion processing on the document data to obtain document data in standard text format;

[0076] Based on document data in standard text format, construct an industry-standard corpus for domain adaptation;

[0077] The pre-set language model is adjusted based on an industry standard corpus to obtain the adjusted language model.

[0078] Specifically, to overcome the enormous computational cost, storage overhead, and catastrophic forgetting risk associated with full fine-tuning, this invention employs a specific, non-obvious, and technically advantageous method—Low-Rank Adaptation (LoRA)—to achieve the tool's speed and high scalability. The specific implementation process is as follows:

[0079] 1.1 The efficient fine-tuning principle of Low-Rank Adaptive Modeling (LoRA): The core idea of ​​LoRA is that when adapting a pre-trained large language model to downstream tasks, the change in model weights (ΔW) inherently possesses low-rank characteristics. Therefore, it is unnecessary to update all billions or even more parameters of the model. This invention freezes the original weight matrix of the pre-trained model. W 0, and inject two trainable, small, low-rank matrices A and B into a specific layer of the model (typically the self-attention module in a Transformer architecture). Mathematically, the weight update amount Δ W It is decomposed and approximated as the product of these two low-rank matrices: Δ W =BA.

[0080] Wherein, if the original weight matrix W The dimension of 0 is d × k Then the dimension of matrix B is d × r The dimension of matrix A is r × k Here, the rank *r* is a key hyperparameter, and its value is much smaller than the original dimension. d and k ,Right now r ≪min( d , k ). By training only the matrix A and B The number of trainable parameters is reduced by thousands of times. Ultimately, the effective weights of the model for this task are:

[0081] W′ = W 0+BA;

[0082] 1.2 The LoRA implementation process for industry-specific corpora includes:

[0083] (1) Construction and preprocessing of industry corpus; text-based processing of PDF, DOCX and other document formats for specific industries. The PyMuPDF library is used to process complex PDF layouts and the python-docx library is used to process DOCX documents, converting the original files into clean text format and constructing a dedicated corpus for domain adaptation.

[0084] Specifically, for DOCX files, the python-docx library can be used. This library can directly read text paragraphs, tables, and other content from .docx files and extract them into string format.

[0085] For PDF files, use the PyMuPDF library, a powerful tool capable of handling complex PDF layouts. It can extract not only regular text but also multi-column layouts, headers and footers, and table data, transforming them into an ordered, clean text stream while preserving the original information to the greatest extent possible.

[0086] A specialized corpus refers to a collection of plain text documents such as standards, specifications, and reports related to a specific industry sector.

[0087] For example, to adapt to the "food safety" domain, we would typically collect all relevant national standards (GB), industry standards (QB), and other PDF and Word documents. Through the conversion process described above, these documents are all processed into plain text files in .txt format. This collection of TXT files constitutes a "food safety-specific corpus," which is used for subsequent LoRA fine-tuning of the large language model.

[0088] (2) LoRA Adapter Configuration; LoraConfig is used to precisely configure the adapter, demonstrating a deep application of this technology; LoraConfig provides precise adapter configuration. The core parameters that need to be configured are as follows:

[0089] rank r The dimension of the low-rank matrix represents a trade-off between performance and efficiency. By setting the rank of the low-rank matrix, a value of r=64 represents a trade-off between model fit and training efficiency.

[0090] The scaling factor (lora_alpha) is used to adjust the scaling factor of the merged weights. By setting a scaling factor, the size of the merged weights can be adjusted; for example, it can be set to... r Twice as of [time], to stabilize the training process.

[0091] Target modules: Specifies which parts of the model the LoRA matrix should be injected into. This invention explicitly specifies the query (q) and value (v) matrices in the self-attention layer. The LoRA matrix is ​​injected into the query (q) and value (v) matrices in the Transformer's self-attention layer. q p roj ) and value ( v p roj Projection matrices. Existing research has shown that adapting to these modules is the most efficient.

[0092] Dropout (LoRa_dropout): This is a regularization technique to prevent overfitting. Applying Dropout to the LoRa matrix serves as a regularization method to prevent the model from overfitting on small-scale, domain-specific data. In machine learning, Dropout is often translated as random deactivation or dropout method. The dropout rate used in this invention refers to a key parameter when implementing this method, namely the proportion of neurons (or parameters) that need to be dropped or deactivated.

[0093] Specifically, Dropout is a regularization technique used during neural network training. It works by randomly and temporarily dropping a subset of neurons from the network with a certain probability (i.e., a dropout rate) in each training iteration. This means that these dropped neurons will not participate in forward or backward propagation in that iteration. This prevents the model from becoming overly reliant on any single neuron or feature, thus forcing the network to learn more robust and generalizable features. Its ultimate goal, as described in the patent, is to prevent the model from overfitting to small-scale, domain-specific data.

[0094] Here, `lora_dropout` is a compound term that can be understood as the random dropout rate applied to LoRA, and it is also a specific configuration parameter. It indicates that the Dropout technique is specifically applied to the LoRA matrix. In this invention, it means that during the fine-tuning process, a portion of the parameters in the low-rank LoRA matrices A and B will be randomly set to zero at a set dropout rate to achieve a regularization effect and prevent the model from overfitting to a small-scale, industry-specific corpus.

[0095] Furthermore, during training, only the parameters of LoRA matrices A and B are updated via gradient descent, while the hundreds of millions of parameters of the base model remain frozen. After training, the generated LoRA adapter file is extremely small (typically only a few megabytes). During inference, the learned weights are incremented.

[0096] In addition, BA can be seamlessly integrated with the original weights, meaning that the LoRA adapter provides domain expertise without introducing any additional inference latency, which is crucial for enabling fast extraction tools.

[0097] 1.3 Supporting a dynamically adaptable architecture, the application of LoRA technology is not only an efficiency optimization but also a fundamental innovation in system architecture, giving rise to a novel, non-obvious multi-tenant service model. This invention leverages this characteristic to construct a multi-tenant inference platform: the system deploys a shared, general-purpose base language model in the background. When a document processing request for a specific domain is received (e.g., a construction industry standard), the system can dynamically and almost instantaneously load the corresponding construction domain LoRA adapter from storage into the GPU memory and combine it with the base model to process the request. If the next request concerns medical device regulations, the system will switch to the medical domain LoRA adapter.

[0098] This dynamically loaded, instant-switching architecture is a major innovation at the system level. It perfectly solves the core pain point of weak cross-domain adaptability mentioned in the background technology, elevating the invention from a single-purpose tool to an infinitely scalable, flexible, and efficient platform.

[0099] Dynamic document chunking module 2 is used for context-aware dynamic semantic chunking technology to perform semantic segmentation on industry standard documents and obtain several text chunks.

[0100] As a preferred embodiment, the context-aware dynamic semantic segmentation technology performs semantic segmentation on industry standard documents to obtain several text blocks, including:

[0101] Obtain industry standard documents and use sentence segmentation tools to parse and process them to obtain a list of sentences that retain their original order;

[0102] Using a pre-trained sentence embedding model, each sentence in the sentence list is encoded into a vector representation, resulting in the first vector matrix;

[0103] Based on the first vector matrix, a boundary detection algorithm based on cosine similarity mutation is used, combined with a dynamic threshold strategy, to perform semantic segmentation on industry standard documents, resulting in several text blocks.

[0104] In a preferred embodiment, the boundary detection algorithm based on the first vector matrix, utilizing cosine similarity mutation, and combined with a dynamic threshold strategy, performs semantic segmentation on industry standard documents to obtain several text blocks, including:

[0105] Using the cosine similarity calculation formula, cosine similarity is calculated for all adjacent sentence pairs in the first vector matrix;

[0106] The cosine similarity results of all sentence pairs are sorted in descending order, and a dynamic thresholding strategy is used to determine the boundary detection threshold.

[0107] The cosine similarity of each adjacent sentence pair is compared with the boundary detection threshold. If the cosine similarity is less than or equal to the threshold, it is determined that there is a semantic boundary between the two adjacent sentence pairs and the semantic boundary point is obtained. Otherwise, it is determined that there is no semantic boundary between the two adjacent sentence pairs.

[0108] Based on the determined semantic boundary points, semantic segmentation is performed on industry standard documents to obtain several text blocks.

[0109] Specifically, this invention abandons the traditional fragile block segmentation method based on regular expressions (Regex) and adopts an advanced, context-aware, dynamic semantic block segmentation technology. The specific implementation steps are as follows:

[0110] 2.1 Sentence Vectorization Based on Transformer Embedding Model: Traditional regular expression segmentation methods have a fundamental flaw: they are entirely based on the surface syntactic structure of the text (such as punctuation and line breaks) and cannot understand semantic content. Therefore, they often incorrectly segment semantically connected clauses or fail to effectively split a long paragraph containing multiple topics, thus creating noisy and semantically incomplete "dirty" data blocks for downstream LLM extraction tasks.

[0111] The solution proposed in this invention is "Semantic Splitting." It utilizes vectors and cosine similarity calculated by SBERT, not to find similarity points, but to detect "abrupt changes in similarity." When the semantic similarity score between adjacent sentences drops sharply, the system determines that a topic switch has occurred and sets it as the semantic boundary. By providing the model with text blocks that are highly semantically cohesive and have an extremely high signal-to-noise ratio, the accuracy and efficiency of subsequent information extraction are fundamentally improved. The specific steps are as follows:

[0112] The entire document is parsed into an ordered list of sentences. This is done using sentence segmentation tools from a mature NLP library (NLTK). These tools don't simply segment by period; instead, they use pre-trained models that accurately identify and handle complex sentence boundaries, such as periods within abbreviations (e.g., Mr.). A list of strings that preserves the original sentence order is generated, for example, ["Sentence 1", "Sentence 2", "Sentence 3", ...]. This list will be used as input for SBERT encoding.

[0113] Using a specially trained sentence embedding model (Sentence-BERT), such as all-MiniLM-L6-v2, each sentence is encoded into a high-dimensional vector (Embedding) (i.e., the first vector matrix is ​​constructed based on the encoded high-dimensional vectors). This type of model is optimized to ensure that the cosine similarity between generated vectors accurately reflects the semantic similarity between the original sentence pairs.

[0114] Specifically, when encoding each sentence into a high-dimensional vector, a pre-trained Sentence-BERT model, such as all-MiniLM-L6-v2, is loaded from an open-source library (Hugging FaceTransformers). The generated list of sentences is then input into the model's `encode()` method. The `encode()` method generates a high-dimensional (384-dimensional for all-MiniLM-L6-v2) floating-point vector for each sentence in the list. These vectors are designed to bring semantically similar sentences closer together in the vector space. This results in a vector matrix (the first vector matrix), where each row corresponds to a sentence from the original document.

[0115] 2.2 A boundary detection algorithm based on cosine similarity mutation is proposed, which iteratively traverses all adjacent sentence pairs in the document. sentence i , sentence i+1 ), and calculate their corresponding embedding vectors ( v i , v i+1 Cosine similarity between Similarity :

[0116] ;

[0117] The cosine similarity value ranges from -1 to 1, with values ​​closer to 1 indicating greater semantic similarity. A sharp drop in similarity scores between adjacent sentences indicates a topic shift, necessitating the establishment of a semantic boundary. To avoid the rigidity issues associated with fixed thresholds, this invention employs a dynamic threshold strategy. For example, the system can calculate the distribution of similarity scores for all adjacent sentences in a document and set the threshold at a point significantly deviating from the mean (e.g., mean minus 1.5 standard deviations). Another more robust approach is to identify semantic breakpoints when the similarity score falls into the lowest percentile of the overall document similarity score distribution (e.g., the lowest 5%). This adaptive method allows the segmentation process to self-adjust based on the language style and structure of different documents. Ultimately, the document is segmented into a series of semantic blocks, each consisting of a string of highly topic-related sentences. This enables large language models to create optimal input contexts, fundamentally improving the performance of core extraction tasks.

[0118] The semantic segmentation method proposed in this invention aims to maximize the semantic signal-to-noise ratio within each text block. Each generated block is designed to be internally coherent and focused on a single subtopic. When such clean and concentrated text blocks are fed into LLM for feature extraction, the task of identifying entities and their relationships becomes far less ambiguous, thereby directly and significantly improving the precision and recall of downstream extraction modules.

[0119] Entity alignment module 3 is used to extract entities and relations from text blocks based on the Prompt prompting project and the adjusted language model, combined with entity recognition and relation extraction technology, and to perform entity alignment by combining uniform manifold approximation and projection methods to obtain entity alignment results.

[0120] In a preferred embodiment, the text blocks are segmented for entity and relation extraction based on the Prompt prompting engineering and the adjusted language model, combined with entity recognition and relation extraction techniques. Entity alignment is then performed using uniform manifold approximation and projection methods, resulting in entity alignment results including:

[0121] For each text block, based on the Prompt prompting project and the adjusted language model, entity extraction is performed using entity recognition and relation extraction techniques to obtain entity strings;

[0122] It's important to note that Prompt engineering refers to a technical approach that guides large language models (LLMs) to complete specific tasks more accurately and efficiently by carefully designing and optimizing the text instructions (i.e., prompts) input to them. The goal is to optimize the way questions are asked so that the model understands the task intent and outputs the expected results without requiring parameter adjustments to the model itself.

[0123] Specifically, in the process of entity extraction, entity relation extraction is also required, which means extracting the element indicators from the large language model. The indicator elements include core content such as entities and relations, which can form the key elements needed to form a knowledge graph.

[0124] Using a pre-trained natural language processing model, each entity string is encoded into a semantic vector of a preset dimension to obtain an entity embedding vector;

[0125] The entity embedding vector is nonlinearly reduced in dimension by uniform manifold approximation and projection algorithms to obtain the second vector matrix.

[0126] In a preferred embodiment, the nonlinear dimensionality reduction of the entity embedding vector using a uniform manifold approximation and projection algorithm to obtain the second vector matrix includes:

[0127] The entity embedding vector is used as the input vector to the uniform manifold approximation and projection algorithm;

[0128] Based on the fitting and transformation method in the uniform manifold approximation and projection algorithm, a graph representation of entity embedding vectors is constructed in the preset first-dimensional space (high-dimensional space);

[0129] In the pre-defined second-dimensional space (low-dimensional space), the layout of the graph representation of the entity embedding vector is optimized to obtain the second vector matrix.

[0130] Based on the hierarchical density clustering method, the second vector matrix is ​​clustered to obtain several synonymous entity clusters, thereby achieving entity alignment and obtaining entity alignment results.

[0131] In a preferred embodiment, the step of clustering the second vector matrix based on hierarchical density clustering to obtain several synonymous entity clusters to achieve entity alignment includes:

[0132] Based on the hierarchical density clustering method, noise is identified in the second vector matrix, and the identified noise points are marked.

[0133] Assign cluster labels to the non-noise points in the second vector matrix to form an initial synonymous entity cluster;

[0134] Each initial synonymous entity cluster is subjected to paradigm rotation and normalization to obtain the final synonymous entity cluster, thereby achieving entity alignment.

[0135] Specifically, when performing entity alignment, a non-obvious, multi-stage entity alignment process is adopted to standardize entities extracted from different text blocks that refer to the same concept but have different expressions. The specific implementation steps are as follows:

[0136] 3.1 High-Dimensional Entity Embedding and UMAP Dimensionality Reduction: After the initial extraction stage, the system obtains a series of entity strings, such as moisture content, water content, and hydration level. Alignment aims to group these synonyms or near-synonyms into one category. Using the SBERT model, each unique entity string is encoded into a high-dimensional semantic vector. This maps semantically equivalent terms to neighboring points in a high-dimensional space.

[0137] The Uniform Manifold Approximation and Projection (UMAP) algorithm performs nonlinear dimensionality reduction on entity embeddings (from 384 dimensions to 10 dimensions). UMAP can preserve meaningful topological structure between data points to the greatest extent while reducing dimensionality, which is a key preprocessing step to ensure that subsequent clustering algorithms can run accurately and robustly.

[0138] Specifically, when creating an instance of the UMAP algorithm, key parameters can be configured, such as the target dimension n_components (set to 10 dimensions in this invention), and n_neighbors and min_dist for controlling manifold structure learning. This is done by calling the instance's fit_transform method, taking a high-dimensional entity vector as input. This method executes the core UMAP algorithm: constructing a graph representation of the data in the high-dimensional space and optimizing the layout of this graph in the low-dimensional space to preserve the original topology to the greatest extent possible. This generates a low-dimensional (10-dimensional) vector matrix (i.e., the second vector matrix), where each row corresponds to an entity. This low-dimensional representation preserves the semantic proximity relationships between entities, providing high-quality input for subsequent HDBSCAN clustering.

[0139] 3.2 Semantic Grouping Based on Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) Algorithm. HDBSCAN is an advanced algorithm that does not require a pre-defined number of clusters, can handle clusters of arbitrary shapes, and can automatically identify noise. Its key parameter configurations are as follows:

[0140] min_cluster_size: An intuitive parameter that defines the minimum number of synonyms required to form a valid entity concept (e.g., set to 2). This allows the system to automatically ignore isolated terms that may be extracted incorrectly, treating them as noise.

[0141] Metric: The distance metric used in the space after UMAP dimensionality reduction, usually 'euclidean' (Euclidean distance).

[0142] HDBSCAN's core technological advantages lie in:

[0143] It can automatically discover the number of naturally occurring clusters of entity concepts in the data based on the density distribution of the data itself. It can identify complex, non-spherical clusters, which is crucial for capturing complex relationships between semantic concepts. It marks entity points that do not belong to any dense region as noise (labeled -1). This feature provides a fully automatic mechanism to filter out irrelevant or erroneous extraction results, thereby greatly improving the purity and quality of the final knowledge graph.

[0144] Specifically, when identifying noise, the algorithm seeks out regions with sufficiently high density as clusters. Any point that does not belong to any high-density region is automatically marked as a noise point (usually labeled -1). These noise-marked points are typically isolated or may be incorrectly extracted terms; the system automatically ignores them, thus improving data purity. The algorithm assigns a cluster label (an integer) to non-noise points. All entity vectors with the same label are grouped together, representing that they refer to the same real-world concept. For example, water content, moisture percentage, and moisture content might be grouped into the same cluster.

[0145] Furthermore, for each identified cluster, the system selects a normalized representation (CanonicalForm), for example, choosing the entity closest to the cluster center as its representative. All other entity representations within that cluster are then mapped to this normalized representation, thus achieving cross-block entity alignment and ensuring naming consistency across the knowledge graph.

[0146] 3.3 Paradigm Selection and Standardization: After HDBSCAN completes the grouping of synonymous entities, the system needs to select a unique, canonical form for each cluster. A robust approach is to select the entity that is closest to all other points within the cluster in the embedding space (i.e., the cluster's centroid or medoid) as the representative of that cluster. Subsequently, all other entity representations within that cluster will be mapped to this paradigm, thus ensuring consistency in entity naming across the entire knowledge graph.

[0147] To clearly demonstrate the innovation of this invention in algorithm selection, Table 1 compares the applicability of traditional methods and the UMAP-HDBSCAN pipeline used in this invention for entity alignment tasks.

[0148] Table 1 Comparison between traditional methods and the method used in this invention

[0149]

[0150] The relational reasoning and knowledge graph completion module 4 is used to construct a preliminary knowledge graph based on the entity alignment results, complete the preliminary knowledge graph through logical relational reasoning, and store the completion results.

[0151] In a preferred embodiment, the step of constructing a preliminary knowledge graph based on the entity alignment results, performing completion processing on the preliminary knowledge graph through logical relationship reasoning, and storing the completion results includes:

[0152] Based on the entities and entity relationships in the entity alignment results, a preliminary knowledge graph is constructed;

[0153] Using the knowledge gap identification method, knowledge gaps are identified in the preliminary knowledge graph, generating a list of entities with insufficient connections.

[0154] In a preferred embodiment, the step of using the knowledge gap identification method to identify knowledge gaps in the preliminary knowledge graph and generate a list of entities with insufficient connections includes:

[0155] Based on a pre-configured query graph database, a query operation is performed on each entity node in the preliminary knowledge graph, and the node degree of each entity node is calculated based on the query operation results.

[0156] The node degree of each entity node is compared with a preset node degree threshold. If the node degree of each entity node is less than the preset node degree threshold, the entity node is considered to be an entity with insufficient connections, and the entity with insufficient connections is regarded as a knowledge gap.

[0157] Collect all entities identified as knowledge gaps and generate a target list of entities with insufficient connectivity.

[0158] For each entity in the entity target list, generate rich contextual hints for relational reasoning;

[0159] Based on rich contextual hints for relational reasoning, a modified language model is used, combined with triple verification detection, to generate verified relational triples.

[0160] In a preferred embodiment, the rich contextual hints based on relational reasoning, utilizing an adjusted language model and combined with triplet verification detection, generate verified relational triplets including:

[0161] Based on the adjusted language model, we perform contextual association analysis on rich contextual cues for relational reasoning to identify missing relations;

[0162] The identified missing relations are generated into triplet strings in the form of meta-triples;

[0163] The triplet validation strategy is used to validate triplet strings, retaining the valid triplet strings and deleting the invalid triplet strings, thus generating validated relation triplet strings.

[0164] The verified relation triples are stored, and the preliminary knowledge graph is completed to fill in the gaps in the knowledge graph.

[0165] It should be noted that the relational reasoning and knowledge graph completion module 4 elevates the system from a simple data extraction tool to a knowledge engine with basic reasoning capabilities. It introduces a complex knowledge completion closed loop driven by an LLM (Large Language Model). Specifically, it includes the following implementation steps:

[0166] 4.1 Knowledge Gap Identification: The reasoning process begins with a structural analysis of the initially constructed knowledge graph to locate knowledge gaps. This involves much more than simply finding isolated nodes with zero degree. The algorithm of this invention aims to identify nodes with insufficient connectivity, i.e., entities whose connectivity (degree) is below a preset threshold for a specific domain. Although these entities have some connections, they may still lack crucial relationships, representing weak links in the knowledge graph. Specifically, this includes:

[0167] A structural analysis is performed on the initially constructed knowledge graph. This is typically achieved by querying a graph database (such as Neo4j). For each entity node in the graph, a query is executed to calculate its degree, i.e., the number of relations (edges) connected to that node. The degree of each node is compared to a preset, domain-specific threshold. If a node's degree is below this threshold, it is identified as an underconnected entity, i.e., a knowledge gap. All entities identified as underconnected are collected into a list as targets for subsequent relational reasoning modules.

[0168] 4.2 Rich Contextual Hint Generation for Relational Reasoning: For each identified entity with insufficient connectivity, a structured hint containing rich contextual information is dynamically generated to guide LLM in high-quality relational reasoning. This hint engineering technique draws on the ideas of context-aware prompting and graph-reinforced generation (GraphRAG), and its components include:

[0169] Graph schema provides a textual description of the legal node labels and relationship types in a knowledge graph. This significantly constrains the output space of LLM, ensuring that it generates results that conform to predefined specifications.

[0170] The target entity is the name of the entity whose connection is insufficient and requires relation completion.

[0171] Local text context provides the original text block containing the entity, obtained from semantic chunking. It offers the most direct, unstructured contextual information for LLM.

[0172] Local graph context provides a textual description of the entity's existing one-hop neighbor nodes and their relationships within the current knowledge graph. It provides structured, known relational information for LLM.

[0173] The task instruction provides a clear question to guide LLM reasoning and predict a missing relation. For example: "Given entity '[entity name]', with a textual context of '...', and an existing relation of '...' in the graph. Based on the given graph pattern, predict the most likely missing relation and provide the answer in the form of a triple of (head entity, relation, tail entity)."

[0174] 4.3 LLM-Driven Relation Prediction and Validation: After receiving the rich contextual hints mentioned above, the LLM generates one or more candidate relation triples. To prevent the model from generating illusions and to ensure the accuracy of the knowledge, these candidate triples must pass through a validation layer, including:

[0175] The pattern conformity check requires that the predicted relationship types and entity types match the provided graph pattern.

[0176] Fact consistency checks are used for secondary verification, which requires the LLM to look for evidence in the original document that supports the newly generated triples.

[0177] Redundancy checks are performed to ensure that the triple does not yet exist in the knowledge graph.

[0178] Only verified triples are officially added to the knowledge graph, thus effectively "filling" the knowledge gap.

[0179] Specifically, such as Figure 3As shown in the diagram, h (head) refers to the head entity, i.e., the subject of a relation. r (relation) refers to the relation, i.e., the edge connecting the head and tail entities. t (tail) refers to the tail entity, i.e., the object of a relation. The LLM receives and understands the constructed rich contextual hints. These hints contain various information, including graph patterns, target entities, local textual context, and local graph context. The LLM model leverages its powerful language understanding and reasoning capabilities to correlate information from these different sources. It analyzes clues in the textual context and combines them with known relations in the graph context to infer the most likely missing relation. Based on the explicit task instructions in the hint (e.g., "...predict a most likely missing relation and give the answer in the form of a triple of (head entity, relation, tail entity)"), the LLM generates text in the specified format. The type of relation it predicts is constrained by the graph pattern provided in the hint. The LLM outputs one or more relation triple strings that it considers most plausible, such as (moisture content, upper limit, 5%). These are the candidate relation triples.

[0180] For the obtained candidate relation triples, such as entity A, relation R, entity B, and the associated original text block, the system dynamically generates a new prompt for the validation task. This prompt directly asks the LLM to look for evidence in the original text. For example: "In the following text: '[original text block]', is there evidence supporting the existence of 'relation R' between 'entity A' and 'entity B'? If so, please quote the original sentence directly." The LLM receives this validation prompt and re-examines the original text, attempting to find direct or indirect evidence that supports the triple. If the LLM can return one or more sentences from the original text as evidence, and these sentences do indeed support the relation, the triple passes secondary validation. If the LLM cannot find evidence, or the returned "evidence" is irrelevant to the triple, the candidate triple is considered to have failed validation and is discarded.

[0181] Specifically, the triple verification optimization achieves data cleaning optimization, which can remove triples that do not meet the requirements, thereby improving the accuracy of the knowledge graph after completion.

[0182] 4.4 The symbiotic reasoning loop of LLM and knowledge graph: In this process, LLM is the active extractor, while KG is the passive knowledge container. A second path is introduced: KG→LLM→KG. The system utilizes LLM to reason about the structural information of the knowledge graph itself. A positive feedback loop is achieved, specifically including:

[0183] The initial extraction phase constructs a preliminary KG that may have fragmentation and incompleteness issues.

[0184] The relational reasoning module intervenes, using the reasoning capabilities of LLM to enrich and complete this KG, generating a higher-quality V2 version KG.

[0185] When processing the next document or task, this structured knowledge can be back-injected into the LLM's hints (i.e., the GraphRAG pattern), providing the LLM with a more solid factual foundation, thereby significantly improving the accuracy and depth of all future extraction tasks.

[0186] This dynamic symbiotic architecture, in which LLM and KG mutually promote and evolve together, provides a powerful technical solution with self-improving capabilities to address the challenges of knowledge integrity and accuracy in the field of automated knowledge base construction.

[0187] In summary, by utilizing the above-mentioned technical solutions of this invention, the present invention can directly improve the efficiency of element extraction without pre-defining rule templates or data annotation. Furthermore, for element extraction tasks in different domains, adaptation can be achieved simply by adjusting the Prompt content, demonstrating strong scalability. The large language model used in this invention can extract elements based on the semantic environment and pre-set prompts, making it particularly suitable for indicator element extraction tasks in complex semantic relationship scenarios. This invention primarily uses a large oracle model for knowledge extraction because the pre-trained large model possesses a general semantic understanding foundation, allowing for rapid adaptation to new domains through lightweight domain fine-tuning (e.g., incremental training with only an industry corpus). This invention achieves intelligent text segmentation, automated indicator element extraction, entity alignment, and relation reasoning, significantly reducing the need for manual intervention and improving processing speed and efficiency, making it particularly suitable for large-scale indicator element extraction scenarios. In entity alignment, this invention automatically integrates equivalent entities extracted across blocks through similarity clustering, reducing manual intervention; the relation reasoning module combines large model semantic understanding with graph algorithms to automatically complete missing relationships in the knowledge graph, improving the efficiency and quality of knowledge graph construction.

[0188] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.

[0189] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A pre-trained large model-based document element rapid extraction system, characterized in that, The system comprises: a parameter field adaptation module, configured to perform text processing on a document, and based on a text processing result, construct an industry standard corpus, adjust a preset language model based on the industry standard corpus, and obtain an adjusted language model; a dynamic document blocking module, configured to perform semantic segmentation processing on an industry standard document based on a context-aware dynamic semantic blocking technology, and obtain a plurality of text blocks; an entity alignment module, configured to, for each text block, perform entity extraction based on a Prompt prompt engineering and the adjusted language model, in combination with entity recognition and relationship extraction technology, obtain an entity string, encode each entity string into a semantic vector of a preset dimension using a pre-trained natural language processing model, obtain an entity embedding vector, perform nonlinear dimension reduction on the entity embedding vector using a uniform manifold approximation and projection algorithm, obtain a second vector matrix, and perform clustering processing on the second vector matrix based on a hierarchical density clustering method, obtain a plurality of synonymous entity clusters, and achieve entity alignment to obtain an entity alignment result; a relationship reasoning and knowledge graph completion module, configured to construct a preliminary knowledge graph according to the entity alignment result, perform completion processing on the preliminary knowledge graph through logical relationship reasoning, and store a completion result.

2. The pre-trained large model-based document element rapid extraction system according to claim 1, wherein The text processing on the document, and based on a text processing result, constructing an industry standard corpus, adjusting a preset language model based on the industry standard corpus, and obtaining an adjusted language model comprises: acquiring document data of different industries, and constructing a text processing library for document text processing; based on the format attribute of the document, performing text conversion processing on the document data using the text processing library to obtain document data in a standard text format; based on the document data in the standard text format, constructing an industry standard corpus for field adaptation; adjusting the preset language model based on the industry standard corpus to obtain the adjusted language model. 3.The document element fast extraction system based on a pre-trained large model of claim 1, wherein, The context-aware dynamic semantic blocking technology performs semantic segmentation processing on the industry standard document to obtain a plurality of text blocks, which comprises: acquiring the industry standard document, and performing parsing processing on the industry standard document using a sentence segmentation tool to obtain a sentence list that retains the original order; encoding each sentence in the sentence list into a vector representation using a pre-trained sentence embedding model to obtain a first vector matrix; based on the first vector matrix, using a boundary detection algorithm based on cosine similarity mutation, and in combination with a dynamic threshold strategy, performing semantic segmentation processing on the industry standard document to obtain a plurality of text blocks.

4. The pre-trained large model-based document element rapid extraction system according to claim 3, characterized in that, The context-aware dynamic semantic blocking technology performs semantic segmentation processing on the industry standard document to obtain a plurality of text blocks, which comprises: using a cosine similarity calculation formula to calculate the cosine similarity of all adjacent sentence pairs in the first vector matrix; sorting the cosine similarity calculation results of all sentence pairs in descending order, and determining a boundary detection threshold value using a dynamic threshold strategy; The cosine similarity calculation result of each adjacent sentence pair is compared with a boundary detection threshold, if the cosine similarity calculation result is less than or equal to the threshold, it is determined that there is a semantic boundary between the two adjacent sentence pairs, a semantic boundary point is obtained, otherwise, it is determined that there is no semantic boundary between the two adjacent sentence pairs; Based on the determined semantic boundary point, the industry standard document is subjected to semantic segmentation processing to obtain a plurality of text blocks.

5. The pre-trained large model-based document element rapid extraction system according to claim 4, characterized in that, The second vector matrix is obtained by using the uniform manifold approximation and projection algorithm to perform nonlinear dimensionality reduction on the entity embedding vector, including: The entity embedding vector is input as an input vector into the uniform manifold approximation and projection algorithm; Based on the fitting and transformation method in the uniform manifold approximation and projection algorithm, a graph representation of the entity embedding vector is constructed in a preset first-dimensional space; In a preset second-dimensional space, the graph representation of the entity embedding vector is subjected to layout optimization to obtain a second vector matrix. 6.The document element fast extraction system based on a pre-trained large model according to claim 5, wherein, The second vector matrix is subjected to clustering processing based on the hierarchical density clustering method to obtain a plurality of synonymous entity clusters to realize entity alignment, including: Based on the hierarchical density clustering method, noise points in the second vector matrix are identified and marked; Non-noise points in the second vector matrix are assigned cluster labels to form initial synonymous entity clusters; Each initial synonymous entity cluster is subjected to paradigm rotation and standardization processing to obtain a final synonymous entity cluster to realize entity alignment.

7. The pre-trained large model-based document element rapid extraction system according to claim 1, wherein, The preliminary knowledge graph is constructed according to the entity alignment result, the preliminary knowledge graph is subjected to completion processing through logical relationship reasoning, and the completion result is stored, including: Based on the entities and entity relationships in the entity alignment result, a preliminary knowledge graph is constructed; Using the knowledge gap identification method, the knowledge gap identification of the preliminary knowledge graph is performed to generate an entity target list with insufficient connections; For each entity in the entity target list, a rich context prompt for relationship reasoning is generated; Based on the rich context prompt for relationship reasoning, an adjusted language model is used in combination with triple verification detection to generate verified relationship triples; The verified relationship triples are stored, and the preliminary knowledge graph is completed to repair the gaps in the knowledge graph.

8. The pre-trained large model-based document element rapid extraction system according to claim 7, wherein, The knowledge gap identification method is used to identify the knowledge gap of the preliminary knowledge graph, and an entity target list with insufficient connections is generated, including: Based on the pre-configured query graph database, a query operation is performed on each entity node in the preliminary knowledge graph, and the node degree of each entity node is calculated according to the query operation result; Each entity node degree is compared with a preset node degree threshold, if each entity node degree is less than the preset node degree threshold, the entity node is considered as an entity with insufficient connections, and the entity with insufficient connections is considered as a knowledge gap; All entities identified as knowledge gaps are collected to generate an entity target list with insufficient connections.

9. The pre-trained large model based document element rapid extraction system according to claim 8, characterized in that, Based on the rich context prompt for relationship reasoning, an adjusted language model is used in combination with triple verification detection to generate verified relationship triples, including: Based on the adjusted language model, the rich context prompt for relation reasoning is analyzed in context association, and missing relations are identified; The identified missing relations are generated into triple strings in the form of meta-triplets; Based on the triple verification strategy, the triple strings are verified, and the triple strings that pass the verification are retained, and the triple strings that fail the verification are deleted, generating verified relation triples.

Images