A method and system for automatic document structuring processing based on LLM

Abstract

The present invention discloses a method and system for automatic document structuring based on LLM, which relates to the field of document processing technology and includes the following steps: S1, document input and preprocessing: removing special characters from disordered text, unifying the encoding format, and adjusting its semantic density; S2, de-redundancy and keyword extraction: removing repeated information from the disordered text and extracting the first keyword; S3, semantic extraction and structuring: parsing the disordered text sentence by sentence based on the first keyword, extracting the main meaning and related semantic information corresponding to each sentence, and obtaining extraction results, including the second keyword and sentence meaning of the same sentence; S4, semantic stack construction and updating: constructing and updating the semantic stack based on the extraction results to store semantic units; one semantic unit corresponds to one extraction result; S5, structured output: outputting the data in the semantic stack in a preset format. The present invention is suitable for the structured processing of complex disordered documents in multiple fields and types.

Description

Technical Field

[0001] The present invention relates to the technical field of document processing, and in particular to an LLM-based automatic document structured processing method and system. Background Art

[0002] Document structuring refers to organizing and arranging document content according to a specific logic and hierarchy to facilitate information retrieval, understanding, and processing. Structured documents typically have a clear format and organization, making them easier for computers to understand and process. This effectively supports automated processes and data analysis, and makes information more organized, searchable, and easier to analyze. Document structuring also allows readers to quickly find the information they need, making it easier for users to read intuitively.

[0003] Document structuring involves the processing of text. Traditional text processing methods usually analyze and organize documents by artificially constructing document structure rules and using rule matching technology. For example, in certain documents in specific fields, specific paragraph contents are identified by predefined keywords or fixed formats to determine the logical structure of the document. For example, the patent document with announcement number CN117093589B provides a method and device for storing unstructured data, which requires the use of several specific storage rules to match unstructured data / documents. However, such methods are highly dependent on rules and have poor scalability. They are difficult to cope with scenarios with complex document structures or diverse content, especially in the processing of disordered, non-standardized or cross-domain documents, where rule matching methods are prone to failure. For example, when there are implicit relationships or non-explicit structural information in a document, rule matching cannot accurately capture them.

[0004] Basic natural language processing techniques use machine learning to model large amounts of text data to achieve document structuring and semantic understanding. For example, semantic analysis can be performed by training bag-of-words models or statistical methods to generate document analysis results with a certain level of semantic understanding. Building on this foundation, deep learning models such as BERT (Bidirectional Encoder Representation from Transformers), a pre-trained model developed by Google AI Research that specializes in reading comprehension, are widely used for semantic understanding and extraction of text. By modeling contextual relationships, these models can generate coherent summaries or extract key information. While these methods have improved document processing to some extent, they are primarily used for analyzing short texts. For long texts with hierarchical or nested structures, existing technologies struggle to effectively restore the semantic logic. Furthermore, existing technologies generally lack the ability to dynamically manage document hierarchies. As a result, in complex document processing scenarios, the output structured results often exhibit logical disorganization or lack intuitiveness. Furthermore, these models are highly dependent on data quality, and changes in data distribution can affect their applicability.

[0005] Furthermore, when it comes to document structuring across diverse domains, existing rule-based or statistical solutions have limited adaptability to new scenarios and requirements. Rule-matching methods require manual adjustments to the rule base, while machine learning-based models require retraining, which is time-consuming, labor-intensive, and costly. Existing technologies are particularly difficult to meet when dealing with dynamically changing document formats or when rapidly expanding into new domains.

[0006] During the document structuring process, existing solutions often fail to effectively filter out redundant information, resulting in lengthy and unrefined output. This redundancy also creates challenges when extracting information from structured data. This is because existing technologies fail to fully integrate semantic analysis with redundancy identification techniques, lacking targeted redundancy removal measures for redundant or repetitive content. This problem is particularly prominent in long documents or complex reports. Furthermore, text scanned with OCR often contains numerous typos, missing characters, and formatting issues, making it difficult to achieve both semantic meaning and grammatical and sentence-correcting errors using traditional word segmentation methods. Summary of the Invention

[0007] In response to the problems existing in the prior art, the present invention provides a method and system for automatic document structuring based on LLM, which can realize semantic analysis and extraction of complex and disordered documents, dynamically track document hierarchical relationships, effectively process multi-layer nested content, effectively identify and remove redundant information, and is suitable for multi-field and multi-type documents.

[0008] The technical solution of the present invention is achieved as follows:

[0009] A method for automatically processing document structure based on LLM includes the following steps:

[0010] S1. Document input and preprocessing: The document includes disordered text; the input document format includes DOCX, TXT, etc. The preprocessing includes text standardization and semantic repair. The text standardization includes removing special characters and unifying the encoding format in the disordered text. The semantic repair adjusts the semantic density of the disordered text and expands or deletes paragraphs of the disordered text to ensure that the keywords and semantics of the disordered text can be correctly identified in subsequent steps. Specifically, semantic repair is aimed at deleting meaningless, incoherent, erroneous characters or words, or modal particles.

[0011] S2. Redundancy removal and keyword extraction: Remove duplicate information from the disordered text; analyze the disordered text and extract keywords from the disordered text, which are recorded as first keywords. That is, natural language processing technology is used to identify important nouns, verbs, and proper nouns in the disordered text; this is used to subsequently assist the large model in understanding the text and performing semantic extraction, ultimately maintaining the accuracy and compactness of the information;

[0012] S3. Semantic Extraction and Structuring: Using a pre-trained large language model (LLM, Large Language Models), such as GPT3 or BERT, the unordered text is parsed sentence by sentence based on the first keyword, and the main meaning and related semantic information corresponding to each sentence in the text are extracted to obtain an extraction result. The extraction result includes several keywords from the same sentence, which are recorded as second keywords, and sentence meaning. The sentence meaning is equivalent to the understanding of the sentence by the large model.

[0013] S4. Semantic stack construction and update: constructing and updating a semantic stack based on the extraction results; the data stored in the semantic stack are semantic units; one semantic unit corresponds to the second keyword and the sentence meaning contained in the same sentence; different semantic units have a hierarchical relationship of parent-child or peer, or no hierarchical relationship;

[0014] The semantic stack is used to maintain the information hierarchy of the document and ensure the logic and coherence of the document structure;

[0015] In semantic analysis, a semantic unit is the smallest fragment of semantic information and the smallest part of a sentence that can independently express meaning. These parts are semantically complete and do not change due to subsequent changes in the text. For example, in the sentence "I went to the park at 10 am," four semantic units can be extracted: "10 am," "I," "went," and "park." Each semantic unit is an independent piece of semantic information, and its content does not change with changes in the following text.

[0016] S5. Structured output: Output the data in the semantic stack in a preset format, such as XML, JSON, etc. Furthermore, output in the form of a mind map, etc., which can well demonstrate the hierarchical relationship.

[0017] In each of the above steps, document preprocessing, redundancy removal, keyword extraction, semantic extraction, and structuring can all involve the Large Language Model (LLM) or other customized programs. For example, when performing semantic adjustments, the large model initially checks the text based on user requirements to determine if adjustments are needed, outputting a True or False response. Later, it can independently identify the results, resulting in a user-defined + model-checked process, similar to the Chain of Thought (CoT) approach.

[0018] The deep semantic analysis capabilities of the Large Language Model (LLM) enable comprehensive understanding of complex document content and logical relationship modeling. By tracking and adjusting document hierarchical relationships in real time, it effectively handles nested and complex document content, ensuring that the structured output is clearly structured and logically coherent. This is particularly suitable for multi-level reports or legal documents.

[0019] As a further optimization of the above solution, in step S4, before updating the semantic stack, a hierarchical judgment operation is further included; the hierarchical judgment operation includes calculating the similarity r between the two semantic units using semantic embedding:

[0020]

[0021] Wherein, a is the semantic unit stored in the semantic stack, and b corresponds to the new semantic unit; S a and S b The embedding vectors corresponding to a and b respectively;

[0022] If the similarity r is within a preset threshold range, b is a child of a.

[0023] Each semantic unit includes several second keywords. Using a pre-trained model (such as BERT or GPT), the second keywords or sentences / sentence meanings can be converted into digital coding representations, namely embedding vectors.

[0024] For example, in m3e (abbreviation for Moka Massive Mixed Embedding, which is an open-source Chinese text embedding model), when the output of a sentence passes through a trained embedding model (The Embedding Model is a machine learning model widely used in fields such as natural language processing (NLP) and computer vision (CV). Its working principle is to transform high-dimensional data into a low-dimensional embedding space and retain the features and semantic information of the original data, thereby improving the efficiency and accuracy of the model.), the text will be converted into a vector.

[0025] As a further optimization of the above solution, in step S4, before updating the semantic stack, a hierarchical judgment operation is further included; the hierarchical judgment operation includes calculating the information entropy of the semantic unit:

[0026]

[0027] Among them, H(X) represents the information entropy of the semantic unit; X represents the semantic unit; N represents the number of second keywords; x i is the i-th second keyword; p(x i ) represents the probability that the component of X includes x i ; that is, in a sentence, the probability value of a certain word appearing at a specific position;

[0028] For a representing the semantic units stored in the semantic stack and b representing the new semantic unit;

[0029] If H(b) < H(a), then b is a sub-level of a.

[0030] p(x i ) can be calculated by the BERT model or a custom algorithm.

[0031] As a further optimization of the above solution, the calculation of the information entropy is completed by the BERT model, and the calculation process is as follows:

[0032]

[0033] Among them, |X| represents the number of second keywords of a semantic unit;

[0034] V represents a preset vocabulary; |V| represents the number of words in the vocabulary; v j represents the j-th word in the vocabulary; p(x i = v j |X) represents the probability that x i and v j are the same.

[0035] p(x i =v j |X) can also be replaced by the results calculated by other models or other custom algorithms.

[0036] Due to the differences between English and Chinese in natural language processing, that is, Chinese is not sorted by word structure like English, the model needs to segment Chinese words when processing Chinese. The word segmentation table used as a reference is the vocabulary, that is, the built-in vocabulary of the large model is the above-mentioned vocabulary.

[0037] As a further optimization of the above solution, in step S4, before updating the semantic stack, a hierarchical judgment operation is also included; the hierarchical judgment operation includes constructing a syntax tree and determining the hierarchical relationship between the two semantic units in combination with syntactic analysis.

[0038] For example, if the predefined syntax tree is: chapter-section-item, the text can be stacked in the order of chapters and items.

[0039] As a further optimization of the above solution, the data structure of the semantic stack adopts a last-in-first-out stack; the operations of the semantic stack include push, pop, replace and traverse;

[0040] The stacking is to push the new semantic unit into the top of the semantic stack;

[0041] The pop stack is to remove and return the semantic unit at the top of the stack;

[0042] The replacement is to replace the semantic unit at the top of the stack with the new semantic unit;

[0043] The traversal is to traverse from the top of the stack downwards to search for the specified semantic unit;

[0044] The triggering of the operation of the semantic stack includes:

[0045] A. Same-level replacement: If the new semantic unit current is at the same level as the semantic unit top on the stack, the replacement operation is performed. current =L top , then Replace(top,current); where L represents the level of the semantic unit;

[0046] B. Pushing the lower level: If current is a child of top, then the push operation is performed; that is, If L current =L top +1, then Push(current);

[0047] C. Pop stack n levels higher: If the current level is n levels higher than the top level, perform the pop operation n times and then perform the replace operation; that is, If L current =L top -n, then n times Pop(); Pop(); ... Pop() and Replace (new top, current); where new top represents the semantic unit at the top of the stack after popping the stack n times; the lower the level, the closer it is to the top of the stack, and the higher the level, the farther away from the top of the stack.

[0048] Push(), Pop() and Replace() represent the push operation, the pop operation and the replace operation respectively.

[0049] As a further optimization of the above solution, dynamic management of the semantic stack is also included; the dynamic management includes depth control and width adjustment;

[0050] The attributes of the semantic stack include the depth and width of the stack, wherein the maximum stack depth is Dmax and the maximum stack width is Wmax;

[0051] The depth control is that if the level of the new semantic unit is greater than Dmax, the information entropy maximum planning strategy is triggered, that is, the semantic unit at the top of the semantic stack is popped out or replaced;

[0052] The width adjustment is to merge similar semantic units by a clustering method or a classification method so that the number of semantic units in the same layer is less than or equal to Wmax.

[0053] The present invention also provides an LLM-based document structuring automatic processing system, which applies the above-mentioned LLM-based document structuring automatic processing method; including an interactive interface module, a semantic repair module, a redundancy removal and reorganization module, a semantic extraction module, and a semantic stack management module;

[0054] The redundancy removal and reorganization module also has submodules, including a keyword system module;

[0055] The user uploads the document, inputs processing instructions and receives output results through the interactive interface module;

[0056] The semantic repair module is used to perform the semantic repair operation in step S1;

[0057] The redundancy removal and reorganization module and its submodules are used to perform step S2;

[0058] The semantic extraction module is used to execute step S3. Furthermore, the semantic extraction module's settings include adjusting the model's sensitivity. Specifically, this adjustment is accomplished by combining different prompt words with user instructions. For example, if a user wishes to focus on words related to theft, the model will pay particular attention to these words when extracting text.

[0059] The semantic stack management module is used to execute step S4;

[0060] The interactive interface module, the semantic repair module, the redundancy removal and reorganization module, the semantic extraction module and the semantic stack management module are connected in sequence; the semantic stack management module is also connected to the interactive interface module.

[0061] As a further optimization of the above solution, a keyword table is further included; in step S2, the generated first keyword is stored in the keyword table;

[0062] In step S3, the semantic extraction module loads the first keyword from the keyword table.

[0063] As a further optimization of the above solution, it also includes a keyword system module; the keyword system module includes a keyword table and a management component; the data stored in the keyword table also includes custom keywords; the function of the management component includes automatically generating a new first keyword based on the custom keyword.

[0064] Compared with the prior art, the present invention achieves the following beneficial effects:

[0065] (1) Deep semantic understanding and precise structuring: Through the collaborative work of the semantic induction module and the semantic extraction module, the deep semantic analysis capability of the large language model (LLM) is utilized to achieve a comprehensive understanding of the complex document content and logical relationship modeling, and output structured results with clear semantics and rigorous logic.

[0066] (2) Dynamic hierarchical management: The semantic stack management module can track and adjust the hierarchical relationship of documents in real time, effectively handle the nested and complex document content, and ensure that the structured output is clearly layered and logically clear. It is particularly suitable for multi-level reports or legal documents.

[0067] (3) Redundant information filtering: Through the redundancy removal processing module, combined with semantic analysis technology, redundant information in the document can be accurately identified and filtered. The generated structured documents are more concise and efficient, improving the information quality and practicality.

[0068] (4) Flexible expansion capability: The present invention adopts a modular design and combines the generalization capability of LLM. It can adapt to different fields and new types of documents without redefining rules or training models, which significantly enhances the flexibility of the system.

[0069] (5) Efficient computing and large-scale processing: Optimizing data flow and module collaboration, combined with the parallel processing capabilities of LLM, enables rapid structured processing of large-scale documents while maintaining high quality and accuracy.

[0070] (6) High degree of automation: Through full-process automation design, the need for manual intervention is reduced, operating costs are lowered, and the efficiency and effectiveness of document processing are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a system structure diagram of an LLM-based automatic document structure processing system provided in Example 1 of the present invention;

[0072] Figure 2 1 is a flowchart of an automatic document structuring processing method based on LLM provided in Example 1 of the present invention;

[0073] Figure 3 This is a schematic diagram of the semantic stack before updating provided by Example 1 of the present invention;

[0074] Figure 4 This is a schematic diagram of the updated semantic stack provided by Example 1 of the present invention. DETAILED DESCRIPTION

[0075] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0076] Example 1

[0077] like Figure 1 As shown, this embodiment provides a document structured automatic processing system based on LLM, including an interactive interface module, a semantic repair module, a redundancy removal and reorganization module, a semantic extraction module, and a semantic stack management module;

[0078] The redundancy removal and reorganization module also has submodules, including a keyword system module;

[0079] Users upload documents, input processing instructions, and receive output results through the interactive interface module;

[0080] The interactive interface module, the semantic repair module, the redundancy removal and reorganization module, the semantic extraction module and the semantic stack management module are connected in sequence; the semantic stack management module is also connected to the interactive interface module.

[0081] It also includes a keyword system module; the keyword system module includes a keyword table and a management component.

[0082] like Figure 2 As shown, this embodiment provides a method for automatically processing document structure based on LLM, including the following steps:

[0083] S1. Document Input and Preprocessing: Documents include unordered text in formats such as DOCX or TXT. Preprocessing includes text normalization and semantic repair through the semantic repair module. Text normalization involves removing special characters and unifying the encoding format in unordered text. Semantic repair involves adjusting the semantic density of the unordered text, expanding or deleting paragraphs, and ensuring that keywords and semantics are correctly identified in subsequent steps. Specifically, semantic repair removes meaningless, incoherent, or incorrect characters, words, or modal particles.

[0084] S2. De-redundancy and keyword extraction: Remove duplicate information from disordered text through redundancy removal and reorganization modules; analyze disordered text, extract keywords from disordered text, and record them as first keywords, that is, use natural language processing technology to identify important nouns, verbs and proper nouns in disordered text; use it to assist large models in understanding text and perform semantic extraction, ultimately maintaining the accuracy and compactness of information; the data stored in the keyword table also includes custom keywords; the functions of the management component include automatically generating new first keywords based on custom keywords.

[0085] S3, semantic extraction and structuring: Using pre-trained large language models, also known as LLMs, Large Language Models, specific models such as GPT3 or BERT; the semantic extraction module loads the first keyword from the keyword table. Based on the first keyword, the semantic extraction module parses the disordered text sentence by sentence or paragraph by paragraph, and extracts the main meaning and related semantic information corresponding to each sentence in the text to obtain the extraction result; the extraction result includes several keywords in the same sentence, recorded as the second keyword, and also includes the sentence meaning; the sentence meaning is equivalent to the understanding result of the large model on the sentence;

[0086] S4. Semantic stack construction and update: In this embodiment, the semantic stack data structure adopts a last-in, first-out (LIFO) stack. The semantic stack management module constructs and updates the semantic stack based on the extraction results. The data stored in the semantic stack are semantic units. A semantic unit corresponds to the second keyword and sentence meaning contained in the same sentence. Different semantic units have a hierarchical relationship of parent-child or peer, or no hierarchical relationship.

[0087] In this embodiment, before updating the semantic stack, a hierarchical judgment operation needs to be performed. The hierarchical judgment operation includes calculating the similarity r between two semantic units using semantic embedding:

[0088]

[0089] Among them, a is the semantic unit stored in the semantic stack, b corresponds to the new semantic unit; S a and S b The embedding vectors corresponding to a and b respectively;

[0090] If the similarity r is within a preset threshold range, b is a child of a.

[0091] Each semantic unit includes several second keywords. Using a pre-trained model (such as BERT or GPT), the second keywords or sentences / sentence meanings can be converted into digital coding representations, namely embedding vectors.

[0092] For example, in m3e (short for Moka Massive Mixed Embedding, an open source Chinese text embedding model), the sentence is output to the trained embedding model (Embedding Model is a machine learning model widely used in fields such as natural language processing (NLP) and computer vision (CV). It works by converting high-dimensional data into a low-dimensional embedding space while retaining the features and semantic information of the original data, thereby improving the efficiency and accuracy of the model.), which converts the text into a vector.

[0093] The semantic stack is used to maintain the information hierarchy of the document and ensure the logic and coherence of the document structure;

[0094] In semantic analysis, a semantic unit is the smallest fragment of semantic information and the smallest part of a sentence that can independently express meaning. These parts are semantically complete and do not change due to subsequent changes in the text. For example, in the sentence "I went to the park at 10 am," four semantic units can be extracted: "10 am," "I," "went," and "park." Each semantic unit is an independent piece of semantic information, and its content does not change with changes in the following text.

[0095] The operations of the semantic stack include push, pop, replace and traverse;

[0096] Pushing is to push the new semantic unit to the top of the semantic stack;

[0097] Popping the stack is to remove and return the semantic unit at the top of the stack;

[0098] Replacement is to replace the semantic unit at the top of the stack with a new semantic unit;

[0099] Traversal is to traverse from the top of the stack downwards to find the specified semantic unit;

[0100] The triggering of semantic stack operations includes:

[0101] A. Same-level replacement: If the new semantic unit current is at the same level as the semantic unit top on the stack, the replacement operation is performed, i.e., If L current =L top , then Replace(top,current); where L represents the level of the semantic unit;

[0102] B. Push down: If current is the child of top, then push operation is performed; that is, If L current =L top +1, then Push(current);

[0103] C. Pop stack n levels higher: If the current level is n levels higher than the top level, perform n pop operations and then perform the replacement operation; that is,

[0104] If L current =L top -n, then n times Pop();Pop();…Pop()and Replace(newtop,current); where new top represents the semantic unit at the top of the stack after popping the stack n times; the lower the level, the closer it is to the top of the stack, and the higher the level, the farther away from the top of the stack.

[0105] Push(), Pop() and Replace() represent stack push operation, stack pop operation and replace operation respectively.

[0106] In this embodiment, dynamic management of the semantic stack is also included; dynamic management includes depth control and width adjustment;

[0107] The properties of the semantic stack include the depth and width of the stack, where the maximum stack depth is Dmax and the maximum stack width is Wmax;

[0108] Depth control is that if the level of the new semantic unit is greater than Dmax, the information entropy maximum planning strategy is triggered, that is, the semantic unit at the top of the semantic stack is popped out or replaced; for example, when Dmax=4, if the level of the new semantic unit is determined to be 5, depth control is triggered.

[0109] Width adjustment involves merging similar semantic units through clustering or classification methods, so that the number of semantic units at the same level is less than or equal to Wmax. For example, when Wmax = 4, if a semantic unit has 4 subordinate semantic units, width adjustment is triggered when a new subordinate semantic unit is associated.

[0110] Update effect of semantic stack Figure 3 、 Figure 4 As shown, it involves the document structuring operation of the "XXXXX Law".

[0111] Figure 3 In the semantic stack, the elements from the bottom to the top are "Title: XXXXX Law", "Part I General Provisions", "Chapter I Tasks, Basic Principles and Scope of Application of XX", "Article 2 [Tasks] Tasks of XXXXX Law...", and the elements contained in the semantic unit that has been popped out are "Article 1 [Purpose of XX] For the purpose of XXX...".

[0112] Figure 4 In the example, the new semantic unit contains the elements "Article 3 XXXX law expressly stipulates that XX behavior is...", then the top element of the stack "Article 2..." is popped out.

[0113] S5. Structured Output: Output the data in the semantic stack to the interactive interface module in a pre-set format, such as XML or JSON. Furthermore, output in the form of a mind map can effectively display hierarchical relationships.

[0114] In each of the above steps, document preprocessing, redundancy removal, keyword extraction, semantic extraction, and structuring can all involve the Large Language Model (LLM) or other customized programs. For example, when performing semantic adjustments, the large model initially checks the text based on user requirements to determine if adjustments are needed, outputting a True or False response. Later, it can independently identify the results, resulting in a user-defined + model-checked process, similar to the Chain of Thought (CoT) approach.

[0115] The deep semantic analysis capabilities of the Large Language Model (LLM) enable comprehensive understanding of complex document content and logical relationship modeling. By tracking and adjusting document hierarchical relationships in real time, it effectively handles nested and complex document content, ensuring that the structured output is clearly structured and logically coherent. This is particularly suitable for multi-level reports or legal documents.

[0116] Example 2

[0117] Features not explained in this embodiment are explained in Example 1 and will not be described in detail here. The difference between this embodiment and Example 1 is that:

[0118] The hierarchical judgment operation includes calculating the information entropy of the semantic unit:

[0119]

[0120] Among them, H(X) represents the information entropy of the semantic unit; X represents the semantic unit; N represents the number of the second keywords; x i is the i-th second keyword; p(x i ) represents the probability that the component of X includes x i ; that is, in a sentence, the probability value of a certain word appearing at a specific position;

[0121] For a representing the semantic units stored in the semantic stack and b representing the new semantic unit;

[0122] If H(b) < H(a), then b is a child level of a.

[0123] p(x i ) can be calculated by the BERT model or a custom algorithm.

[0124] In this embodiment, the calculation of the information entropy is completed by the BERT model, and the calculation process is as follows:

[0125]

[0126] Among them, |X| represents the number of the second keywords of a semantic unit;

[0127] V represents a preset vocabulary; |V| represents the number of words in the vocabulary; v j represents the j-th word in the vocabulary; p(x i = v j |X) represents the probability that x i and v j are the same.

[0128] p(x i = v j |X) can also be replaced by the results calculated by other models or other custom algorithms.

[0129] Due to the differences between English and Chinese in natural language processing, that is, Chinese is not sorted in the form of words like English, so when the model processes Chinese, Chinese needs to be segmented, and the segmentation table referred to is the word table, that is, the built-in word table of the large model is the above-mentioned vocabulary.

[0130] Embodiment 3

[0131] For the features not explained in this embodiment, the explanations in Embodiment 1 are adopted and will not be elaborated here. The differences between this embodiment and Embodiment 1 are as follows:

[0132] In this embodiment, the hierarchical determination operation includes constructing a syntax tree and determining the hierarchical relationship between two semantic units by combining syntactic analysis.

[0133] For example, if the predefined syntax tree is: chapter-section-item, the text can be stacked in the order of chapters and items.

[0134] Based on the disclosure and teachings of the above description, those skilled in the art may also make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and modifications and variations of the present invention should also fall within the scope of protection of the claims of the present invention. In addition, although certain specific terms are used in this description, these terms are only for convenience of description and do not constitute any limitation to the present invention.

Claims

1. A document structuring automatic processing method based on LLM, characterized in that: It includes the following steps: S1. Document input and preprocessing: The document includes unordered text; the preprocessing includes text normalization and semantic repair; the text normalization includes removing special characters from the unordered text and unifying the encoding format; the semantic repair is to adjust the semantic density of the unordered text, and expand or delete paragraphs of the unordered text; S2. Redundancy removal and keyword extraction: Remove duplicate information from the unordered text; analyze the unordered text, extract keywords of the unordered text, denoted as the first keywords; S3. Semantic extraction and structuring: Use a pre-trained large language model, according to the first keywords, parse the unordered text sentence by sentence, and extract the main meaning and relevant semantic information corresponding to each sentence in the text, to obtain an extraction result; the extraction result includes several keywords of the same sentence, denoted as the second keywords, and also includes the sentence meaning; S4. Semantic stack construction and update: Construct and update a semantic stack according to the extraction result; the data stored in the semantic stack is semantic units; one semantic unit corresponds to the second keywords and the sentence meaning included in the same sentence; between different semantic units, there is a parent-child or sibling hierarchical relationship, or no hierarchical relationship; Before updating the semantic stack, it also includes a hierarchical judgment operation; The hierarchical judgment operation includes calculating the information entropy of the semantic unit: Wherein, H(X) represents the information entropy of the semantic unit; X represents the semantic unit; N represents the number of the second keywords; x i is the second keyword of the ith one; p(x i ) indicates that the components of X include x i probability; The calculation of the information entropy is completed by a BERT model, and the calculation process is: where, |X| represents the number of the second keywords of one semantic unit; V represents a preset vocabulary; |V| represents the number of words in the vocabulary; v j represents the jth word in the vocabulary; p(x i =v j |X) represents x i and v j Same probability; For a representing the semantic units already stored in the semantic stack, and b representing a new semantic unit; If H(b) < H(a), then b is a child of a; S5. Structured output: Output the data in the semantic stack in a preset format.

2. The method for automatically processing document structure based on LLM according to claim 1, characterized in that: The hierarchical judgment operation includes calculating the similarity r between two semantic units by using semantic embedding: Wherein, a is the semantic unit stored in the semantic stack, and b corresponds to the new semantic unit; S a and S b The embedding vectors corresponding to a and b respectively; If the similarity r is within a preset threshold range, then b is a child of a.

3. The method for automatically processing document structure based on LLM according to claim 1, characterized in that: The hierarchical judgment operation includes constructing a syntax tree and determining the hierarchical relationship between two semantic units by combining syntactic analysis.

4. The method for automatically processing document structure based on LLM according to claim 1, characterized in that: The data structure of the semantic stack adopts a last-in-first-out stack; the operations of the semantic stack include pushing, popping, replacing, and traversing; The push operation is to push a new semantic unit onto the top of the semantic stack; The pop operation is to remove and return the semantic unit at the top of the stack; The replace operation is to replace the semantic unit at the top of the stack with a new semantic unit; The traversal is to traverse downward from the top of the stack to find a specified semantic unit; The triggers for the operations of the semantic stack include: A. Same-level replacement: If the new semantic unit current is at the same level as the semantic unit top on the stack, the replacement operation is performed. current =L top , then Replace(top,current); where L represents the level of the semantic unit; B. Pushing the lower level: If current is a child of top, then the push operation is performed; that is, If L current =L top +1,thenPush(current); C. High-n-level pop: If the level of current is n levels higher than the level of top, then perform the pop operation n times, and then perform the replace operation; that is If L current =L top -n, then n times Pop();Pop();…Pop()and Replace(new top,current); where new top represents the semantic unit at the top of the stack after popping the stack n times; Push(), Pop(), and Replace() respectively represent the push operation, the pop operation, and the replace operation.

5. The method for automatically processing document structure based on LLM according to claim 1, characterized in that: It also includes dynamic management of the semantic stack; the dynamic management includes depth control and width adjustment; The attributes of the semantic stack include the depth and width of the stack, where the maximum stack depth is Dmax and the maximum stack width is Wmax; The depth control is that if the level of the new semantic unit is greater than Dmax, the information entropy maximum planning strategy is triggered, that is, the semantic unit at the top of the semantic stack is popped out or replaced; The width adjustment is to merge similar semantic units by a clustering method or a classification method so that the number of semantic units in the same layer is less than or equal to Wmax.

6. A document structuring automatic processing system based on LLM, applying a document structuring automatic processing method based on LLM according to any one of claims 1 to 5; characterized in that: It includes interactive interface module, semantic repair module, redundancy removal and reorganization module, semantic extraction module, and semantic stack management module; The redundancy removal and reorganization module also has submodules, including a keyword system module; The user uploads the document, inputs processing instructions and receives output results through the interactive interface module; The semantic repair module is used to perform the semantic repair operation in step S1; The redundancy removal and reorganization module and its submodules are used to perform step S2; The semantic extraction module is used to execute step S3; The semantic stack management module is used to execute step S4; The interactive interface module, the semantic repair module, the redundancy removal and reorganization module, the semantic extraction module and the semantic stack management module are connected in sequence; the semantic stack management module is also connected to the interactive interface module.

7. The document structuring automatic processing system based on LLM according to claim 6 is characterized in that: Also included is a keyword table; in step S2, the generated first keyword is stored in the keyword table; In step S3, the semantic extraction module loads the first keyword from the keyword table.

8. The document structuring automatic processing system based on LLM according to claim 7 is characterized in that: Also included is a keyword system module; the keyword system module includes a keyword table and management components; The data stored in the keyword table also includes user-defined keywords; the function of the management component includes automatically generating a new first keyword based on the user-defined keywords.

Images