LLM-based document structuring automatic processing method and system

Abstract

The invention discloses an LLM-based document structuring automatic processing method and system, and relates to the technical field of document processing, and the method comprises the following steps: S1, document input and preprocessing: removing special characters in a disordered text, unifying a coding format, and adjusting the semantic density; s2, redundancy removal and keyword extraction: removing repeated information in the disordered text, and extracting a first keyword; s3, semantic extraction and structuring: parsing the disordered text sentence by sentence according to the first keyword, and extracting main meaning and related semantic information corresponding to each sentence to obtain an extraction result which comprises a second keyword and sentence meaning of the same sentence; s4, constructing and updating a semantic stack: constructing and updating the semantic stack according to an extraction result, and storing a semantic unit; one semantic unit corresponds to the same extraction result; and S5, structured output: outputting the data in the semantic stack in a preset format. The method is suitable for structured processing of complex disordered documents in multiple fields and multiple types.

Description

Technical Field

[0001] The present invention relates to the technical field of document processing, and in particular to a method and system for automatic document structure processing based on LLM. Background Art

[0002] Document structuring refers to organizing and arranging document content according to a certain logic and hierarchy to facilitate information retrieval, understanding and processing. Structured documents usually have a clear format and organization, which makes it easy for computers to understand and process, can effectively support automated processes and data analysis, and also make information more orderly, searchable and easy to analyze. Document structuring can also enable readers to quickly find the information they need, making it easier for users to read intuitively.

[0003] Document structuring involves the processing of text. Traditional text processing methods usually analyze and organize documents by artificially constructing document structure rules and using rule matching technology. For example, in certain documents in specific fields, specific paragraph contents are identified by predefined keywords or fixed formats to determine the logical structure of the document. For example, a method and device for storing unstructured data in a database, such as the patent document with announcement number CN117093589B, requires the use of several specific storage rules to match unstructured data / documents. However, such methods are highly dependent on rules and have poor scalability. They are difficult to cope with scenarios with complex document structures or diverse contents, especially in disordered, non-standardized or cross-domain document processing, where rule matching methods are prone to failure. For example, when there are implicit relationships or non-explicit structural information in a document, rule matching cannot accurately capture them.

[0004] Basic natural language processing technology models a large amount of text data through machine learning to achieve document structuring and semantic understanding. For example, by training a bag-of-words model or performing semantic analysis based on statistical methods, document analysis results with certain semantic understanding capabilities are generated. On this basis, deep learning models such as BERT (Bidirectional Encoder Representation from Transformers, a pre-trained model proposed by Google AI Research Institute that specializes in reading comprehension) are widely used in the semantic understanding and extraction of texts. By modeling contextual relationships, they can generate coherent summary content or extract key information. Although such methods have improved the effect of document processing to a certain extent, they are mainly used for the analysis of short texts. For long texts with hierarchical relationships or nested structures, it is difficult for existing technologies to effectively restore their semantic logic. In addition, the existing technologies generally lack the ability to dynamically manage document hierarchical relationships, resulting in the output structured results often having logical disorder or lack of intuitiveness in complex document processing scenarios. On the other hand, such models are highly dependent on data quality, and when the data distribution changes, the applicability of the model may be affected.

[0005] In addition, when it comes to document structuring of texts in different fields, existing rule-based or statistical technical solutions have poor adaptability to new scenarios or new requirements. The rule matching method requires manual adjustment of the rule base, while the machine learning-based model needs to be retrained, which is time-consuming, labor-intensive and costly. In particular, when it is necessary to process dynamically changing document formats or to quickly expand to new fields, existing technologies are difficult to meet actual needs.

[0006] In the process of document structuring, existing solutions often fail to effectively filter redundant information, resulting in lengthy and unrefined output results. When extracting information from structured data, there will also be certain problems due to information redundancy. This is because existing technologies fail to fully combine semantic analysis with redundancy identification technology, and lack targeted redundancy removal measures for redundant or repeated content, especially in long texts or complex report documents. At the same time, texts scanned by OCR often have a large number of typos, missing words, and typesetting problems. It is difficult to achieve the purpose of both obtaining semantics and correcting grammatical / sentence errors through traditional word segmentation methods. Summary of the invention

[0007] In response to the problems existing in the prior art, the present invention provides a method and system for automatic document structuring based on LLM, which can realize semantic analysis and extraction of complex and disordered documents, dynamically track document hierarchical relationships, effectively process multi-layer nested content, effectively identify and remove redundant information, and is suitable for multi-field and multi-type documents.

[0008] The technical solution of the present invention is achieved in this way:

[0009] A method for automatically processing document structure based on LLM includes the following steps:

[0010] S1. Document input and preprocessing: The document includes disordered text; the input document formats include DOCX, TXT, etc. The preprocessing includes text standardization and semantic repair; the text standardization includes removing special characters and unified encoding formats in the disordered text; the semantic repair is to adjust the semantic density of the disordered text, expand or delete the paragraphs of the disordered text, and ensure that the keywords and semantics of the disordered text can be correctly identified in the subsequent steps; specifically, semantic repair is to delete meaningless, incoherent, wrong characters or words, or modal particles, etc.

[0011] S2, redundancy removal and keyword extraction: remove repeated information in the disordered text; analyze the disordered text, extract the keywords of the disordered text, record them as the first keywords, that is, use natural language processing technology to identify important nouns, verbs and proper nouns in the disordered text; use them to assist the large model in understanding the text and perform semantic extraction, and ultimately maintain the accuracy and compactness of the information;

[0012] S3, semantic extraction and structuring: using a pre-trained large language model, i.e., LLM, LargeLanguageModels, such as GPT3 or BERT; according to the first keyword, parse the disordered text sentence by sentence, and extract the main meaning and related semantic information corresponding to each sentence in the text to obtain an extraction result; the extraction result includes several keywords of the same sentence, recorded as the second keyword, and also includes sentence meaning; the sentence meaning is equivalent to the understanding result of the large model on the sentence;

[0013] S4, semantic stack construction and update: construct and update the semantic stack according to the extraction result; the data stored in the semantic stack is a semantic unit; one semantic unit corresponds to the second keyword and the sentence meaning contained in the same sentence; different semantic units are in a hierarchical relationship of parent and child or the same level, or have no hierarchical relationship;

[0014] The semantic stack is used to maintain the information hierarchy of the document and ensure the logic and coherence of the document structure;

[0015] Semantic units are the smallest fragments that constitute semantic information in semantic analysis, and are also the smallest parts of a sentence that can express meaning independently. These parts are semantically complete and will not change due to changes in subsequent texts. For example, in the sentence "I went to the park at 10 a.m.", four semantic units can be extracted: "10 a.m.", "I", "went", and "park". Each semantic unit is an independent fragment of semantic information, and its content will not change with changes in the following text.

[0016] S5. Structured output: output the data in the semantic stack in a preset format, such as XML, JSON, etc. Furthermore, output in the form of a mind map, etc., which can well display the hierarchical relationship.

[0017] In the above steps, document preprocessing, redundancy removal and keyword extraction, semantic extraction and structuring can all involve the execution of large language models (LLM), or other customized programs. For example, when adjusting semantics, the large model checks whether the text needs to be adjusted according to user requirements in the early stage, and outputs True or False. In the later stage, it can independently identify and form user customization + model detection, which is similar to the result of chain of thought (CoT).

[0018] The deep semantic analysis capability of the Large Language Model (LLM) enables comprehensive understanding of complex document content and logical relationship modeling. By real-time tracking and adjustment of document hierarchical relationships, it effectively handles nested and complex document content, ensuring that the structured output is well-organized and logically clear. This is particularly suitable for multi-level reports or legal documents.

[0019] As a further optimization of the above solution, in step S4, before updating the semantic stack, a hierarchical judgment operation is also included; the hierarchical judgment operation includes calculating the similarity r between the two semantic units using semantic embedding:

[0020]

[0021] Wherein, a is the semantic unit stored in the semantic stack, and b corresponds to the new semantic unit; S a and S b The embedding vectors corresponding to a and b respectively;

[0022] If the similarity r is within a preset threshold range, b is a child of a.

[0023] Each semantic unit includes several second keywords. Using a pre-trained model (such as BERT or GPT), the second keyword or sentence / sentence meaning can be converted into a digital coding representation, which is an embedding vector.

[0024] For example, in m3e (abbreviation for Moka Massive Mixed Embedding, which is an open-source Chinese text embedding model), when the output of a sentence passes through a trained embedding model (the Embedding Model is a machine learning model widely used in fields such as natural language processing (NLP) and computer vision (CV). Its working principle is to transform high-dimensional data into a low-dimensional embedding space and retain the features and semantic information of the original data, thereby improving the efficiency and accuracy of the model), the text will be transformed into a vector.

[0025] As a further optimization of the above solution, in step S4, before updating the semantic stack, a hierarchical judgment operation is also included; the hierarchical judgment operation includes calculating the information entropy of the semantic unit:

[0026]

[0027] Among them, H(X) represents the information entropy of the semantic unit; X represents the semantic unit; x i is the i-th second keyword; p(x i ) represents the probability that the component of X includes x i ; that is, in a sentence, the probability value of a certain word appearing at a specific position;

[0028] For a representing the semantic units stored in the semantic stack and b representing the new semantic unit;

[0029] If H(b) < H(a)H(b) < H(a), then b is a sub-level of a.

[0030] p(x i ) can be calculated by the BERT model or a custom algorithm.

[0031] As a further optimization of the above solution, the calculation of the information entropy is completed by the BERT model, and the calculation process is:

[0032]

[0033] Among them, |X| represents the number of second keywords of a semantic unit;

[0034] V represents a preset vocabulary; |V| represents the number of words in the vocabulary; v j represents the j-th word in the vocabulary; p(x i = v j |X) represents the probability that x i is the same as v j .

[0035] p(x i =v j |X) can also be replaced by the results calculated by other models or other custom algorithms.

[0036] Due to the differences between English and Chinese in natural language processing, that is, Chinese is not sorted by word structure like English, the model needs to segment Chinese words when processing Chinese. The word segmentation table referred to is the vocabulary, that is, the built-in vocabulary of the large model is the above-mentioned vocabulary.

[0037] As a further optimization of the above solution, in step S4, before updating the semantic stack, a hierarchical judgment operation is also included; the hierarchical judgment operation includes constructing a syntax tree and determining the hierarchical relationship between the two semantic units in combination with syntactic analysis.

[0038] For example: predefined syntax tree: chapter-section-item, then the text can be stacked in the order of chapters and items.

[0039] As a further optimization of the above solution, the data structure of the semantic stack adopts a last-in-first-out stack; the operations of the semantic stack include push, pop, replace and traverse;

[0040] The stacking is to push the new semantic unit into the top of the semantic stack;

[0041] The pop stack is to remove and return the semantic unit at the top of the stack;

[0042] The replacement is to replace the semantic unit at the top of the stack with the new semantic unit;

[0043] The traversal is to traverse from the top of the stack downwards to search for the specified semantic unit;

[0044] The triggering of the operation of the semantic stack includes:

[0045] A. Same-level replacement: If the new semantic unit current is at the same level as the semantic unit top at the top of the stack, the replacement operation is performed, i.e., If L current =L top , then Replace(top,current); where L represents the level of the semantic unit;

[0046] B. Pushing the lower level: If current is a child of top, the push operation is performed; that is, If L current =L top +1, then Push(current);

[0047] C. Pop the stack n levels higher: If the level of current is n levels higher than the level of top, perform the pop operation n times, and then perform the replace operation; that is,

[0048] If L current =L top -n, then n times Pop(); Pop(); ... Pop() and Replace(newtop, current); where new top represents the semantic unit at the top of the stack after popping the stack n times; the lower the level, the closer it is to the top of the stack, and the higher the level, the farther it is from the top of the stack.

[0049] Push(), Pop() and Replace() represent the push operation, the pop operation and the replace operation respectively.

[0050] As a further optimization of the above solution, dynamic management of the semantic stack is also included; the dynamic management includes depth control and width adjustment;

[0051] The attributes of the semantic stack include the depth and width of the stack, wherein the maximum stack depth is Dmax and the maximum stack width is Wmax;

[0052] The depth control is that if the level of the new semantic unit is greater than Dmax, the information entropy maximum planning strategy is triggered, that is, the semantic unit at the top of the semantic stack is popped out or replaced;

[0053] The width is adjusted by merging similar semantic units through a clustering method or a classification method so that the number of semantic units in the same layer is less than or equal to Wmax.

[0054] The present invention also provides a document structured automatic processing system based on LLM, which applies the above-mentioned document structured automatic processing method based on LLM; it includes an interactive interface module, a semantic repair module, a redundancy removal and reorganization module, a semantic extraction module, and a semantic stack management module;

[0055] The redundancy removal and reorganization module also has submodules, including a keyword system module;

[0056] The user uploads the document, inputs processing instructions and receives output results through the interactive interface module;

[0057] The semantic repair module is used to perform the semantic repair operation in step S1;

[0058] The redundancy removal and reorganization module and its submodules are used to perform step S2;

[0059] The semantic extraction module is used to execute step S3; further, the setting of the semantic extraction module includes adjusting the sensitivity of the model. Specifically, the adjustment is completed by combining different prompt words with user instructions. For example, if the user wants to focus on the crime of theft, the model will pay special attention to the relevant words when extracting the text.

[0060] The semantic stack management module is used to execute step S4;

[0061] The interactive interface module, the semantic repair module, the redundancy removal and reorganization module, the semantic extraction module and the semantic stack management module are connected in sequence; the semantic stack management module is also connected to the interactive interface module.

[0062] As a further optimization of the above scheme, a keyword table is also included; in step S2, the generated first keyword is stored in the keyword table;

[0063] In step S3, the semantic extraction module loads the first keyword from the keyword table.

[0064] As a further optimization of the above solution, it also includes a keyword system module; the keyword system module includes a keyword table and a management component; the data stored in the keyword table also includes custom keywords; the function of the management component includes automatically generating a new first keyword based on the custom keyword.

[0065] Compared with the prior art, the present invention achieves the following beneficial effects:

[0066] (1) Deep semantic understanding and precise structuring: Through the collaborative work of the semantic induction module and the semantic extraction module, the deep semantic analysis capability of the large language model (LLM) is utilized to achieve a comprehensive understanding of complex document content and logical relationship modeling, and output structured results with clear semantics and rigorous logic.

[0067] (2) Dynamic hierarchical management: The semantic stack management module can track and adjust the document hierarchy in real time, effectively handle the nested and complex document content, and ensure that the structured output is clearly structured and logically clear. It is particularly suitable for multi-level reports or legal documents.

[0068] (3) Redundant information filtering: Through the redundancy removal processing module, combined with semantic analysis technology, redundant information in the document can be accurately identified and filtered. The generated structured document is more concise and efficient, improving the information quality and practicality.

[0069] (4) Flexible expansion capability: The present invention adopts a modular design and combines the generalization capability of LLM. It can adapt to different fields and new types of documents without redefining rules or training models, and the flexibility of the system is significantly enhanced.

[0070] (5) Efficient computing and large-scale processing: Optimizing data flow and module collaboration, combined with the parallel processing capabilities of LLM, enables rapid structured processing of large-scale documents while maintaining high quality and accuracy.

[0071] (6) High degree of automation: Through full-process automation design, the need for manual intervention is reduced, operating costs are reduced, and the efficiency and effectiveness of document processing are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 is a system structure diagram of an LLM-based document structured automatic processing system provided in Example 1 of the present invention;

[0073] Figure 2 It is a flowchart of an automatic document structuring processing method based on LLM provided in Example 1 of the present invention;

[0074] Figure 3 is a schematic diagram before the semantic stack is updated provided by Embodiment 1 of the present invention;

[0075] Figure 4 This is a schematic diagram of the updated semantic stack provided by Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] Example 1

[0078] like Figure 1 As shown, this embodiment provides a document structured automatic processing system based on LLM, including an interactive interface module, a semantic repair module, a redundancy removal and reorganization module, a semantic extraction module, and a semantic stack management module;

[0079] The redundancy removal and reorganization module also has submodules, including a keyword system module;

[0080] Users upload documents, input processing instructions, and receive output results through the interactive interface module;

[0081] The interactive interface module, the semantic repair module, the redundancy removal and reorganization module, the semantic extraction module and the semantic stack management module are connected in sequence; the semantic stack management module is also connected to the interactive interface module.

[0082] It also includes a keyword system module; the keyword system module includes a keyword table and a management component.

[0083] like Figure 2 As shown, this embodiment provides a method for automatically processing document structure based on LLM, including the following steps:

[0084] S1. Document input and preprocessing: Documents include disordered texts; the input document format is DOCX or TXT, etc. Preprocessing includes text standardization and semantic repair through the semantic repair module; text standardization includes removing special characters and unifying the encoding format in disordered texts; semantic repair is to adjust the semantic density of disordered texts, expand or delete paragraphs of disordered texts, and ensure that the keywords and semantics of disordered texts can be correctly identified in subsequent steps; specifically, semantic repair is to delete meaningless, incoherent, wrong characters or words, or modal particles, etc.

[0085] S2. Redundancy removal and keyword extraction: Remove duplicate information from disordered text through redundancy removal and reorganization modules; analyze disordered text, extract keywords from disordered text, and record them as first keywords, that is, use natural language processing technology to identify important nouns, verbs and proper nouns in disordered text; use it to assist large models in understanding text for semantic extraction, and ultimately maintain the accuracy and compactness of information; the data stored in the keyword table also includes custom keywords; the functions of the management component include automatically generating new first keywords based on custom keywords.

[0086] S3, semantic extraction and structuring: using pre-trained large language models, namely LLM, LargeLanguageModels, specific models such as GPT3 or BERT, etc.; the semantic extraction module loads the first keyword from the keyword table, and according to the first keyword, the semantic extraction module parses the disordered text sentence by sentence or paragraph by paragraph, and extracts the main meaning and related semantic information corresponding to each sentence in the text to obtain the extraction result; the extraction result includes several keywords of the same sentence, recorded as the second keyword, and also includes the sentence meaning; the sentence meaning is equivalent to the understanding result of the large model on the sentence;

[0087] S4, semantic stack construction and update: In this embodiment, the data structure of the semantic stack adopts a last-in, first-out stack (LIFO); the semantic stack management module constructs and updates the semantic stack according to the extraction results; the data stored in the semantic stack is a semantic unit; a semantic unit corresponds to the second keyword and sentence meaning contained in the same sentence; different semantic units have a hierarchical relationship of parent and child or the same level, or no hierarchical relationship;

[0088] In this embodiment, before updating the semantic stack, a hierarchical judgment operation needs to be performed; the hierarchical judgment operation includes calculating the similarity r between two semantic units using semantic embedding:

[0089]

[0090] Among them, a is the semantic unit stored in the semantic stack, and b corresponds to the new semantic unit; S a and S b The embedding vectors corresponding to a and b respectively;

[0091] If the similarity r is within a preset threshold range, b is a child of a.

[0092] Each semantic unit includes several second keywords. Using a pre-trained model (such as BERT or GPT), the second keyword or sentence / sentence meaning can be converted into a digital coding representation, which is an embedding vector.

[0093] For example, in m3e (short for Moka Massive Mixed Embedding, an open source Chinese text embedding model), the sentence is output to the trained embedding model (Embedding Model is a machine learning model widely used in fields such as natural language processing (NLP) and computer vision (CV). It works by converting high-dimensional data into a low-dimensional embedding space and retaining the features and semantic information of the original data, thereby improving the efficiency and accuracy of the model.), which will convert the text into a vector.

[0094] The semantic stack is used to maintain the information hierarchy of the document and ensure the logic and coherence of the document structure;

[0095] Semantic units are the smallest fragments that constitute semantic information in semantic analysis, and are also the smallest parts of a sentence that can express meaning independently. These parts are semantically complete and will not change due to changes in subsequent texts. For example, in the sentence "I went to the park at 10 a.m.", four semantic units can be extracted: "10 a.m.", "I", "went", and "park". Each semantic unit is an independent fragment of semantic information, and its content will not change with changes in the following text.

[0096] The operations of the semantic stack include push, pop, replace and traverse;

[0097] Pushing is to push the new semantic unit to the top of the semantic stack;

[0098] Popping the stack is to remove and return the semantic unit at the top of the stack;

[0099] Replacement is to replace the semantic unit at the top of the stack with a new semantic unit;

[0100] Traversal is to traverse from the top of the stack downwards to find the specified semantic unit;

[0101] The triggering of semantic stack operations includes:

[0102] A. Same-level replacement: If the new semantic unit current is at the same level as the semantic unit top on the stack, the replacement operation is performed, that is, If L current =L top , then Replace(top,current); where L represents the level of the semantic unit;

[0103] B. Pushing the lower level: If current is the child of top, the push operation is performed; that is, If L current =L top +1, then Push(current);

[0104] C. Pop the stack n levels higher: If the current level is n levels higher than the top level, perform n pop operations and then perform the replace operation; that is,

[0105] If L current =L top -n, then n times Pop(); Pop(); …Pop() and Replace(newtop, current); where new top represents the semantic unit at the top of the stack after popping the stack n times; the lower the level, the closer it is to the top of the stack, and the higher the level, the farther it is from the top of the stack.

[0106] Push(), Pop() and Replace() represent stack push operation, stack pop operation and replace operation respectively.

[0107] In this embodiment, dynamic management of the semantic stack is also included; the dynamic management includes depth control and width adjustment;

[0108] The properties of the semantic stack include the depth and width of the stack, where the maximum stack depth is Dmax and the maximum stack width is Wmax;

[0109] The depth control is that if the level of the new semantic unit is greater than Dmax, the information entropy maximum planning strategy is triggered, that is, the semantic unit at the top of the semantic stack is popped out or replaced; for example, when Dmax=4, if the level of the new semantic unit is determined to be 5, the depth control is triggered.

[0110] Width adjustment is to merge similar semantic units through clustering or classification methods so that the number of semantic units at the same level is less than or equal to Wmax. For example, when Wmax = 4, if the number of subordinate semantic units of a semantic unit is 4, when a new subordinate semantic unit is associated, width adjustment is triggered.

[0111] Update effect of semantic stack Figure 3 , Figure 4 As shown, the document structuring operation involving the "XXXXX Law" is involved.

[0112] Figure 3 In the example, the elements from the bottom to the top of the semantic stack are "Title: XXXXX Law", "Part I General Provisions", "Chapter I Tasks, Basic Principles and Scope of Application of XX", "Article 2 [Tasks] Tasks of XXXXX Law...", and the elements contained in the semantic unit that has been popped out of the stack are "Article 1 [Purpose of XX] For the purpose of XXX...".

[0113] Figure 4 In the example, the new semantic unit contains the elements "Article 3 XXXX law expressly stipulates that XX behavior is...", then the top element of the stack "Article 2..." is popped out.

[0114] S5. Structured output: Output the data in the semantic stack to the interactive interface module in a preset format, such as XML, JSON, etc. Furthermore, output in the form of a mind map can well display the hierarchical relationship.

[0115] In the above steps, document preprocessing, redundancy removal and keyword extraction, semantic extraction and structuring can all involve the execution of large language models (LLM), or other customized programs. For example, when adjusting semantics, the large model checks whether the text needs to be adjusted according to user requirements in the early stage, and outputs True or False. In the later stage, it can independently identify and form user customization + model detection, which is similar to the result of chain of thought (CoT).

[0116] The deep semantic analysis capability of the Large Language Model (LLM) enables comprehensive understanding of complex document content and logical relationship modeling. By real-time tracking and adjustment of document hierarchical relationships, it effectively handles nested and complex document content, ensuring that the structured output is well-organized and logically clear. This is particularly suitable for multi-level reports or legal documents.

[0117] Example 2

[0118] Features not explained in this embodiment are explained in Example 1 and will not be described in detail here. The difference between this embodiment and Example 1 is that:

[0119] The hierarchical judgment operation includes calculating the information entropy of the semantic unit:

[0120]

[0121] Among them, H(X) represents the information entropy of the semantic unit; X represents the semantic unit; x i is the i-th second keyword; p(x i ) represents the probability that the component of X includes x i , that is, the probability value of a certain word appearing at a specific position in a sentence;

[0122] For a representing the semantic units stored in the semantic stack and b representing the new semantic unit;

[0123] If H(b) < H(a)H(b) < H(a), then b is a child level of a.

[0124] p(x i ) can be calculated through the BERT model or a custom algorithm.

[0125] In this embodiment, the calculation of the information entropy is completed by the BERT model, and the calculation process is as follows:

[0126]

[0127] Among them, |X| represents the number of second keywords of a semantic unit;

[0128] V represents a preset vocabulary; |V| represents the number of words in the vocabulary; v j represents the j-th word in the vocabulary; p(x i = v j |X) represents the probability that x i and v j are the same.

[0129] p(x i = v j |X) can also be replaced by the results calculated by other models or other custom algorithms.

[0130] Due to the differences between English and Chinese in natural language processing, that is, Chinese is not sorted in a word structure like English, so when the model processes Chinese, Chinese needs to be segmented, and the segmentation table referred to is the word list, that is, the built-in word list of the large model is the above-mentioned vocabulary.

[0131] Embodiment 3

[0132] For the features not explained in this embodiment, the explanations in Embodiment 1 are adopted and will not be elaborated here. The differences between this embodiment and Embodiment 1 are as follows:

[0133] In this embodiment, the hierarchical determination operation includes constructing a syntax tree and determining the hierarchical relationship between two semantic units by combining syntactic analysis.

[0134] For example: predefined syntax tree: chapter-section-item, then the text can be stacked in the order of chapters and items.

[0135] According to the disclosure and teaching of the above description, those skilled in the art to which the present invention belongs may also make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the scope of protection of the claims of the present invention. In addition, although some specific terms are used in this specification, these terms are only for the convenience of description and do not constitute any limitation to the present invention.

Claims

1. A method for automatic document structuring based on LLM, characterized in that: It includes the following steps: S1. Document input and preprocessing: The document includes unordered text; the preprocessing includes text normalization and semantic repair; the text normalization includes removing special characters in the unordered text and unifying the encoding format; the semantic repair is to adjust the semantic density of the unordered text, and expand or delete paragraphs of the unordered text; S2. Redundancy removal and keyword extraction: Remove duplicate information in the unordered text; analyze the unordered text, and extract keywords of the unordered text, denoted as the first keywords; S3. Semantic extraction and structuring: Use a pre-trained large language model, according to the first keywords, parse the unordered text sentence by sentence, and extract the main meaning and related semantic information corresponding to each sentence in the text to obtain an extraction result; the extraction result includes several keywords of the same sentence, denoted as the second keywords, and also includes the sentence meaning; S4. Semantic stack construction and update: Construct and update a semantic stack according to the extraction result; the data stored in the semantic stack is semantic units; one semantic unit corresponds to the second keywords and the sentence meaning included in the same sentence; there is a parent-child or sibling hierarchical relationship, or no hierarchical relationship between different semantic units; S5. Structured output: Output the data in the semantic stack in a preset format.

2. The method for automatic document structuring based on LLM according to claim 1, characterized in that: In step S4, before updating the semantic stack, a hierarchical judgment operation is further included; the hierarchical judgment operation includes using semantic embedding to calculate the similarity r between two semantic units: Wherein, a is the semantic unit stored in the semantic stack, and b corresponds to the new semantic unit; S a and S b The embedding vectors corresponding to a and b respectively; If the similarity r is within a preset threshold range, then b is a child of a.

3. The method for automatic document structuring based on LLM according to claim 1, characterized in that: In step S4, before updating the semantic stack, a hierarchical judgment operation is further included; the hierarchical judgment operation includes calculating the information entropy of the semantic unit: Where H(X) represents the information entropy of the semantic unit; X represents the semantic unit; x i is the i-th second keyword; p(x i ) means that the components of X include x i probability; For a representing a semantic unit already stored in the semantic stack, and b representing a new semantic unit; If H(b) < H(a), then b is a child of a.

4. The method for automatically processing document structure based on LLM according to claim 3, characterized in that: The calculation of the information entropy is completed by a BERT model, and the calculation process is: where |X| represents the number of the second keywords of a semantic unit; V represents a preset vocabulary; |V| represents the number of words in the vocabulary; v j represents the jth word in the vocabulary; p(x i =v j |X) means x i and v j Same probability.

5. The method for automatic document structuring based on LLM according to claim 1, characterized in that: In step S4, before updating the semantic stack, a hierarchical judgment operation is further included; the hierarchical judgment operation includes constructing a syntax tree and using syntactic analysis to determine the hierarchical relationship between two semantic units.

6. The method for automatic document structuring based on LLM according to claim 1, characterized in that: The data structure of the semantic stack adopts a last-in, first-out stack; the operations of the semantic stack include pushing, popping, replacing, and traversing; The push operation is to push a new semantic unit onto the top of the semantic stack; The pop operation is to remove and return the semantic unit at the top of the stack; The replace operation is to replace the semantic unit at the top of the stack with a new semantic unit; The traversal is to traverse downward from the top of the stack to find a specified semantic unit; The triggers for the operations of the semantic stack include: A. Same-level replacement: If the new semantic unit current is at the same level as the semantic unit top at the top of the stack, the replacement operation is performed, i.e., If L current =L top , then Replace(top,current); where L represents the level of the semantic unit; B. Pushing the lower level: If current is a child of top, the push operation is performed; that is, If L current =L top +1,thenPush(current); C. Popping the stack by n levels higher: If the level of current is n levels higher than the level of top, then perform the pop operation n times, and then perform the replace operation; that is If L current =L top -n, then n times Pop(); Pop(); ... Pop() and Replace(new top, current); where new top represents the semantic unit at the top of the stack after popping the stack n times; Push(), Pop(), and Replace() respectively represent the push operation, the pop operation, and the replace operation.

7. The method for automatic document structuring based on LLM according to claim 1, characterized in that: Also included is dynamic management of the semantic stack; the dynamic management includes depth control and width adjustment; The attributes of the semantic stack include the depth and width of the stack, wherein the maximum stack depth is Dmax and the maximum stack width is Wmax; The depth control is that if the level of the new semantic unit is greater than Dmax, the information entropy maximum planning strategy is triggered, that is, the semantic unit at the top of the semantic stack is popped out or replaced; The width is adjusted by merging similar semantic units through a clustering method or a classification method so that the number of semantic units in the same layer is less than or equal to Wmax.

8. An automatic document structuring processing system based on LLM, applying an automatic document structuring processing method based on LLM as claimed in any one of claims 1 to 7; characterized in that: It includes interactive interface module, semantic repair module, redundancy removal and reorganization module, semantic extraction module, and semantic stack management module; The redundancy removal and reorganization module also has submodules, including a keyword system module; The user uploads the document, inputs processing instructions and receives output results through the interactive interface module; The semantic repair module is used to perform the semantic repair operation in step S1; The redundancy removal and reorganization module and its submodules are used to perform step S2; The semantic extraction module is used to execute step S3; The semantic stack management module is used to execute step S4; The interactive interface module, the semantic repair module, the redundancy removal and reorganization module, the semantic extraction module and the semantic stack management module are connected in sequence; the semantic stack management module is also connected to the interactive interface module.

9. The document structuring automatic processing system based on LLM according to claim 8 is characterized in that: Also includes a keyword table; in step S2, the generated first keyword is stored in the keyword table; In step S3, the semantic extraction module loads the first keyword from the keyword table.

10. The document structuring automatic processing system based on LLM according to claim 9, characterized in that: It also includes a keyword system module; the keyword system module includes a keyword table and a management component; the data stored in the keyword table also includes custom keywords; the function of the management component includes automatically generating a new first keyword based on the custom keyword.

Images