Long document content extraction method and system based on dynamic segmentation and multi-model optimization

Abstract

The invention provides a long document content extraction method and system based on dynamic segmentation and multi-model optimization. The method comprises the following steps: identifying a chapter boundary and a paragraph boundary of a long document of which the content is to be extracted; selecting combinations of undetermined segmentation nodes according to a preset screening rule, and segmenting the long document into different segmented document sets based on the combinations of different undetermined segmentation nodes; calculating the boundary recognition accuracy of different segmented document sets, calculating the semantic unit integrity of the different segmented document sets, and taking the segmented document set which enables the boundary recognition accuracy and the semantic unit integrity to be comprehensively optimal as an optimal segmented document set; and calling a plurality of AI large model interfaces, performing multi-dimensional quantitative evaluation on output of the AI large model called each time for each segmented document, taking the result with the highest evaluation score as a final content extraction result of the current segmented document, and integrating the final content extraction results of all the segmented documents contained in the optimal segmented document set to obtain a long document content extraction result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of long text natural language processing content extraction, and in particular to a long document content extraction method and system based on dynamic segmentation and multi-model optimization. BACKGROUND

[0002] The importance of using AI large models for long text content extraction lies in its significant improvement in long text processing efficiency, knowledge mining depth, and multi-scenario application value. With the acceleration of digitalization, the amount of text data generated by humans is growing exponentially, and traditional manual processing methods have been unable to meet the analysis needs of massive long texts. AI large models, with their powerful semantic understanding and generation capabilities, have become a key technology for breaking through the bottleneck of long text processing.

[0003] Current AI large models (e.g., GPT-4, Claude) have two significant problems when processing long texts: (1) context forgetting: when the input length exceeds the model window (such as 4k tokens), the accuracy of key content extraction decreases by more than 30%; (2) hallucination frequently occurs: the probability of AI large models fabricating content in continuous conversations increases. Experiments show that when the input exceeds 5 interactions, the hallucination rate increases to 18%.

[0004] Currently, methods that can be used for long text processing include segmenting long texts according to sliding window methods (such as Transformer-XL) and processing long texts according to retrieval augmentation methods (such as REALM) combined with external databases. The problems are: (1) the fixed segmentation method used by the sliding window method may damage the coherence of the document structure (such as the rupture of chapter relationships), leading to semantic rupture; (2) the implementation of the retrieval augmentation method relies on additional storage and has poor real-time performance.

[0005] In summary, the existing AI large models for natural language processing have the following problems when processing long texts: (1) the segmentation method may affect the results of long document content extraction; (2) multi-round dialogue cumulative errors cause hallucination. SUMMARY

[0006] In view of this, the embodiments of the present application provide a long document content extraction method based on dynamic segmentation and multi-model optimization to eliminate or improve one or more defects in the prior art.

[0007] One aspect of the present application provides a long document content extraction method based on dynamic segmentation and multi-model optimization, which comprises the following steps: identifying the chapter boundaries and paragraph boundaries of a long document to be content-extracted, calculating the semantic correlation degree of adjacent paragraphs separated by paragraph boundaries; taking the chapter boundaries and paragraph boundaries with a semantic correlation degree lower than a preset threshold as pending segmentation nodes, selecting combinations of the pending segmentation nodes according to a preset screening rule, and segmenting the long document into different segmented document sets based on different combinations of the pending segmentation nodes; wherein, the segmented document set contains multiple segmented documents segmented from the long document; obtaining artificially annotated chapter boundary and paragraph boundary information, calculating the boundary recognition accuracy of different segmented document sets based on the artificial annotation, and calculating the semantic unit integrity of different segmented document sets, and taking the segmented document set with the most optimal comprehensive boundary recognition accuracy and semantic unit integrity as the optimal segmented document set; calling multiple AI large model interfaces for natural language processing, taking the combination of one segmented document contained in the optimal segmented document set and a prompt word template as the input each time, and outputting a content extraction result of the segmented document by each AI large model; for each segmented document, multi-dimensionally quantifying and evaluating the output of each time the AI large model is called, taking the one with the highest evaluation score as the final content extraction result of the current segmented document, and integrating the final content extraction results of all segmented documents contained in the optimal segmented document set to obtain the long document content extraction result.

[0008] In some embodiments of the present application, the step of identifying the chapter boundaries and paragraph boundaries of the long document to be content-extracted is obtained by regular expression matching; and the preset screening rule includes the length range of the segmentation window, and the length candidate value range is 500-8000 token numbers.

[0009] In some embodiments of the present application, the data structure of the segmented document set is stored through a structured metadata table, and each item of the structured metadata table includes the segment ID, starting position, length, chapter level and parent node of a segmented document.

[0010] In some embodiments of the present application, the step of calculating the boundary recognition accuracy of different segmented document sets based on artificial annotation comprises: determining the total number of segments based on the artificially annotated chapter boundaries and paragraph boundaries, comparing the artificially annotated chapter boundaries and paragraph boundaries with the identified chapter boundaries and paragraph boundaries to determine the number of correctly segmented segments, and dividing the number of correctly segmented segments corresponding to the current segmented document set by the total number of segments to obtain the boundary recognition accuracy.

[0011] In some embodiments of the present application, the step of calculating the semantic unit integrity of different segmented document sets comprises: calculating the total number of semantic units of the long text, calculating the number of truncated semantic units when the long text is segmented by different pending segmentation nodes, and calculating the semantic unit integrity based on the formula A semantic unit integrity score C(L) is calculated, wherein L represents a length range of a segmentation window included in the preset screening rule; and a segmentation document set that optimizes the boundary recognition accuracy and the semantic unit integrity is determined by calculating a harmonic mean of the boundary recognition accuracy and the semantic unit integrity score.

[0012] In some embodiments of the present application, the prompt word template includes a role played by the AI large model, an interface of the called AI large model, a segmented document as input, an element that must be strictly included in the output, and an output format requirement.

[0013] In some embodiments of the present application, the multi-dimensional quantitative evaluation of the output of each call of the AI large model includes: based on a preset weight, a plurality of dimensions including accuracy, integrity, illusion rate and robustness are quantitatively evaluated.

[0014] In some embodiments of the present application, the method further comprises: for each segmented document, a preset number of expert independent labeled artificial labeling results conforming to a three-level label system are obtained, and the Kappa coefficient of the artificial labeling result is greater than 0.8, and the output of each call of the AI large model is evaluated based on the artificial labeling result; wherein the three-level label system includes core arguments, supporting arguments and abnormal content.

[0015] Corresponding to the above method, the present application also provides a long document content extraction system based on dynamic segmentation and multi-model optimization, comprising a processor, a memory and a computer program / instructions stored on the memory, the processor being configured to execute the computer program / instructions, when the computer program / instructions are executed, the system implements the steps of the method as described in any one of the above embodiments.

[0016] Corresponding to the above method, the present application also provides a computer readable storage medium having a computer program / instructions stored thereon, the computer program / instructions being executed by a processor to implement the steps of the method as described in any one of the above embodiments.

[0017] The long document content extraction method based on dynamic segmentation and multi-model optimization provided by the present application combines automatic recognition and artificial labeling to determine segmentation nodes for text segmentation, traverses the segmentation node combinations to obtain a plurality of segmented document sets, comprehensively considers the boundary recognition accuracy and the semantic unit integrity to evaluate the plurality of segmented document sets, thereby optimally selecting an optimal segmented document set as the input of the AI large model, and optimally selecting a segmented document with the highest multi-dimensional quantitative evaluation score as the final result of content extraction, which is beneficial to improve the accuracy of long document content extraction. Furthermore, by calling the AI large model interface for each segmented document included in the optimal segmented document set, a dialogue isolation mechanism is introduced, which is beneficial to suppress the illusion rate in multi-round dialogue.

[0018] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0019] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0021] Figure 1 This is a flowchart of a long document content extraction method based on dynamic segmentation and multi-model optimization in one embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0023] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0024] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0025] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0026] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0027] To overcome the problems existing in current long text processing, this invention proposes a long document content extraction method based on dynamic segmentation and multi-model optimization, which includes the following steps:

[0028] Step S110: identify the chapter boundaries and the paragraph boundaries of the long document to be content extracted, and calculate the semantic correlation degree of adjacent paragraphs separated by the paragraph boundaries.

[0029] Step S120: take the chapter boundaries and the paragraph boundaries with a semantic correlation degree lower than a preset threshold as pending segmentation nodes, select combinations of the pending segmentation nodes according to a preset screening rule, and segment the long document into different segmented document sets based on different combinations of the pending segmentation nodes; wherein the segmented document sets contain multiple segmented documents segmented from the long document.

[0030] Step S130: obtain artificially labeled chapter boundary and paragraph boundary information, calculate boundary recognition accuracy of different segmented document sets based on the artificial labeling, and calculate semantic unit integrity of different segmented document sets, and take the segmented document set with the most optimal combination of boundary recognition accuracy and semantic unit integrity as the optimal segmented document set.

[0031] Step S140: call multiple AI large model interfaces for natural language processing, each time taking the combination of a segmented document contained in the optimal segmented document set and a prompt word template as input, and each AI large model outputs a content extraction result of a segmented document.

[0032] Step S150: for each segmented document, multi-dimensionally quantitatively evaluate the output of each call to the AI large model, take the one with the highest evaluation score as the final content extraction result of the current segmented document, and integrate the final content extraction results of all segmented documents contained in the optimal segmented document set to obtain the long document content extraction result.

[0033] The long document content extraction method based on dynamic segmentation and multi-model optimization provided by the present application combines automatic recognition and artificial labeling to determine segmentation nodes for text segmentation, traverses combinations of the segmentation nodes to obtain multiple segmented document sets, comprehensively considers boundary recognition accuracy and semantic unit integrity to evaluate the multiple segmented document sets, thereby optimally selecting the optimal segmented document set as input of the AI large model, and optimally selecting the one with the highest evaluation score in multi-dimensionally quantitative evaluation as the final content extraction result of each segmented document, which is conducive to improving the accuracy of long document content extraction. Moreover, by calling the AI large model interface for each segmented document contained in the optimal segmented document set, a conversation isolation mechanism is introduced, which is conducive to suppressing hallucination rate in multiple rounds of conversation.

[0034] In an embodiment of the present application, the step of identifying the chapter boundaries and the paragraph boundaries of the long document to be content extracted is obtained by using a regular expression matching method.

[0035] Further, in an embodiment of the present application, the preset screening rule comprises a length range of the segmentation window, and the length candidate value range is 500-8000 token numbers.

[0036] By combining the regular expression matching and the preset screening rule, the embodiment of the present application is beneficial to realize the dynamic structured long text segmentation, and the semantic accuracy and semantic integrity are guaranteed as much as possible in the text segmentation process.

[0037] In an embodiment of the present application, the data structure of the segmented document set is stored by a structured metadata table, and each entry of the structured metadata table comprises a segment ID, a start position, a length, a chapter level and a parent node of a segmented document.

[0038] By using the embodiment of the present application, the start position and the length of each segmented document in the segmented document set based on the dynamic segmentation can be recorded comprehensively, the different segmented documents are identified by the segment ID, and the position of the segmented document in the long document is located by the chapter level and the parent node, so that the context logic of the long document original text is considered in the process of extracting and integrating the content of the segmented document, and the semantic information in the long document is preserved to the greatest extent.

[0039] In an embodiment of the present application, the step of calculating the boundary recognition accuracy of different segmented document sets based on the artificial marking comprises: determining the total number of segments based on the chapter boundary and the paragraph boundary information marked by the artificial marking, comparing the chapter boundary and the paragraph boundary marked by the artificial marking with the recognized chapter boundary and paragraph boundary to determine the correct segmentation segment number, and dividing the correct segmentation segment number corresponding to the current segmented document set by the total segment number to obtain the boundary recognition accuracy.

[0040] By using the embodiment of the present application, the quality of the automatically recognized chapter boundary and paragraph boundary can be analyzed by means of the artificial marking, and then the segmented document set that can make the boundary recognition accuracy and the semantic unit integrity comprehensive optimal can be determined.

[0041] In an embodiment of the present application, the step of calculating the semantic unit integrity of different segmented document sets comprises: calculating the total number of semantic units of the long text, calculating the number of truncated semantic units when the long text is segmented by different pending segmentation nodes, and calculating the semantic unit integrity score C(L) based on the formula The semantic unit integrity score C(L) is calculated, wherein L represents the length range of the segmentation window included in the preset screening rule; and the segmented document set that can make the boundary recognition accuracy and the semantic unit integrity comprehensive optimal is determined by calculating the harmonic mean of the boundary recognition accuracy and the semantic unit integrity score.

[0042] By using the embodiment of the present application, the segmented document set that can make the boundary recognition accuracy and the semantic unit integrity comprehensive optimal can be determined by calculating the semantic unit integrity score, so that the optimal solution of the dynamic segmentation of the long text can be determined.

[0043] In an embodiment of the present application, the prompt word template comprises a role played by the AI large model, an interface of the called AI large model, a segmented document as input, an element that needs to be strictly contained in the output, and an output format requirement.

[0044] By using the embodiment of the present application, the content extraction results output by different AI large models for different segmented documents can be obtained through batch input and output of prompt word templates.

[0045] In an embodiment of the present application, the step of multi-dimensionally quantitatively evaluating the output of each call of the AI large model comprises: based on a preset weight, quantitatively evaluating the output of each call of the AI large model from multiple dimensions including accuracy, integrity, illusion rate, and robustness.

[0046] By using the embodiment of the present application, the quality of the model output in multiple dimensions is comprehensively considered, so that the content extraction result with the best quality in the optional range is comprehensively obtained.

[0047] In an embodiment of the present application, the method further comprises: for each segmented document, obtaining a preset number of artificial labeling results labeled by experts independently and conforming to a three-level labeling system, and the Kappa coefficient of the artificial labeling result is greater than 0.8, and evaluating the output of each call of the AI large model based on the artificial labeling result; wherein the three-level labeling system comprises a core argument, a supporting argument, and abnormal content.

[0048] By using the embodiment of the present application, the output of the called AI large model can be evaluated in combination with expert labeling.

[0049] In the specific implementation process, in step S110, the long document is first structured segmented, different dynamic segmentation methods are tried, and the document segmentation evaluation result is calculated. The length L of si corresponding to each segmented document is recorded to form a segmented document set Si ([s1, s2, s3]).

[0050] The process of structured segmentation of the long document comprises: (1) chapter boundary identification, which can adopt a double-modal segmentation of regular expression matching + manual verification. The manual verification (intervention rule) is: if the semantic correlation degree of adjacent paragraphs > threshold (such as cosine similarity > 0.7), the segmentation is prohibited. (2) The length range of the segmentation window used for long document segmentation is dynamically adjusted according to the dynamic window adjustment formula. By traversing different segmentation lengths L, different combination modes of segmentation nodes are tried, so as to determine the window size that can make the accuracy and integrity comprehensive optimal, and avoid being too long (content redundancy) or too short (semantic break). The segmented document set Si ([s1, s2, s3]) is formed.

[0051] The regular expression is: r'(?m)^#{1,3}\s*(?:\d+\.)*\d+\s+[\u4e00-\u9fa5a-zA-Z]+'. The meaning of this regular expression is to match the Markdown format title line (1-3 level title, with serial number and Chinese and English title text). And use the structured metadata table record segmentation parameters, the constructed structured metadata table is as follows.

[0052] Table 1 constructed structured metadata table example.

[0053] Segment ID of a split document Start position Length Chapter hierarchy Parent node S1 0 1024 1 NULL S2 1025 768 2 S1

[0054] The dynamic window adjustment formula is:

[0055]

[0056] In the above formula, L is the length range of the segmentation window (unit: token), and the candidate value range is usually 500-8000 tokens. A(L): The average accuracy (Accuracy) under the current segmentation length, which measures whether the segmentation boundary is consistent with the manual annotation, and the calculation method is: C(L): Completeness score (Completeness), which measures whether the complete semantic unit is retained after segmentation. The calculation method is: The harmonic mean formula is expressed as Balancing accuracy and completeness to avoid single index bias. The physical meaning of the above formula is to find the length L of the segmentation window that maximizes the harmonic mean.

[0057] In the specific implementation process, in step S140, different AI large models are tried, prompt words are written and optimized, one segmentation document si is input at a time, and output results are obtained. In addition, based on the dynamic variable injection technology, the calling and input of different AI large models are realized. Different prompt words Pro={p1, p2, p3} are tried, where Pro is the prompt word set, p1, p2, and p3 are single prompt words, and the best prompt word is obtained according to the quantitative evaluation model evaluation result.

[0058] In the actual encoding process, the prompt word template can be in the following format: prompt_template = {"role": "system", "content": "As a {domain} expert, please extract the core content of the {section_type} section in the following text, which must strictly include: {elements}. The output format is JSON key-value pair: {format}."} # Example variable replacement: {domain: "computer science", section_type: "methodology", elements: ["assumption conditions", "experimental parameters"]}.

[0059] Further, adversarial test designs can be inserted into the prompt word template to detect the model's resistance to interference by inserting interfering sentences (such as irrelevant mathematical formulas) and to test robustness R by using confusing section titles (such as "3.1 Experimental Design" vs. "3.1 Experimental Design").

[0060] In the implementation process, the quantification evaluation process in step S150 can use an expert annotation system or a multi-dimensional scoring algorithm. The output results of different large models and different length segmented documents are compared using a quantification evaluation model to obtain the large model with the best content extraction result and the most suitable segmented document set.

[0061] For the expert annotation system, it is based on the gold standard (Gold Standard). The expert annotation system includes a defined three-level label system: {"core argument": {"required": true, "weight": 0.6}, "supporting argument": {"required": true, "subtypes": ["data", "quote"]}, "abnormal content": {"type": "hallucination detection", "penalty": -0.2}}

[0062] In the annotation process, double-blind annotation can be used, that is, two experts independently annotate, and the Kappa coefficient needs to be greater than 0.8.

[0063] For the multi-dimensional scoring algorithm, multiple dimensions such as accuracy, completeness, hallucination rate, and robustness are considered.

[0064] The defined hallucination rate quantification formula is:

[0065]

[0066] Parameter description:

[0067] H: Hallucination Rate, the proportion of incorrect or fabricated content in the model output.

[0068] n: Total number of test samples (i.e., the number of labeled gold standard paragraphs).

[0069] f model (xi): The set of output results of the model on the input paragraph xi (such as extracted arguments and evidence).

[0070] G(xi): The gold standard result set (i.e., the correct answer) annotated by experts.

[0071] II(): Indicator function, takes the value 1 if the model output is not in the gold standard, otherwise takes the value 0.

[0072] Calculation example: If 10 paragraphs are tested, and 3 of them contain unlabeled content, then H = 3 / 10 × 100% = 30%.

[0073] The evaluation formula for the model, which integrates various dimensions, is as follows:

[0074] Parameter description:

[0075] Q: The overall quality score of the model is used for horizontal comparison of different models.

[0076] A: Accuracy is used to evaluate the matching rate between the model output and the gold standard.

[0077] C: Completeness, used to analyze the extent to which the model covers all the necessary content of the gold standard.

[0078] H: Hallucination Rate.

[0079] R: Robustness, used to evaluate stable performance under adversarial tests (such as distractors or misspellings).

[0080] A, β, γ, δ: Weighting coefficients used to adjust priority based on document type.

[0081] In the specific implementation process, the weight coefficients for different documents are as follows: (1) Scientific papers: α=0.6, β=0.2, γ=0.15, δ=0.05 (emphasizing accuracy). (2) Legal documents: α=0.4, β=0.4, γ=0.1, δ=0.1 (emphasizing completeness).

[0082] Its practical significance lies in the fact that by adjusting the weights, the scoring system can be adapted to the needs of different scenarios. For example, legal documents require the complete retention of clauses, while academic papers must strictly avoid fabricating data.

[0083] Further, in some embodiments of the application, a dialogue isolation execution mechanism is introduced. Each time the model interface is called, a new session is created to prevent hallucination and reduce the hallucination rate.

[0084] The long document content extraction method based on dynamic segmentation and multi-model optimization provided by the application combines automatic recognition and manual annotation to determine segmentation nodes for text segmentation, traverses segmentation node combinations to obtain multiple segmented document sets, comprehensively considers boundary recognition accuracy and semantic unit integrity to evaluate the multiple segmented document sets, thereby optimally selecting an optimal segmented document set as the input of an AI large model, and optimally selecting a multi-dimensional quantitative evaluation score as the final result of content extraction of each segmented document, which is beneficial to improving the accuracy of long document content extraction. Furthermore, by calling an AI large model interface for each segmented document in the optimal segmented document set, a dialogue isolation mechanism is introduced, which is beneficial to suppressing the hallucination rate in multi-round dialogue.

[0085] The long document content extraction method based on dynamic segmentation and multi-model optimization provided by the application adopts a long document dynamic segmentation strategy based on semantic structure, further establishes a quantitative mapping relationship between segmentation length and model performance, flexibly adjusts the segmentation strategy, dynamically selects the combination of the optimal model and segmentation strategy, and suppresses hallucination through a dialogue isolation mechanism.

[0086] The method provided by the application is tested through experiments, and the beneficial effects thereof include but are not limited to: (1) accuracy improvement: on the arXiv paper test set, the accuracy is improved from 72% to 89%; (2) hallucination suppression: the hallucination rate of continuous processing is reduced to below 3% through the isolation mechanism; (3) adaptive optimization: dynamically selecting model-segmentation combinations, reducing resource consumption by 20%.

[0087] Corresponding to the above method, the application further provides a long document content extraction system based on dynamic segmentation and multi-model optimization. The system comprises a computer device, the computer device comprises a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as described above.

[0088] Corresponding to the above method, the present application also provides a computer readable storage medium, which stores computer program / instructions, and the computer program / instructions are executed by a processor to realize the steps of the method according to any one of the above embodiments. The computer readable storage medium can be a tangible storage medium, such as random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable memory disk, CD-ROM, or any other form of storage medium known in the art.

[0089] Corresponding to the above method, the present application also provides a computer program product, which comprises computer program / instructions, and the computer program / instructions are executed by a processor to realize the steps of the method according to any one of the above embodiments.

[0090] Those of ordinary skill in the art should understand that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether the implementation is in hardware or software depends on the specific application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, the hardware can be implemented in, for example, electronic circuitry, an application-specific integrated circuit (ASIC), suitable firmware, a plug-in card, a function card, etc. When implemented in software, the elements of the present application are the program or code segments to perform the necessary tasks. The program or code segments can be stored in a machine-readable medium, or carried in a data signal in a carrier wave modulated by a carrier, on a transmission medium or a communication link.

[0091] It is to be understood that the present application is not limited to the particular configurations and processes described herein and shown in the drawings. For simplicity, detailed descriptions of well-known methods are omitted. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0092] In the present application, the features described and / or illustrated for one embodiment can be used in the same way or in a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0093] The above merely illustrates the preferred embodiments of the present application, and is not used to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall fall within the scope of protection of the present application.

Claims

1. A long document content extraction method based on dynamic segmentation and multi-model optimization, characterized in that, The method comprises the following steps: Identify the chapter boundaries and paragraph boundaries of a long document to be content-extracted, and calculate the semantic correlation of adjacent paragraphs separated by paragraph boundaries; Take the chapter boundaries and paragraph boundaries with a semantic correlation lower than a preset threshold as pending segmentation nodes, select combinations of the pending segmentation nodes according to a preset screening rule, and segment the long document into different sets of segmented documents based on different combinations of the pending segmentation nodes; wherein, the sets of segmented documents comprise a plurality of segmented documents segmented from the long document; Obtain artificially annotated chapter boundaries and paragraph boundaries, calculate the boundary recognition accuracy of different sets of segmented documents based on the artificial annotation, and calculate the semantic unit integrity of different sets of segmented documents, and take the set of segmented documents with the most optimal comprehensive boundary recognition accuracy and semantic unit integrity as the optimal set of segmented documents; Call a plurality of AI large model interfaces for natural language processing, each time taking the combination of a segmented document contained in the optimal set of segmented documents and a prompt word template as input, and each AI large model outputs a content extraction result of the segmented document; For each segmented document, multi-dimensionally quantify and evaluate the output of each call to the AI large model, take the one with the highest evaluation score as the final content extraction result of the current segmented document, and integrate the final content extraction results of all segmented documents contained in the optimal set of segmented documents to obtain the long document content extraction result.

2. The method of claim 1, wherein, The step of identifying the chapter boundaries and paragraph boundaries of the long document to be content-extracted is obtained by regular expression matching; The preset screening rule includes the length range of the segmentation window, and the length candidate value range is 500-8000 token numbers.

3. The method of claim 1, wherein, The data structure of the set of segmented documents is stored through a structured metadata table, and each item of the structured metadata table includes the segment ID, starting position, length, chapter level and parent node of a segmented document.

4. The method of claim 1, wherein, The step of calculating the boundary recognition accuracy of different sets of segmented documents based on artificial annotation comprises: Determine the total number of segments based on the artificially annotated chapter boundaries and paragraph boundaries, compare the artificially annotated chapter boundaries and paragraph boundaries with the identified chapter boundaries and paragraph boundaries to determine the number of correctly segmented segments, and divide the number of correctly segmented segments corresponding to the current set of segmented documents by the total number of segments to obtain the boundary recognition accuracy.

5. The method of claim 4, wherein, The step of calculating the semantic unit integrity of different sets of segmented documents comprises: The total semantic unit number of the long text is calculated, the truncated semantic unit number of the long text segmented by different pending segmentation nodes is calculated, and the formula The semantic unit integrity score C(L) is calculated, wherein L represents the length range of the segmentation window included in the preset screening rule; and wherein the segmentation document set that optimizes the boundary recognition accuracy and the semantic unit integrity is determined by calculating the harmonic mean of the boundary recognition accuracy and the semantic unit integrity score.

6. The method of claim 1, wherein, The prompt word template includes the role played by the AI large model, the interface of the called AI large model, the input segmented document, the elements that must be strictly contained in the output, and the output format requirements.

7. The method of claim 1, wherein, The step of multi-dimensionally quantifying and evaluating the output of each call to the AI large model comprises: based on a preset weight, quantifying and evaluating the output of each call to the AI large model from multiple dimensions including accuracy, integrity, hallucination rate and robustness.

8. The method of claim 1, wherein, The method further comprises: for each segmented document, obtaining a preset number of expert-independent labeled artificial labeling results conforming to a three-level labeling system, and a Kappa coefficient of the artificial labeling results is greater than 0.8; and evaluating an output of each call of the AI large model based on the artificial labeling results; wherein the three-level labeling system comprises a core argument, a supporting argument, and abnormal content.

9. A long document content extraction system based on dynamic segmentation and multi-model optimization, comprising a processor, a memory and computer programs / instructions stored on the memory, characterized in that, The processor is configured to execute the computer programs / instructions, and the system implements the steps of the method according to any one of claims 1 to 8 when the computer programs / instructions are executed.

10. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer programs / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 8.

Images