Multimodal nested list extraction method and device independent of indentation detection

Abstract

The present application relates to the technical field of text processing, and more particularly to a multi-modal nested list extraction method and device independent of indentation detection, the extraction method comprising: detecting list missing items of a list-containing text to be extracted; detecting list structure integrity of the list-containing text to be extracted; selecting a first extraction mode or a second extraction mode to perform list extraction on the text to be extracted according to the list missing item detection result and the list structure integrity detection result; the first extraction mode and the second extraction mode both comprising complete independent list extraction and composite extraction for retaining nested list structure, the complete independent list extraction being used to extract a single complete independent list with the same label from a text paragraph, and the composite extraction for retaining nested list structure being used to simultaneously extract nested and independent lists from a text paragraph. The present application solves the problems of insufficient precision, small fault tolerance space and unguaranteed correctness of extracted structured data in the prior art.

Description

Technical Field

[0001] This invention relates to the field of text processing technology, and in particular to a method and apparatus for extracting multimodal nested text lists without relying on indentation detection. Background Technology

[0002] Currently, the mainstream document saving formats are generally Microsoft Word's doc(x) format or the PDF format originally proposed by Adobe. A certain degree of mutual conversion is possible between the two. For example, a Word document can be saved as a PDF, usually to fix the document's visual formatting, but this results in the loss of most of the document's editability and Word's formatting attributes. Converting a PDF file to a Word file involves numerous formatting errors or typos caused by OCR (Optical Character Recognition) errors. Furthermore, in traditional industries or with older documents, many original electronic files have been lost, or only image formats remain for printing. Such content can only be restored to a usable data format using OCR combined with manual post-proofreading.

[0003] For extracting list-style content from documents, existing tools that can directly extract information from Word documents are based on parsing Word's object model. This model is similar to the DOM (Document Object Model) of a webpage. Essentially, it uses a markup language to encapsulate information beyond the text content itself within objects that can be objectified and manipulated using a programming language. The difference is that the style information contained in the Word document's object model primarily comes from manual settings within the visual document editing interface, while the webpage DOM is generated based on structured code. In the former case, which is also the most common, manual formatting is highly subjective, even under certain document specifications. This inevitably leads to errors in the parsed list structure or content.

[0004] Methods for extracting visual information from lists in documents require computers to recognize or distinguish the general features of lists as a textual form within a document, as well as the features that distinguish lists from other content formats. The main technology for achieving this is machine learning, using a large, manually labeled training set to train a model capable of recognizing lists from documents and extracting their content. Currently, there are no publicly available models for extracting list content, but some tools and research exist for extracting table content. However, in practical applications, models using this method perform poorly, primarily due to the following limiting factors:

[0005] (1) Since model feature recognition is based on visual features, the training set and the input in actual use of the model need to be unified into a format that carries text information in images, such as PDF. The conversion between formats increases the processing cost.

[0006] (2) The construction cost of the training set is high. The construction of the training set involves feature engineering. The selection and labeling of list features and subsequent data labeling each step involve a lot of manual expenses. In addition, the subjective attributes of feature definition and selection will introduce uncertainty. Coupled with the inevitable jitter and errors in the manual labeling process, it is almost impossible to obtain a high-quality and sufficient training set at a low cost.

[0007] (3) The model performance is highly uncertain. Many factors may lead to this result. First, there is the issue of ensuring the quality and quantity of the training set mentioned above. Second, there are no mature precedents for training this type of model. Model training relies more on repeated trial and error and repeated testing within a certain range. In addition, the generalization ability of the model is also highly uncertain. As it is a special task model, the model may be more sensitive to changes in some non-key dimension features or non-essential changes in key features in the input data, which may lead to fluctuations in the model performance.

[0008] In general, existing extraction techniques for list extraction from different types of files rely on information dimensions attached to the list itself by the file format. This part of the generation involves a lot of human subjectivity and error information. The conversion between different formats can also cause the accumulation and transmission of errors. These factors ultimately lead to insufficient accuracy and small fault tolerance in list extraction, and the correctness of the extracted structured data cannot be guaranteed. Summary of the Invention

[0009] In view of this, the purpose of the present invention is to provide a method and apparatus for extracting multimodal nested text lists that does not rely on indentation detection, so as to at least solve the problems of insufficient accuracy, small fault tolerance, and inability to guarantee the correctness of the extracted structured data in the prior art.

[0010] The present invention solves the above-mentioned technical problems through the following technical means:

[0011] In a first aspect, the present invention provides a method for extracting multimodal nested text lists that does not rely on indentation detection, comprising the following steps:

[0012] The text to be extracted, which contains a list, is subjected to list missing item detection, which includes first item missing detection and middle item missing detection;

[0013] Perform list structure integrity detection on the text to be extracted containing lists, and configure a first threshold for missing item tolerance in the structure integrity detection;

[0014] Based on the list missing item detection results and the list structure integrity detection results, a first extraction method or a second extraction method is selected to perform list extraction on the text to be extracted. Both the first extraction method and the second extraction method contain a composite extraction method that preserves the nested list structure and extracts completely independent lists. The completely independent list extraction method is used to extract a complete independent list of a single label of the same type from the text paragraph. The completely independent list extraction method is configured with a second threshold for missing item tolerance. The composite extraction method that preserves the nested list structure is used to extract both nested lists and independent lists from the text paragraph at the same time.

[0015] In conjunction with the first aspect, in some optional implementations, based on the detection results that the list has no missing items and the list structure is complete, a first extraction method is selected to perform list extraction on the text to be extracted. The first extraction method includes:

[0016] Perform compound extraction on the text to be extracted, preserving the nested list structure, to obtain nested lists and independent lists, thus completing the list extraction;

[0017] Based on the detection results indicating missing items or incomplete list structure, the second extraction method is selected to perform list extraction on the text to be extracted. The second extraction method includes:

[0018] Perform a completely independent list extraction on the text to be extracted, extract the independent list, then exclude the extracted list items from the original text to be extracted, and merge the remaining lines;

[0019] Perform a composite extraction on the remaining rows after merging, preserving the nested list structure.

[0020] In conjunction with the first aspect, in some alternative implementations, the first-term missing detection includes:

[0021] Scan the text to be extracted from top to bottom in a forward direction;

[0022] Determine whether the current row of the scanned list is the second item of the sorted list;

[0023] Based on the fact that the current list row is the second item of the ordered list, the initial balance value is set to 1, and the list label type of the current row is obtained. Then, the reverse scanning from row to row begins.

[0024] When a reverse scan encounters the second item of the same label type, the balance value is incremented by 1; if the first item of the same label type is encountered, the balance value is decremented by 1.

[0025] After scanning all rows in reverse, check if the balance value is 0. If the balance value is 0, there is no missing first item in the list; if the balance value is not 0, there is a missing first item in the list.

[0026] In conjunction with the first aspect, in some optional implementations, the intermediate term missing detection is performed based on the absence of a first term missing, and the intermediate term missing detection includes:

[0027] Perform completely independent list extraction on the text to be extracted to obtain a list set A. The second threshold in this step is 0.

[0028] Perform a compound extraction that preserves the nested list structure on the remaining text paragraphs after completely independent list extraction to obtain list set B;

[0029] Calculate the number of list items in list set A and list set B respectively, and obtain CA and CB accordingly;

[0030] Compare and confirm whether the values ​​of CA and CB are equal. If they are equal, there are no missing terms; if they are not equal, there are missing terms.

[0031] In conjunction with the first aspect, in some optional implementations, the step of performing list structure integrity detection on the text to be extracted containing a list, and configuring a first threshold for missing item tolerance in the structure integrity detection, includes:

[0032] Perform completely independent list extraction on the text to be extracted to obtain all independent lists in the text to be extracted. Calculate the list items of each independent list and sum them to obtain the total list item sum E.

[0033] Perform a compound extraction on the text to be extracted, preserving the nested list structure, to obtain a nested list and an independent list.

[0034] Perform completely independent list extraction on each of the obtained nested lists to obtain independent list two;

[0035] Calculate the list items in independent list one and independent list two respectively, and sum them to obtain the total list item sum F;

[0036] Determine if E and F are equal. If they are equal, there is no inserted row in the list that could compromise the structural integrity. If they are not equal, there is an inserted row that could compromise the structural integrity.

[0037] In conjunction with the first aspect, in some optional implementations, the completely independent list extraction includes ordered list extraction and unordered list extraction, wherein the ordered list extraction includes:

[0038] Locate the last line of the text to be extracted, and scan the text backwards from the last line.

[0039] During the scanning process, the first item is identified, and the forward scanning begins. It is determined whether the label type of the next row is the same as that of the current row and whether the label value is within the second threshold range. If the label type of the next row is the same as that of the current row and the label value is within the second threshold range, extraction is performed. If the label type of the next row is different from that of the current row or the label value is not within the second threshold range, extraction is interrupted, thus completing the extraction of an independent list.

[0040] Remove all list items from the extracted independent lists from the text to be extracted, and merge the remaining lines;

[0041] The remaining text lines are then scanned and extracted again in reverse.

[0042] The extraction of the unordered list includes:

[0043] Locate the last line of the text to be extracted, and scan the text backwards from the last line.

[0044] Retrieve the label type of list items during scanning;

[0045] Determine if the label type of the current row is the same as the label type of the adjacent previous row. If they are the same, extract the label; otherwise, continue scanning in reverse.

[0046] In conjunction with the first aspect, in some optional embodiments, the identification of the first term includes the identification of Roman numeral first terms and the differentiation of easily confused symbols. The identification of Roman numeral first terms includes:

[0047] During the scanning process, when the label of the current row is identified as i or I, the next row is scanned downwards in a positive direction and the label of the next row list is obtained. It is then determined whether the label of the next row list is ii or II. If it is, the current row is a list of Roman numeral first items; otherwise, the current row is not a list of Roman numeral first items.

[0048] The distinction and recognition of easily confused labels is used to identify Roman numeral and non-Roman numeral neighboring items in ordered list extraction.

[0049] In conjunction with the first aspect, in some optional implementations, the composite extraction that preserves the nested list structure includes:

[0050] Locate the first line of the text to be extracted, and scan the text downwards in a forward direction starting from the first line;

[0051] Starting from the first list item, continuously extract adjacent list items until a non-list item is encountered, at which point the extraction stops, resulting in a complex list.

[0052] Perform completely independent list extraction on complex lists, and determine the list type based on the number of extracted lists.

[0053] In conjunction with the first aspect, in some optional implementations, determining the list type based on the number of extracted lists includes:

[0054] Calculate the number of extracted lists;

[0055] Check if the list count is greater than or equal to 1. If the list count is equal to 1, mark it as an independent list. If the list count is greater than 1, mark it as a nested list.

[0056] The multimodal nested text list extraction method of this invention, which does not rely on indentation detection, selects its extraction strategy based on the detection results of two types of errors: missing list items and incomplete list structure. The core purpose of the extraction strategy design for cases containing errors is to isolate errors to the greatest extent possible, ensuring the completeness and accuracy of list extraction within the largest unit range. Extracting independent lists eliminates the possibility of list item extraction errors caused by structurally broken lines, ensuring that extractable independent lists are obtained from the text. Then, after merging the remaining lines, a composite extraction that preserves the nested list structure is performed. This ensures that no list lines are missed. If complete independent lists still exist in the merged lines, they can also be extracted correctly. The parts that cannot be extracted as independent lists will exist in the form of nested lists.

[0057] The multimodal nested text list extraction method of this invention, which does not rely on indentation detection, has several advantages. Firstly, list extraction no longer depends on visual features or additional attribute features inherent in specific text formats; it extracts only based on the core symbolic features of the text list. Compared to similar machine learning techniques, this invention replaces costly machine learning feature engineering with simple manual cleaning. Secondly, this invention recognizes multiple labeling patterns in ordered and unordered lists and effectively handles easily confused items between English and Roman numeral labels. Furthermore, this invention implements two detection functions: missing item detection and list structure integrity detection. In addition, this invention implements configurable missing item tolerance extraction and inter-item row insertion skipping functions to address various possible errors. It also implements three selectable extraction strategies: completely independent list extraction, composite extraction that preserves the nested list structure, and a composite list extraction function with a configurable fault-tolerant mechanism that integrates error detection and the first two extraction functions to ensure the integrity and correctness of the extracted data.

[0058] Secondly, the present invention provides a multimodal nested text list extraction device that does not rely on indentation detection, comprising:

[0059] The missing item detection module is used to perform list missing item detection on the text to be extracted containing a list. The list missing item detection includes first item missing detection and middle item missing detection.

[0060] The integrity detection module is used to perform list structure integrity detection on the text to be extracted containing a list, and to configure the first threshold for missing item tolerance in the structure integrity detection.

[0061] The extraction method selection module is used to select either a first extraction method or a second extraction method to perform list extraction on the text to be extracted based on the list missing item detection results and the list structure integrity detection results. Both the first and second extraction methods include a composite extraction method that preserves the nested list structure and a completely independent list extraction method. The completely independent list extraction method is used to extract a complete independent list of a single label of the same type from the text paragraph. The completely independent list extraction method is configured with a second threshold for missing item tolerance. The composite extraction method that preserves the nested list structure is used to extract both nested lists and independent lists from the text paragraph simultaneously.

[0062] It is understandable that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating a multimodal nested text list extraction method that does not rely on indentation detection.

[0064] Figure 2This is an example of easily confused labels;

[0065] Figure 3 This is a reference example two for easily confused labels;

[0066] Figure 4 This is a reference example where the last item of the parent list is followed by a nested sublist;

[0067] Figure 5 This is a reference example of a middle item in a parent list followed by a nested sublist;

[0068] Figure 6 This is an example of an incomplete list;

[0069] Figure 7 This is a diagram illustrating the process of extracting a list of text paragraphs.

[0070] Figure 8 This is a flowchart of the workflow for a composite list extraction function with a configurable fault-tolerant mechanism.

[0071] Figure 9 This is a flowchart of the workflow for a completely independent list extraction function;

[0072] Figure 10 This is a flowchart of the workflow for a composite extraction function that preserves the nested list structure;

[0073] Figure 11 This is a flowchart of the workflow for determining the continuity relationship of adjacent items with missing item tolerance configuration;

[0074] Figure 12 This is a flowchart of the workflow for extracting from an ordered list with missing item tolerance configuration;

[0075] Figure 13 This is the workflow diagram for the first missing list judgment function;

[0076] Figure 14 This is a schematic diagram of a multimodal nested text list extraction device that does not rely on indentation detection. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0078] Unless the context otherwise specifies, the terms “contains,” “has,” and “includes” are synonyms. The phrase “A / B” means “A or B,” and the phrase “A and / or B” means “(A and B) or (A or B).”

[0079] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. In the description of embodiments of this application, unless otherwise stated, "a plurality of" means two or more, for example, "a plurality of processing units" means two or more processing units, "a plurality of elements" means two or more elements, etc.

[0080] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0081] As used herein, the term "module or unit" may refer to or include application-specific integrated circuits (ASICs), electronic circuits, processors (shared, dedicated, or grouped) and / or memories (shared, dedicated, or grouped) that execute one or more software or firmware programs, combinational logic circuits, and / or other suitable components that provide the described functionality, or may be part of an application-specific integrated circuit (ASIC), electronic circuits, processors (shared, dedicated, or grouped) that execute one or more software or firmware programs, combinational logic circuits, and / or other suitable components that provide the described functionality.

[0082] To facilitate understanding of the embodiments of this application, the following description of some of the embodiments of this application is provided for explanation and illustrative purposes, so that those skilled in the art can understand them.

[0083] For extracting list content from documents, existing tools that can directly extract information from Word documents are based on parsing Word's object model. This model is similar to the DOM (Document Object Model) of a webpage. Essentially, it uses a markup language to encapsulate information other than the text content itself in a programming language into objects that can be objectified and manipulated. The difference is that the various style information contained in the Word document object model primarily comes from manual settings on the visual document editing interface, while the webpage DOM is generated based on structured code. In the former case, which is also the most common, manual formatting is highly subjective, and even under certain document specifications, this inevitably leads to errors in the parsed list structure or content. Methods for extracting visual information from lists in documents require computers to be able to recognize or distinguish the general characteristics of lists as a textual form within a document, as well as the features that distinguish lists from other content formats. The main technology for achieving this is machine learning, using a large, manually labeled training set to train a model capable of recognizing lists from documents and extracting their content. Currently, there are no publicly available models of this type for extracting list content. In general, existing list extraction techniques for different types of files rely on information dimensions additional to the list itself from the file format. This reliance involves significant human subjectivity and errors, and conversions between different formats can lead to the accumulation and propagation of errors. These factors ultimately result in insufficient accuracy and limited error tolerance in list extraction, making it impossible to guarantee the correctness of the extracted structured data. In view of these problems, this invention proposes a multimodal nested text list extraction method and apparatus that does not rely on indentation detection.

[0084] This invention presents a multimodal nested text list extraction method and apparatus that does not rely on indentation detection. It employs "dimensionality reduction processing" independent of specific file formats, using plain text content as the basis. It extracts data starting from the core information of the "list" concept and designs and implements multiple configurable fault-tolerant mechanisms. In the "list" concept, the most basic dimensions include: 1) line breaks, 2) list item identifiers, and 3) dependencies between list items. These (textual) symbolic features are essentially visual features, but because they are based on human consensus on the information organization of lists, information is not lost during normal transmission, and even small amounts of error accumulation can be logically recovered from this consensus. In contrast, extraction methods based on manually added dimensions such as indentation and object-oriented encapsulation, which are not core dimensions, suffer from functional instability and poor generalization performance due to their variability.

[0085] The multimodal nested text list extraction method of the present invention, which does not rely on indentation detection, combines two list extraction methods: completely independent list extraction and composite extraction that retains the nested list structure, as well as error detection mechanisms such as missing item detection and integrity detection. Combined with fault tolerance mechanisms such as skipping inserted rows between items and tolerating missing items, it forms a composite list extraction function with configurable fault tolerance mechanisms.

[0086] Please refer to Figure 1 , Figure 7 and Figure 8 The multimodal nested text list extraction method of the present invention, which does not rely on indentation detection, includes the following steps:

[0087] Step 110 involves performing list missing item detection on the text containing lists to be extracted. List missing item detection includes first item missing detection and middle item missing detection. For a single list, a missing item refers to the absence of one or more consecutive or non-consecutive labeled items in the ordered list. For example, if the original list labels are: a), b), c), d), e), f), and for some reason the list labels change to: b), c), d), e), f) or a), b), c), f). For a completely independent list extraction function, the logical starting point for ordered list extraction is when the first item of the list is encountered from bottom to top during scanning. When the first item of the list is missing (also known as a headless list), the starting point for extraction of that list cannot be confirmed, which may lead to extraction failure of that list or cause unknown interference to the extraction of other lists in the same text, especially when other lists in the document also have formal or format errors. Therefore, it is necessary to perform list missing item detection.

[0088] Step 120: Perform list structure integrity detection on the text to be extracted, which contains lists, and configure the first threshold for missing item tolerance in the structure integrity detection. Structure integrity detection mainly targets nested lists where non-list item lines are inserted before the first item of a sub-nested list, disrupting the nested structure. This can be caused by problems during file format conversion, such as the insertion of lines between items, or by inappropriate information organization methods or non-standard formatting during list creation. Structural incompleteness may make it impossible to determine whether the original two lists are nested or independent lists based on format alone, and may also cause information structure confusion during table reading. Therefore, it is necessary to perform list structure integrity detection.

[0089] Step 130: Based on the list missing item detection results and the list structure integrity detection results, select either the first extraction method or the second extraction method to perform list extraction on the text to be extracted; both the first and second extraction methods contain a composite extraction method that preserves the nested list structure and a completely independent list extraction method. The completely independent list extraction method is used to extract a complete independent list of a single label of the same type from the text paragraph. The completely independent list extraction method is configured with a second threshold for missing item tolerance. The composite extraction method that preserves the nested list structure is used to extract both nested lists and independent lists from the text paragraph at the same time.

[0090] Based on the detection results that the list has no missing items and the list structure is complete, the first extraction method is selected to perform list extraction on the text to be extracted. The first extraction method includes: performing compound extraction on the text to be extracted while preserving the nested list structure, obtaining nested lists and independent lists, and completing the list extraction.

[0091] Based on the detection results of missing items or incomplete list structure, the second extraction method is selected to perform list extraction on the text to be extracted. The second extraction method includes: performing completely independent list extraction on the text to be extracted, extracting independent lists, then excluding the extracted list items from the original text to be extracted, and merging the remaining lines; performing compound extraction that preserves the nested list structure on the merged remaining lines.

[0092] The multimodal nested text list extraction method of this invention, which does not rely on indentation detection, selects its extraction strategy based on the detection results of two types of errors: missing list items and incomplete list structure. The core purpose of the extraction strategy design for cases containing errors is to isolate errors to the greatest extent possible, ensuring the completeness and accuracy of list extraction within the largest unit range. Extracting independent lists eliminates the possibility of list item extraction errors caused by structurally broken lines, ensuring that extractable independent lists are obtained from the text. Then, after merging the remaining lines, a composite extraction that preserves the nested list structure is performed. This ensures that no list lines are missed. If complete independent lists still exist in the merged lines, they can also be extracted correctly. The parts that cannot be extracted as independent lists will exist in the form of nested lists.

[0093] Please refer to Figure 9 The fully independent list extraction function is a method that extracts lists from a text paragraph containing lists into individual, complete lists of the same type. The list format is not limited to lists originally embedded as independent lists within the text paragraph; it also includes nested or even complexly nested lists. Furthermore, list labeling formats include two main categories: unordered lists and ordered lists. Ordered lists contain numbers, letters, and Roman numerals (which overlap with letter types). Therefore, fully independent list extraction includes both ordered list extraction and unordered list extraction.

[0094] The extraction of ordered lists within completely independent lists employs a design that scans backwards from the last line, then forwards when encountering the first item until the last item is extracted. Once an independent list is extracted, all items contained within it are removed from the text paragraph line, and this extraction process is repeated. A crucial implicit logic is that extraction must begin with the last independent list. This ensures that inner lists within nested lists are extracted first, allowing adjacent items from the previously separated outer lists to rejoin, guaranteeing that all lists are ultimately extracted completely. The steps for extracting ordered lists are as follows:

[0095] (1) Locate the last line of the text to be extracted and scan the text to be extracted backwards from the last line.

[0096] (2) During the scanning process, the first item of the list item is identified for each row, such as identifying list items containing label patterns such as 1, a, i, etc.

[0097] (a) If the first item of the list is identified in step (2) above, an independent list extraction is started, and the forward scanning begins. It is determined whether the label type of the next row is the same as that of the current row and whether the label value is within the range of the second threshold. If the second threshold is set to 0, skipping items is not allowed. If the second threshold is set to 1, one missing item is allowed, and so on.

[0098] I. If the next row has the same label type as the current row and the label value is within the second threshold range, then extract the data.

[0099] II. If the label type of the next row is different from that of the current row or the label value is not within the second threshold range, the extraction is interrupted, thus completing the extraction of an independent list.

[0100] (b) If it is detected in step (2) above that the current row is not the first item of the list, continue scanning upwards in reverse.

[0101] (3) Remove all list items from the extracted independent list from the text to be extracted, and merge the remaining lines.

[0102] (4) Return to step (1) to perform reverse scanning and extraction on the remaining text lines.

[0103] A major challenge in ordered list extraction is the shared identifier letters between alphabetic and Roman numeral lists, such as i, I, v, V, x, X, l, and L. These shared letters can interfere with the extraction process, leading to unexpected issues like interruptions or distorted list structures. Therefore, it is necessary to identify the first Roman numeral item and distinguish easily confused identifiers.

[0104] In the fully independent list extraction function, the identification of Roman numeral first items requires a method to determine whether an item is a middle item of a letter label type or the start item of a Roman numeral label type when encountering the letter label 'i' or 'I' during bottom-to-top scanning. This determination will decide whether the program begins extracting a list of Roman numeral labels or continues scanning upwards. The basic implementation logic is that upon encountering the 'i' or 'I' identifier, it is also necessary to retrieve the next line of list data and determine whether the identifier of the next line is 'ii' or 'II', thereby detecting whether the current line is the start item of a Roman numeral list.

[0105] Please refer to Figure 11 To distinguish easily confused labels, this invention designs and implements a next-neighbor generation function. This function generates the label of the next item based on the current item's label and whether it is a Roman numeral type. This next-neighbor generation function is used to identify Roman numeral and non-Roman numeral (letter and number) neighboring items in ordered list extraction. During independent list extraction, when the first item is encountered, based on the aforementioned Roman numeral first item identification function and the different labels of non-Roman numeral list first items, it can be confirmed at the start of extraction whether the currently extracted list is of Roman numeral type. This determination will be passed as a parameter in subsequent extraction procedures to continuously identify whether the next list item is a neighbor of the previous list item. This function ensures that when encountering the preceding and following items of easily confused labels (i, I, v, V, x, X, l, L) during list extraction, the continuity of the next list item can be determined based on the parameter of whether the current list is of Roman numeral type, thus deciding whether to interrupt the extraction operation. For lists of letter and Roman letter types, the correspondences between the preceding and following items for easily confused labels (i, I, v, V, x, X, l, L) are as follows:

[0106] Letter prefixes: h, H, u, U, w, W, k, K

[0107] Roman type antecedent: i has no antecedent, iv, IV, ix, IX, xlix, XLIX

[0108] Letter-type suffixes: j, J, w, W, y, Y, m, M

[0109] Roman type suffixes: ii, II, vi, VI, xi, XI, li, LI

[0110] The interference caused by easily confused labels is most noticeable when two levels of nested lists intersect, such as when an 'h' item in the first level of an alphabetic list is followed by a list of Roman numerals (starting with 'i'), or when the last item 'u' of the nested alphabetic list intersects with the middle item 'v' of the parent list. Typical examples of this can be found in... Figure 2 and Figure 3 .

[0111] Unordered list extraction in completely independent list extraction also employs a reverse scanning method. However, since there is no order relationship or starting index concept between the list item labels of an unordered list, the extraction method uses an adjacent item extraction strategy, that is, extracting adjacent list items of the same label type. The steps of unordered list extraction are as follows:

[0112] (1) Locate the last line of the text to be extracted and scan the text to be extracted backwards from the last line.

[0113] (2) Obtain the label type of list items during scanning.

[0114] (3) Determine whether the label type of the current row is the same as the label type of the adjacent previous row. If they are the same, extract the label. If they are different, continue the reverse scan.

[0115] Please refer to Figure 10 The composite extraction function, which preserves the nested list structure, extracts both nested and independent lists from text paragraphs. Its implementation partially relies on the aforementioned fully independent list extraction function. While this method preserves the original structure of nested lists, for certain types of text content, such as instruction manuals where some operational processes contain nested sub-operation processes, simply using fully independent list extraction to extract these sub-operation processes individually can lead to the loss of the execution context or specific situation for the relevant operation items. The composite extraction function, which preserves the nested list structure, retains the extraction requirements for such lists, obtaining both nested and independent lists simultaneously while preserving the nested list structure. The basic extraction process for the composite extraction function, which preserves the nested list structure, includes:

[0116] 1) Locate the first line of the text to be extracted, and scan the text downwards in a forward direction starting from the first line.

[0117] 2) Starting from the first list item encountered, continuously extract adjacent list items without considering whether the list items are ordered or unordered or whether the labels are consecutive, until a non-list item is encountered, at which point the extraction is interrupted, resulting in a complex list.

[0118] 3) Perform completely independent list extraction on complex lists and determine the list type based on the number of extracted lists.

[0119] In step 3) above, determining the list type based on the number of extracted lists includes:

[0120] a) Calculate the number of lists extracted.

[0121] b) Determine if the list count is greater than or equal to 1. If the list count is equal to 1, mark it as an independent list. If the list count is greater than 1, mark it as a nested list.

[0122] The logic of compound extraction that preserves nested list structures works correctly only if the lists in the original text are organized correctly and in a standardized manner. When there are anomalies in the lists, such as missing items, inappropriately inserted lines in nested lists, or line break anomalies, it may not work correctly. In such cases, it is necessary to integrate list missing item detection and list structure integrity detection, and then correct the errors.

[0123] The multimodal nested text list extraction method of this invention, which does not rely on indentation detection, has several advantages. Firstly, list extraction no longer depends on visual features or additional attribute features inherent in specific text formats; it extracts only based on the core symbolic features of the text list. Compared to similar machine learning techniques, this invention replaces costly machine learning feature engineering with simple manual cleaning. Secondly, this invention recognizes multiple labeling patterns in ordered and unordered lists and effectively handles easily confused items between English and Roman numeral labels. Furthermore, this invention implements two detection functions: missing item detection and list structure integrity detection. In addition, this invention implements configurable missing item tolerance extraction and inter-item row insertion skipping functions to address various possible errors. It also implements three selectable extraction strategies: completely independent list extraction, composite extraction that preserves the nested list structure, and a composite list extraction function with a configurable fault-tolerant mechanism that integrates error detection and the first two extraction functions to ensure the integrity and correctness of the extracted data.

[0124] Please continue to refer to Figure 1 , Figure 7 and Figure 8 The following section will elaborate on each step of the multimodal nested text list extraction method that does not rely on indentation detection:

[0125] In step 110, the text to be extracted, which contains a list, is subjected to list missing item detection. List missing item detection includes first item missing detection and middle item missing detection. The missing item detection function will serve as a prerequisite for the missing item-tolerant extraction function, which will trigger different extraction strategies when a missing item is detected.

[0126] The missing item error forms mentioned in the embodiments of this invention are as follows:

[0127] For a single list, a missing item refers to the absence of one or more consecutive or non-consecutive labeled items in the ordered list. For example, if the original list labels were a), b), c), d), e), f), and for some reason the labels changed to b), c), d), e), f) or a), b), c), f), f), the original list labels were abruptly changed to ab), b), c), f), the original list labels were abruptly changed to abruptly changed list labels between adjacent rows within a single ordered list, the cause of the error might be:

[0128] First, the list items are correct, only the labeling is incorrect;

[0129] Second, both the list labels and the content of the corresponding items are lost.

[0130] Third, formatting errors, such as incorrect line breaks, cause the list number and list item content of the next line to "drift" to the end of the previous item (line) content.

[0131] For the completely independent list extraction function, the logical starting point for extracting an ordered list is when the first item of the list is encountered from bottom to top during scanning. If the first item of the list is missing (also known as a headless list), confirming the starting point for extraction cannot be achieved. This may lead to extraction failure or cause unknown interference to the extraction of other lists in the same text, especially if other lists in the document also have formatting errors. The missing items in the middle of an ordered list are relatively safe to handle due to the "enclosure and isolation" of the first and last items, requiring only a looping mechanism specific to the current list format. The last item and the first item are both list boundary items in position, but the difference is that the first item has a definite identifiable label, such as 1, a, i, while the last item does not have this feature; therefore, the missing last item cannot be confirmed.

[0132] In summary, missing first terms and missing middle terms can be detected using different methods.

[0133] To prevent excessive functional overlap between first-item missing detection and middle-item missing detection, and to avoid the risk of overly tolerating missing item errors (such as the risk of missing the first n items), the first-item missing detection function is predicated on the existence of the second item in the list. This limits the error range to cases where the second item exists but the first item does not. The detection focuses on the entire text paragraph used for list extraction; specifically, it checks if a text paragraph contains a list with a missing first item. Since a text paragraph may contain only one list, this function also supports first-item missing detection for individual lists. A key aspect of the detection process is considering nested lists between the first and second items. To address this, a scan counting rule based on a quantity balancing mechanism is implemented: the balance count is incremented by 1 when the second item of the list is encountered, and decremented by 1 when the first item of the same label type is encountered. If the balance count is not zero after scanning upwards, it indicates that a list with a missing first item exists. Please refer to [reference needed]. Figure 13 The implementation steps for the first missing item detection are as follows:

[0134] (1) Given a complete text paragraph, locate the first line of the text paragraph and scan the text lines from top to bottom.

[0135] (2) Determine whether the current list row being scanned is the second item of an ordered list (Roman or non-Roman label type). If it is not the second item of an ordered list, skip it.

[0136] (3) Based on the fact that the current list row is the second item of the ordered list, set the initial balance value to 1, obtain the list label type of the current row, and then start scanning upwards row by row.

[0137] (4) When the reverse scan encounters the second item of the same label type, the balance value is increased by 1; if the first item is encountered, it is decreased by 1.

[0138] (5) After scanning all rows in reverse, check if the balance value is 0. If the balance value is 0, there is no missing first item in the list. If the balance value is not 0, there is a missing first item in the list.

[0139] The missing middle item detection function also detects the entire text paragraph, while also being compatible with cases where the text paragraph contains only one list. Its design and implementation assume that the list does not have a missing first item. The detection mechanism no longer relies on balanced counting rules, but instead combines an existing completely independent list extraction module with a composite extraction module that preserves the nested list structure. The implementation steps for missing middle item detection are as follows:

[0140] (1) Given a complete text paragraph, locate the first line of the text paragraph and perform a completely independent list extraction on it (skipping items is not allowed, i.e. the second threshold is 0) to obtain the list set A.

[0141] (2) Perform compound extraction that preserves the nested list structure on the remaining text paragraphs to be extracted after the completely independent list extraction to obtain list set B.

[0142] (3) Calculate the number of list items in list set A and list set B respectively, and obtain CA and CB accordingly.

[0143] (4) Compare and confirm whether the values ​​of CA and CB are equal. If they are equal, there is no missing term. If they are not equal, there is a missing term.

[0144] The key logic in implementing the missing intermediate item detection function is that the composite extraction function, which preserves the nested list structure, may incorrectly assign list items to different lists without any fault tolerance configuration. However, no items will be missed during the extraction process, thus ensuring a correct count of the total number of list items across all lists in a text. In contrast, the completely independent list extraction function, which does not allow skipping items (explained later), will lose subsequent items due to missing intermediate items. Therefore, when missing intermediate items exist, the total number of list items extracted from all lists will be less than the actual total number of all list items.

[0145] Please refer to Figure 12 The combination of the first missing item detection and the middle missing item detection functions mentioned above can comprehensively determine whether a text paragraph has missing items. The judgment logic is as follows:

[0146] (1) Given a text paragraph, perform a first-item missing detection on it. If the first item is missing, then the text paragraph has missing items.

[0147] (2) If there is no missing first item, then perform a missing middle item detection. If there is a missing middle item, then the text paragraph has a missing item.

[0148] (3) There are no missing items in other cases.

[0149] Based on the above, this embodiment of the invention also designs an independent list extraction function with a configurable missing item tolerance mechanism, so that when there are missing items in a single list, all relevant list items can be extracted correctly. At the same time, it implements the function of configuring the threshold for the number of missing items, that is, it can specify the maximum number of consecutive missing list items that can be allowed before the extraction starts.

[0150] The key design element of the missing item tolerance mechanism lies in the top-down scanning process. At each current list row position, the next list row is simultaneously obtained. Based on a given threshold, it is determined whether the label type of the next list row is correct and whether the label value is within the threshold range. When the threshold is set to 0, skipping items is not allowed; that is, the labels must be adjacent in value (or letter). For skipped items exceeding the threshold, the extraction of the current list will end. This missing item tolerance mechanism is embedded in the sub-functional modules of Roman numeral type and non-Roman ordered list type lists.

[0151] In addition, the missing first item should also be incorporated into the missing item tolerance extraction mechanism. In the extraction process of all types of ordered lists, when scanning the second item of an ordered list from bottom to top, if it is determined at the same time that the current list has no first item, and the missing item tolerance threshold is set to be greater than 0, then the extraction of a list should also begin.

[0152] The missing item tolerance mechanism is not only a fault tolerance mechanism for formatting errors, but also a content integrity protection mechanism. First, it protects against the loss of subsequent items caused by premature termination of extraction due to missing items. Second, it protects against content loss caused by missing line breaks between list rows.

[0153] In step 120, the list structure integrity detection is performed on the text to be extracted containing a list, and a first threshold for missing item tolerance in the structure integrity detection is configured.

[0154] Structural integrity detection primarily targets nested lists. Inserting lines of non-list items before the first item of a sub-nested list disrupts the nested structure. This can be caused by issues during file format conversion, such as leading to the insertion of lines between items, or by inappropriate information organization or improper formatting when creating the list. There are two typical scenarios where the nested structure is disrupted: one is when the last item of the parent list is followed by a sub-nested list. Please refer to... Figure 4 Another scenario is where a middle item in the parent list is followed by a nested sublist; please refer to [reference needed]. Figure 5 .

[0155] Inter-item insertion refers to irrelevant or interfering content inserted between consecutive list items for various reasons. Typical causes of this error include list items spanning multiple pages, content in the header or footer of a Word or PDF file being embedded between list items when converting it to plain text, or footnotes at the bottom of the page. When this occurs, the normal list extraction process will be unexpectedly interrupted. Furthermore, when this situation occurs simultaneously with missing list items, it will cause even greater interference to list extraction. The implementation of the inter-item insertion skipping function relies on a top-down loop skipping mechanism, the main implementation steps of which are as follows:

[0156] 1) During the extraction of an ordered list, the next row is obtained at the same time as each row of the list is scanned.

[0157] 2) If the next row is not an adjacent item within the missing item tolerance threshold, then continue to perform the same judgment on the next row and repeat this step.

[0158] 3) Continue the extraction process until a suitable next neighboring item is encountered and stored as a list item, and the current row is updated to this item.

[0159] 4) When all rows have been scanned downwards and no next item matching the requirements is found, the extraction process of the current list is completed.

[0160] The aforementioned inter-item row insertion mechanism is embedded in all ordered list extraction submodules. Furthermore, a key technical detail lies in the top-level logic of Roman numeral extraction. When determining whether an item containing 'i' or 'I' is the first item of a Roman numeral list, it's necessary to simultaneously determine whether the next item contains 'ii' or 'II'. This determination step also requires an inter-item row insertion skipping mechanism to ensure that the first item determination mechanism fails when a distracting row is inserted between the first and second Roman numeral items.

[0161] When a parent list's last item is followed by a nested sublist, if the original two lists are indeed nested, the inserted row makes it impossible to determine from the format alone whether the original two lists are nested or independent lists. When a parent list's middle item is followed by a nested sublist, the one or more lines inserted before the first item of the nested sublist can interfere with the extraction process and easily cause information structure confusion during table reading.

[0162] The steps for structural integrity detection are as follows:

[0163] (1) Perform completely independent list extraction on the text to be extracted to obtain all independent lists in the text to be extracted. Calculate the list items of each independent list and sum them to obtain the total list item E.

[0164] (2) Perform compound extraction on the text to be extracted, preserving the nested list structure, to obtain a nested list and an independent list;

[0165] (3) Perform completely independent list extraction on each of the obtained nested lists to obtain independent list two;

[0166] (4) Calculate the list items in Independent List 1 and Independent List 2 respectively and sum them to obtain the total list item sum F;

[0167] (5) Determine whether E and F are equal. If they are equal, there is no inserted row in the list that may damage the structural integrity. If they are not equal, there is an inserted row that may damage the structural integrity.

[0168] The principle behind the structural integrity detection method described above is that the list extraction proceeds from top to bottom, collecting adjacent list item rows. When a non-list row is encountered, it checks whether the label type and value of the nearest next list item are connected to the previous list item. If they are, the list extraction continues; otherwise, it completes. Under this mechanism, if a row is inserted before the first item of a nested list, the extraction of the nested list cannot continue because no item in the parent list is connected to the first item of another list.

[0169] For example Figure 5In the example above, if different extraction methods are applied to count list items according to the above logic, using the completely independent list extraction function alone will yield 5 list items because it can obtain two complete lists. However, using the second combined extraction method, the first round will extract items a) and b) into an independent list, and items i, ii, and c) into a nested list. Then, performing completely independent list extraction on these three items will only yield items i and ii as independent lists, while item c) will be lost. Thus, the first counting method will yield 5 list items, while the second method will only yield 4. The difference indicates that the text may have a broken list structure.

[0170] The core of this detection mechanism lies in the fact that after the nested list extraction is disrupted, the secondary extraction of the completely independent list will result in the loss of some items, thus causing the total number of items to differ between the two extraction methods. It is important to note that the list integrity detection mechanism relies on a missing item tolerance function when both missing items and structurally disruptive inserted rows exist in the list. This is because, when a list is missing items, the completely independent list extracted from a nested list twice will yield the same two sets of independent lists as the one obtained by directly extracting the completely independent list from the relevant rows.

[0171] For example Figure 6 In the example, item d of the first-level list is missing, and there is a line of structural disruption before the first item of the nested sublist. Without setting a missing item tolerance threshold, directly using fully independent list extraction will yield 6 items: a., b., c., and i., ii., iii., . Using a composite extraction that preserves the nested list structure followed by fully independent list extraction will first yield a., b., c., d. as one group and i. to g. as another group. Then, from the first group, we obtain a., b., c., 3 items as one independent list, and from the second group, we obtain i., ii., iii., 3 items as yet another independent list, also totaling 6 items. Ultimately, this list will be judged as having no structural disruption because both processing methods yield the same number of items. However, if the missing item tolerance threshold is set to 1, we can obtain 6 + 3 = 9 items from the first processing method and 4 + 3 = 7 items from the second processing method, resulting in a potentially disrupted list structure.

[0172] In cases where there are no missing items, configuring missing item tolerance will not affect the list integrity detection function. Therefore, a default missing item tolerance threshold, such as 1 or 2, can be set as needed. This support setting does not overlap or conflict with the missing item detection or missing item tolerance extraction function.

[0173] In step 130, based on the list missing item detection results and the list structure integrity detection results, either the first extraction method or the second extraction method is selected to perform list extraction on the text to be extracted.

[0174] Based on the detection results that the list has no missing items and the list structure is complete, the first extraction method is selected to perform list extraction on the text to be extracted. The first extraction method includes: performing compound extraction on the text to be extracted while preserving the nested list structure, obtaining nested lists and independent lists, and completing the list extraction.

[0175] Based on the detection results of missing items or incomplete list structure, the second extraction method is selected to perform list extraction on the text to be extracted. The second extraction method includes: performing completely independent list extraction on the text to be extracted, extracting independent lists, then excluding the extracted list items from the original text to be extracted, and merging the remaining lines; performing compound extraction that preserves the nested list structure on the merged remaining lines.

[0176] Please refer to Figure 14 The present invention also provides a multimodal nested text list extraction device 200 that does not rely on indentation detection. The multimodal nested text list extraction device 200 that does not rely on indentation detection includes at least one software function module that can be stored in a storage module or embedded in an operating system (OS) in the form of software or firmware.

[0177] A multimodal nested text list extraction device 200 that does not rely on indentation detection includes:

[0178] The missing item detection module 210 is used to perform list missing item detection on the text to be extracted containing a list. The list missing item detection includes first item missing detection and middle item missing detection.

[0179] The integrity detection module 220 is used to perform list structure integrity detection on the text to be extracted containing a list, and to configure the first threshold for missing item tolerance in the structure integrity detection.

[0180] The extraction method selection module 230 is used to select either a first extraction method or a second extraction method to perform list extraction on the text to be extracted based on the list missing item detection results and the list structure integrity detection results. Both the first and second extraction methods include a composite extraction method that preserves the nested list structure and a completely independent list extraction method. The completely independent list extraction method is used to extract a complete independent list of a single label of the same type from the text paragraph. The completely independent list extraction method is configured with a second threshold for missing item tolerance. The composite extraction method that preserves the nested list structure is used to extract both nested lists and independent lists from the text paragraph at the same time.

[0181] It should be noted that the missing item detection module 210, the integrity detection module 220, and the extraction method selection module 230 correspond to steps 110 to 130 in the above embodiment of the multimodal nested text list extraction method that does not rely on indentation detection. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the content disclosed in the above embodiment of the multimodal nested text list extraction method that does not rely on indentation detection. Furthermore, the above modules and other operations and / or functions in the multimodal nested text list extraction device 200 that does not rely on indentation detection are respectively for implementing the corresponding process of the method. For the sake of brevity, they will not be described in detail here.

[0182] This application also discloses an electronic device, which includes a processor and a memory. The memory stores at least one program, which is loaded and executed by the processor to implement the above-described method for extracting multimodal nested text lists without relying on indentation detection.

[0183] It is understood that, in order to achieve the above-mentioned functions, electronic devices include hardware and / or software modules that perform the respective functions. Based on the algorithm steps of the examples described in the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0184] The electronic device provided in this application embodiment can be user equipment (UE), such as a mobile terminal (e.g., a user's mobile phone), tablet computer, desktop computer, laptop computer, handheld computer, netbook, personal digital assistant (PDA), etc.

[0185] This application also discloses a computer-readable storage medium containing at least one computer program configured to execute at runtime to implement the above-described method for extracting multimodal nested text lists without relying on indentation detection.

[0186] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications and substitutions should be covered within the scope of the claims of the present invention. Technical aspects, shapes, and structures not described in detail in this invention are all well-known technologies.

Claims

1. A method for extracting multimodal nested text lists without relying on indentation detection, characterized in that, Includes the following steps: The text to be extracted, which contains a list, is subjected to list missing item detection, which includes first item missing detection and middle item missing detection; Perform list structure integrity detection on the text to be extracted containing lists, and configure a first threshold for missing item tolerance in the structure integrity detection; Based on the list missing item detection results and the list structure integrity detection results, a first extraction method or a second extraction method is selected to perform list extraction on the text to be extracted; both the first extraction method and the second extraction method contain a composite extraction of completely independent list extraction and nested list structure preservation. The completely independent list extraction is used to extract a complete independent list of a single label of the same type from the text paragraph. The completely independent list extraction is configured with a second threshold for missing item tolerance. The composite extraction of nested list structure preservation is used to extract nested lists and independent lists from the text paragraph at the same time. Based on the detection results showing that the list has no missing items and a complete list structure, the first extraction method is selected to perform list extraction on the text to be extracted. The first extraction method includes: Perform compound extraction on the text to be extracted, preserving the nested list structure, to obtain nested lists and independent lists, thus completing the list extraction; Based on the detection results indicating missing items or incomplete list structure, the second extraction method is selected to perform list extraction on the text to be extracted. The second extraction method includes: Perform completely independent list extraction on the text to be extracted, extract the independent lists, then exclude the extracted list items from the original text to be extracted, and merge the remaining lines; perform compound extraction that preserves the nested list structure on the merged remaining lines; The first missing item detection includes: The process begins by scanning the text to be extracted from top to bottom in a forward direction. It then determines whether the current row is the second item in an ordered list. Based on this result, an initial balance value of 1 is set, and the list label type of the current row is obtained. The process then begins scanning upwards row by row in reverse. When a second item with the same label type is encountered during the reverse scan, the balance value is incremented by 1; if a first item with the same label is encountered, the balance value is decremented by 1. After scanning all rows in reverse, the balance value is checked to see if it is 0. If the balance value is 0, there is no missing first item in the list; if the balance value is not 0, there is a missing first item in the list. The intermediate term missing detection is performed under the condition that there is no first term missing, and the intermediate term missing detection includes: Perform completely independent list extraction on the text to be extracted to obtain list set A. In this step, the second threshold is 0. Perform compound extraction that preserves the nested list structure on the remaining text paragraphs to be extracted after completely independent list extraction to obtain list set B. Calculate the number of list items in list set A and list set B respectively to obtain CA and CB. Compare and confirm whether the values ​​of CA and CB are equal. If they are equal, there are no missing items. If they are not equal, there are missing items. The step of performing list structure integrity detection on the text to be extracted containing lists, and configuring a first threshold for missing item tolerance in the structure integrity detection, includes: Perform fully independent list extraction on the text to be extracted to obtain all independent lists in the text. Calculate the list items of each independent list and sum them to obtain the total list item sum E. Perform compound extraction on the text to be extracted while preserving the nested list structure to obtain a nested list and independent list one. Perform fully independent list extraction on each of the obtained nested lists to obtain independent list two. Calculate the list items in independent list one and independent list two and sum them to obtain the total list item sum F. Determine whether E and F are equal. If they are equal, there are no inserted rows in the list that may compromise the structural integrity. If they are not equal, there are inserted rows that may compromise the structural integrity.

2. The multimodal nested text list extraction method independent of indentation detection according to claim 1, characterized in that, The completely independent list extraction includes ordered list extraction and unordered list extraction. The ordered list extraction includes: Locate the last line of the text to be extracted, and scan the text backwards from the last line. During the scanning process, the first item is identified, and the forward scanning begins. It is determined whether the label type of the next row is the same as that of the current row and whether the label value is within the second threshold range. If the label type of the next row is the same as that of the current row and the label value is within the second threshold range, extraction is performed. If the label type of the next row is different from that of the current row or the label value is not within the second threshold range, extraction is interrupted, thus completing the extraction of an independent list. Remove all list items from the extracted independent lists from the text to be extracted, and merge the remaining lines; The remaining text lines are then scanned and extracted again in reverse. The extraction of the unordered list includes: Locate the last line of the text to be extracted, and scan the text backwards from the last line. Retrieve the label type of list items during scanning; Determine if the label type of the current row is the same as the label type of the adjacent previous row. If they are the same, extract the label; otherwise, continue scanning in reverse.

3. The multimodal nested text list extraction method independent of indentation detection according to claim 2, characterized in that, The identification of the first term includes the identification of Roman numeral first terms and the differentiation of easily confused symbols. The identification of Roman numeral first terms includes: During the scanning process, when the label of the current row is identified as i or I, the next row is scanned downwards in a positive direction and the label of the next row list is obtained. It is then determined whether the label of the next row list is ii or II. If it is, the current row is a list of Roman numeral first items; otherwise, the current row is not a list of Roman numeral first items. The distinction and recognition of easily confused labels is used to identify Roman numeral and non-Roman numeral neighboring items in ordered list extraction.

4. The multimodal nested text list extraction method independent of indentation detection according to claim 3, characterized in that, The composite extraction that preserves the nested list structure includes: Locate the first line of the text to be extracted, and scan the text downwards in a forward direction starting from the first line; Starting from the first list item, continuously extract adjacent list items until a non-list item is encountered, at which point the extraction stops, resulting in a complex list. Perform completely independent list extraction on complex lists, and determine the list type based on the number of extracted lists.

5. The multimodal nested text list extraction method independent of indentation detection according to claim 4, characterized in that, The method of determining the list type based on the number of extracted lists includes: Calculate the number of extracted lists; Check if the list count is greater than or equal to 1. If the list count is equal to 1, mark it as an independent list. If the list count is greater than 1, mark it as a nested list.

6. A multimodal nested text list extraction device that does not rely on indentation detection, characterized in that, include: The missing item detection module is used to perform list missing item detection on the text to be extracted containing a list. The list missing item detection includes first item missing detection and middle item missing detection. The integrity detection module is used to perform list structure integrity detection on the text to be extracted containing a list, and to configure the first threshold for missing item tolerance in the structure integrity detection. The extraction method selection module is used to select either a first extraction method or a second extraction method to perform list extraction on the text to be extracted based on the list missing item detection results and the list structure integrity detection results. Both the first and second extraction methods include a composite extraction method that preserves the nested list structure and a completely independent list extraction method. The completely independent list extraction method is used to extract a complete independent list of a single label of the same type from the text paragraph. The completely independent list extraction method is configured with a second threshold for missing item tolerance. The composite extraction method that preserves the nested list structure is used to extract both nested lists and independent lists from the text paragraph simultaneously. Based on the detection results showing that the list has no missing items and a complete list structure, the first extraction method is selected to perform list extraction on the text to be extracted. The first extraction method includes: Perform compound extraction on the text to be extracted, preserving the nested list structure, to obtain nested lists and independent lists, thus completing the list extraction; Based on the detection results indicating missing items or incomplete list structure, the second extraction method is selected to perform list extraction on the text to be extracted. The second extraction method includes: Perform completely independent list extraction on the text to be extracted, extract the independent lists, then exclude the extracted list items from the original text to be extracted, and merge the remaining lines; perform compound extraction that preserves the nested list structure on the merged remaining lines; The first missing item detection includes: The process begins by scanning the text to be extracted from top to bottom in a forward direction. It then determines whether the current row is the second item in an ordered list. Based on this result, an initial balance value of 1 is set, and the list label type of the current row is obtained. The process then begins scanning upwards row by row in reverse. When a second item with the same label type is encountered during the reverse scan, the balance value is incremented by 1; if a first item with the same label is encountered, the balance value is decremented by 1. After scanning all rows in reverse, the balance value is checked to see if it is 0. If the balance value is 0, there is no missing first item in the list; if the balance value is not 0, there is a missing first item in the list. The intermediate term missing detection is performed under the condition that there is no first term missing, and the intermediate term missing detection includes: Perform completely independent list extraction on the text to be extracted to obtain list set A. In this step, the second threshold is 0. Perform compound extraction that preserves the nested list structure on the remaining text paragraphs to be extracted after completely independent list extraction to obtain list set B. Calculate the number of list items in list set A and list set B respectively to obtain CA and CB. Compare and confirm whether the values ​​of CA and CB are equal. If they are equal, there are no missing items. If they are not equal, there are missing items. The step of performing list structure integrity detection on the text to be extracted containing lists, and configuring a first threshold for missing item tolerance in the structure integrity detection, includes: Perform fully independent list extraction on the text to be extracted to obtain all independent lists in the text. Calculate the list items of each independent list and sum them to obtain the total list item sum E. Perform compound extraction on the text to be extracted while preserving the nested list structure to obtain a nested list and independent list one. Perform fully independent list extraction on each of the obtained nested lists to obtain independent list two. Calculate the list items in independent list one and independent list two and sum them to obtain the total list item sum F. Determine whether E and F are equal. If they are equal, there are no inserted rows in the list that may compromise the structural integrity. If they are not equal, there are inserted rows that may compromise the structural integrity.

Images