An adaptive multi-level document cutting method

Abstract

The invention relates to the technical field of document cutting, and discloses an adaptive multi-level document cutting method, which comprises the following steps: parsing an original document, determining a document to be cut based on a parsing result; obtaining document information of the document to be cut, performing a first cutting, and obtaining a plurality of cut blocks; determining a comprehensive index value corresponding to each cut block, and fusing the cut blocks that meet the conditions based on a numerical value sorting result, and obtaining a plurality of fused blocks; collecting the number of cut blocks in each fused block, and calculating a stratification guiding factor; stratifying the fused blocks according to the stratification guiding factor, and obtaining a plurality of hierarchical intervals, and judging whether the hierarchical interval needs to be adjusted; if so, setting an adjustment coefficient based on the fused block, and adjusting the hierarchical interval, and taking the adjusted hierarchical interval as a target cutting document, thereby realizing intelligent cutting of the document, improving the efficiency of document cutting, and ensuring the accuracy and adaptability of cutting, and avoiding the phenomenon of incomplete context information.

Description

Technical Field

[0001] The present invention relates to the technical field of document cutting, and in particular to an adaptive multi-level document cutting method. Background Art

[0002] With the rapid development of information technology, a large amount of text data has been digitized, forming a huge data set. The documents in these data sets often have complex structures and levels and contain rich information. In order to effectively use this information, it is necessary to effectively cut and organize the documents.

[0003] Traditional document segmentation methods often rely on manually set rules or simple algorithms. These methods have limitations when processing complex documents and cannot adapt to the diversity of document content and the complexity of structure. In addition, when traditional document segmentation methods are used for long texts, key contextual information may be lost during the conversion process, and there is no complete contextual information. Summary of the invention

[0004] The embodiment of the present invention provides an adaptive multi-level document cutting method, which can realize intelligent cutting of documents, not only improve the document cutting efficiency, but also ensure the accuracy and adaptability of cutting.

[0005] In order to achieve the above object, the present invention provides an adaptive multi-level document cutting method, comprising:

[0006] Receiving an original document, parsing the original document, determining whether the original document meets the cutting condition, and if so, determining that the original document is a document to be cut;

[0007] Acquiring document information of the document to be cut, and performing a first cutting according to the document information to obtain a plurality of cut blocks;

[0008] Determine the comprehensive index value corresponding to each cutting block, sort all the comprehensive index values ​​by numerical value, and fuse the cutting blocks that meet the conditions based on the sorting results to obtain multiple fused blocks;

[0009] Collect the number of cut blocks in each fusion block, and calculate the hierarchical guidance factor of the corresponding fusion block according to the number of cut blocks;

[0010] The fusion block is layered according to all the layering guiding factors to obtain a plurality of layer intervals, and whether the layer interval needs to be adjusted is determined based on the layering result. When it is determined that the layer interval does not need to be adjusted, the layer interval is used as a target document for segmentation;

[0011] When it is determined that the hierarchical interval needs to be adjusted, an adjustment coefficient is set based on the fusion block, and the hierarchical interval is adjusted according to the adjustment coefficient, and the adjusted hierarchical interval is used as a target segmentation document.

[0012] Furthermore, when receiving an original document, parsing the original document to determine whether the original document meets the cutting condition, and if so, determining that the original document is a document to be cut includes:

[0013] Determining whether the original document is an accessed document based on the identification code carried by the original document, wherein the accessed document is stored in a document database, and the document in the document database is a document that has passed security detection;

[0014] When the original document is a document that has been accessed, determining whether the original document is credible;

[0015] If the original document is credible, determining that the original document meets the cutting condition;

[0016] When the original document is a file that has not been accessed, a query read operation is performed and a security check is performed.

[0017] Furthermore, when the document information of the document to be cut is obtained and the first cutting is performed according to the document information to obtain a plurality of cut blocks, it includes:

[0018] Analyzing the document to be cut based on preset cutting points to determine a plurality of cutting units, wherein each cutting unit is a complete sentence;

[0019] Parse each cutting unit to determine the document content and document structure corresponding to the cutting unit, analyze the document content based on a pre-trained document content index value model, and output the corresponding document content index value, and analyze the document structure based on a pre-trained document structure index value model, and output the corresponding document structure index value;

[0020] Calculate a comprehensive index value of a cutting unit based on the document content index value and the document architecture index value;

[0021] P = β1 × (1 + e1) e2 +β2×(1+e2) e1 ;

[0022] Wherein, P is the comprehensive index value of the cutting unit, β1 is the first calculation coefficient, e1 is the document content index value, β2 is the second calculation coefficient, and e2 is the document architecture index value;

[0023] All cutting units are bound in pairs based on the cutting order. If there are unbound cutting units, the unbound cutting units are used as a cutting block.

[0024] Calculate the comprehensive index difference between every two bound cutting units to obtain a preset comprehensive index difference;

[0025] Extract all comprehensive index difference values ​​that are less than or equal to the preset comprehensive index difference value and use them as a cutting block respectively;

[0026] Unbinding all comprehensive indicator difference values ​​greater than the preset comprehensive indicator difference value to obtain a comprehensive indicator difference value series;

[0027] All cutting units corresponding to the comprehensive index difference series are used as cutting blocks.

[0028] Further, when analyzing the document content based on a pre-trained document content index value model and outputting a corresponding document content index value, and analyzing the document architecture based on a pre-trained document architecture index value model and outputting a corresponding document architecture index value, it includes:

[0029] Performing word segmentation processing on the document content to obtain split words of the document content;

[0030] Performing a first annotation on each split word based on a preset part of speech, performing word sense matching on the document content according to the first annotation, and obtaining a word sense segmentation set of the document content;

[0031] Processing the document content set based on the document content index value model to obtain the document content index value;

[0032] Splitting the document structure to obtain a split layout of the document structure;

[0033] Performing a second annotation on each split layout based on the preset layout, performing layout matching on the document structure according to the second annotation, and obtaining a split layout set of the document structure;

[0034] The split layout set is processed based on the document architecture index value model to obtain the document architecture index value.

[0035] Furthermore, when the comprehensive index value corresponding to each cutting block is determined, and all the comprehensive index values ​​are sorted by numerical value, the cutting blocks that meet the conditions are merged based on the sorting result to obtain multiple fused blocks, including:

[0036] Determine a comprehensive indicator value set based on the sorting result, fuse the cutting blocks corresponding to the first comprehensive indicator value and the second comprehensive indicator value in the comprehensive indicator value set to obtain a first fused block, and use the cutting block corresponding to the third comprehensive indicator value in the comprehensive indicator value set as the cutting block to be updated;

[0037] Obtain a second comprehensive index difference between the cut block to be updated and the cut block corresponding to the second comprehensive index value, and obtain a preset second preset comprehensive index difference;

[0038] Determine whether the second comprehensive index difference is less than or equal to the second preset comprehensive index difference; if so, add the cut block to be updated to the first fusion block to update the first fusion block; if not, use the cut block to be updated as the second fusion block;

[0039] Taking the cutting block corresponding to the fourth comprehensive index value in the comprehensive index value set as the next cutting block to be updated;

[0040] Obtaining a third comprehensive index difference between the next cutting block to be updated and the cutting block corresponding to the third comprehensive index value in the second fusion block or the first fusion block, and determining whether the third comprehensive index difference is less than or equal to the second preset comprehensive index difference; if so, adding the next cutting block to be updated to the second fusion block or the first fusion block; if not, using the next cutting block to be updated as the second fusion block or the third fusion block;

[0041] In this way, updates and iterations are performed to divide the comprehensive indicator value set into multiple fusion blocks.

[0042] Further, when collecting the number of cut blocks in each fusion block and calculating the hierarchical guidance factor of the corresponding fusion block according to the number of cut blocks, it includes:

[0043] A first weight is configured for the fusion block, and a second weight is configured for each cut block;

[0044] The hierarchical guidance factor of the corresponding fusion block is calculated according to the following formula:

[0045]

[0046] Among them, G is the hierarchical guidance factor, D is the first weight, n is the number of cut blocks in the fusion block, and f i is the comprehensive index value of the i-th cutting block, h i is the second weight of the i-th cutting block.

[0047] Further, when the fusion block is layered according to all the layering guiding factors to obtain a plurality of layer intervals, and whether the layer interval needs to be adjusted is determined based on the layering result, it includes:

[0048] Presetting a boundary range for each hierarchical interval, wherein the boundary range includes a left hierarchical guidance factor boundary value and a right hierarchical guidance factor boundary value;

[0049] Traversing all hierarchical guidance factors in the boundary range to determine corresponding hierarchical intervals, wherein the first layer of the hierarchical interval includes the smallest hierarchical guidance factor;

[0050] Determine the height of the hierarchical interval and the number of fusion blocks in each hierarchical interval, wherein the height of the hierarchical interval refers to the number of hierarchical intervals;

[0051] Obtaining a preset height and a preset number, and when the height is less than or equal to the preset height, determining that the level interval does not need to be adjusted;

[0052] When the height is greater than the preset height, a suspected adjustment mark is generated, and it is determined whether the number is less than or equal to the preset number. If so, it is determined that the level interval does not need to be adjusted;

[0053] If not, it is determined that the level interval needs to be adjusted.

[0054] Further, when it is determined that the hierarchical interval needs to be adjusted, an adjustment coefficient is set based on the fusion block, and the hierarchical interval is adjusted according to the adjustment coefficient, including:

[0055] Calculate the hierarchical guidance factors and values ​​within each hierarchical interval, extract them, and construct a hierarchical guidance factor and value set;

[0056] Sorting the stratified guiding factors and value sets by numerical value and determining the median;

[0057] Generate a first sub-stratification guidance factor and value set according to the stratification guidance factor and value between the head stratification guidance factor and value and the median in the stratification guidance factor and value set;

[0058] Generate a second sub-stratification guidance factor and value set according to the stratification guidance factor and value between the tail stratification guidance factor and value in the stratification guidance factor and value set and the median, wherein the first sub-stratification guidance factor and value set and the second sub-stratification guidance factor and value set do not include the median;

[0059] Calculating an adjustment coefficient based on the first sub-layer guidance factor and value set and the second sub-layer guidance factor and value;

[0060] Calculate the product value of the adjustment coefficient and all boundary ranges, and obtain a new boundary range;

[0061] All hierarchical guidance factors are traversed twice within the new boundary range to determine the adjusted hierarchical interval.

[0062] Further, when calculating the adjustment coefficient according to the first sub-layer guiding factor and value set and the second sub-layer guiding factor and value, it includes:

[0063] Calculate the adjustment factor according to the following formula:

[0064]

[0065] Where t is the adjustment coefficient, m1 is the number of stratified guiding factors and values ​​in the first sub-stratified guiding factor and value set, q is the median, and r j is the jth hierarchical guidance factor and value in the first sub-hierarchical guidance factor and value set, m2 is the number of hierarchical guidance factors and values ​​in the second sub-hierarchical guidance factor and value set, v w is the wth hierarchical guidance factor and value in the second sub-hierarchical guidance factor and value set.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] The original document is parsed, and the document to be cut is determined based on the parsing result; the document information of the document to be cut is obtained, and the first cutting is performed to obtain multiple cut blocks; the comprehensive index value corresponding to each cut block is determined, and the cut blocks that meet the conditions are merged based on the numerical value sorting result to obtain multiple fused blocks; the number of cut blocks in each fused block is collected, and the stratification guidance factor is calculated; the fused blocks are stratified according to the stratification guidance factor to obtain multiple hierarchical intervals, and it is determined whether the hierarchical interval needs to be adjusted. If so, an adjustment coefficient is set based on the fused block, and the hierarchical interval is adjusted. The adjusted hierarchical interval is used as the target cutting document, which can realize intelligent cutting of documents, improve the efficiency of document cutting, and ensure the accuracy and adaptability of cutting, and avoid the phenomenon of incomplete context information. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0069] Figure 1 A schematic flow chart of an adaptive multi-level document cutting method in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0070] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0071] In the description of the present application, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0072] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0073] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0074] The following is a description of preferred embodiments of the present invention with reference to the accompanying drawings.

[0075] like Figure 1 As shown, an embodiment of the present invention discloses an adaptive multi-level document cutting method, comprising:

[0076] S110: receiving an original document, parsing the original document, and determining whether the original document meets the cutting condition; if so, determining that the original document is a document to be cut;

[0077] In some embodiments of the present application, when receiving an original document, parsing the original document, determining whether the original document meets the cutting condition, and if so, determining that the original document is a document to be cut, includes:

[0078] Determining whether the original document is an accessed document based on the identification code carried by the original document, wherein the accessed document is stored in a document database, and the document in the document database is a document that has passed security detection;

[0079] When the original document is a document that has been accessed, determining whether the original document is credible;

[0080] If the original document is credible, determining that the original document meets the cutting condition;

[0081] When the original document is a file that has not been accessed, a query read operation is performed and a security check is performed.

[0082] In this embodiment, the identification code of each document is unique, which can effectively identify the document and avoid repeated processing. Through the identification code, the document can be quickly retrieved in the document database to improve work efficiency. In addition, the identification code helps record the security detection status of the document to ensure that all accessed documents meet security requirements.

[0083] In this embodiment, in order to ensure that all documents do not contain malicious code or content and protect the security of the system and users, this application performs security testing on the documents.

[0084] In this embodiment, whether the original document is credible can be determined based on the following methods: document source, content quality, whether it has been tampered with, etc., and the specific method can be selected according to actual needs.

[0085] The beneficial effect of the above technical solution is that the present invention can ensure that the document processing process is both efficient and secure through the use of identification codes, the management of document databases, the implementation of security detection and the optimization of query and reading operations.

[0086] S120: Obtaining document information of the document to be cut, and performing a first cutting according to the document information to obtain a plurality of cut blocks;

[0087] In some embodiments of the present application, when the document information of the document to be cut is obtained and the first cutting is performed according to the document information to obtain a plurality of cut blocks, the method includes:

[0088] Analyzing the document to be cut based on preset cutting points to determine a plurality of cutting units, wherein each cutting unit is a complete sentence;

[0089] Parse each cutting unit to determine the document content and document structure corresponding to the cutting unit, analyze the document content based on a pre-trained document content index value model, and output the corresponding document content index value, and analyze the document structure based on a pre-trained document structure index value model, and output the corresponding document structure index value;

[0090] Calculate a comprehensive index value of a cutting unit based on the document content index value and the document architecture index value;

[0091] P = β1 × (1 + e1) e2 +β2×(1+e2) e1 ;

[0092] Wherein, P is the comprehensive index value of the cutting unit, β1 is the first calculation coefficient, e1 is the document content index value, β2 is the second calculation coefficient, and e2 is the document architecture index value;

[0093] All cutting units are bound in pairs based on the cutting order. If there are unbound cutting units, the unbound cutting units are used as a cutting block.

[0094] Calculate the comprehensive index difference between every two bound cutting units to obtain a preset comprehensive index difference;

[0095] Extract all comprehensive index difference values ​​that are less than or equal to the preset comprehensive index difference value and use them as a cutting block respectively;

[0096] Unbinding all comprehensive indicator difference values ​​greater than the preset comprehensive indicator difference value to obtain a comprehensive indicator difference value series;

[0097] All cutting units corresponding to the comprehensive index difference series are used as cutting blocks.

[0098] In this embodiment, the preset cutting point is a period, and cutting is performed every other period.

[0099] In this embodiment, document content: key information in the cutting units, such as topics, keywords, sentences, etc., document architecture: how the cutting units form the overall structure of the document, such as sequential structure, hierarchical structure or network structure.

[0100] In this embodiment, the document content index value model and the document architecture index value model are pre-trained, and are obtained through training based on different data samples and index values ​​matched by the data samples.

[0101] In this embodiment, the preset comprehensive index difference is a preset value, which is preferably 80.

[0102] In this embodiment, each cutting block includes one or more cutting units.

[0103] The beneficial effect of the above technical solution is that the present invention combines the cutting units to obtain multiple cutting blocks, which can lay the foundation for document cutting and achieve preliminary cutting.

[0104] In some embodiments of the present application, when analyzing the document content based on a pre-trained document content index value model and outputting the corresponding document content index value, and analyzing the document architecture based on a pre-trained document architecture index value model and outputting the corresponding document architecture index value, the method includes:

[0105] Performing word segmentation processing on the document content to obtain split words of the document content;

[0106] Performing a first annotation on each split word based on a preset part of speech, performing word sense matching on the document content according to the first annotation, and obtaining a word sense segmentation set of the document content;

[0107] Processing the document content set based on the document content index value model to obtain the document content index value;

[0108] Splitting the document structure to obtain a split layout of the document structure;

[0109] Performing a second annotation on each split layout based on the preset layout, performing layout matching on the document structure according to the second annotation, and obtaining a split layout set of the document structure;

[0110] The split layout set is processed based on the document architecture index value model to obtain the document architecture index value.

[0111] In this embodiment, a word segmentation algorithm (such as the word segmentation function in the NLP library) is used to split the document content into words or vocabulary, remove meaningless words (such as stop words) and standardize vocabulary (such as stemming) to reduce noise and unify different expressions of vocabulary.

[0112] In this embodiment, each word is classified according to its part of speech (noun, verb, etc.), and the specific meaning of each word is identified using a large corpus or knowledge base, and the context is considered to ensure an accurate understanding of the role and meaning of each word in the current document. All analyzed words and their meanings are integrated into a set.

[0113] In this embodiment, the word segmentation set is input into the pre-trained document content index value model. The model processes these inputs through algorithms (such as deep learning, machine learning algorithms, etc.) to evaluate multiple dimensions of document content, such as accuracy, completeness, consistency, etc., and outputs quantitative index values ​​that reflect multiple quality dimensions of document content.

[0114] In this embodiment, the structural layout of the document is parsed to identify organizational structures such as chapters, titles, and subsections. The document layout is split into independent units, for example, chapters and sub-chapters are separated to understand the logical flow of the document. The relationships between different layout units are analyzed, such as hierarchical relationships, sequential relationships, etc.

[0115] In this example, each layout part is labeled according to its function and meaning (such as introduction, conclusion, etc.). Determine how each layout part works together to form a coherent whole to ensure the readability and logic of the document. Create a collection that contains all layout units and their functions.

[0116] In this embodiment, the split layout set is input into a pre-trained document architecture index value model, and the index value related to the document architecture is output.

[0117] The beneficial effect of the above technical solution is: ensuring the accurate output of document content index values ​​and document architecture index values, and avoiding errors caused by human participation.

[0118] S130: Determine the comprehensive index value corresponding to each cutting block, sort all the comprehensive index values ​​by numerical value, and merge the cutting blocks that meet the conditions based on the sorting result to obtain multiple fused blocks;

[0119] In some embodiments of the present application, when determining the comprehensive index value corresponding to each cutting block, sorting all the comprehensive index values ​​by numerical value, and fusing the cutting blocks that meet the conditions based on the sorting result to obtain multiple fused blocks, it includes:

[0120] Determine a comprehensive indicator value set based on the sorting result, fuse the cutting blocks corresponding to the first comprehensive indicator value and the second comprehensive indicator value in the comprehensive indicator value set to obtain a first fused block, and use the cutting block corresponding to the third comprehensive indicator value in the comprehensive indicator value set as the cutting block to be updated;

[0121] Obtain a second comprehensive index difference between the cut block to be updated and the cut block corresponding to the second comprehensive index value, and obtain a preset second preset comprehensive index difference;

[0122] Determine whether the second comprehensive index difference is less than or equal to the second preset comprehensive index difference; if so, add the cut block to be updated to the first fusion block to update the first fusion block; if not, use the cut block to be updated as the second fusion block;

[0123] Taking the cutting block corresponding to the fourth comprehensive index value in the comprehensive index value set as the next cutting block to be updated;

[0124] Obtaining a third comprehensive index difference between the next cutting block to be updated and the cutting block corresponding to the third comprehensive index value in the second fusion block or the first fusion block, and determining whether the third comprehensive index difference is less than or equal to the second preset comprehensive index difference; if so, adding the next cutting block to be updated to the second fusion block or the first fusion block; if not, using the next cutting block to be updated as the second fusion block or the third fusion block;

[0125] In this way, updates and iterations are performed to divide the comprehensive indicator value set into multiple fusion blocks.

[0126] In this embodiment, when determining the comprehensive index value corresponding to each cutting block, if there are two or more cutting units in the cutting block, the corresponding sum is calculated as the comprehensive index value corresponding to the cutting block.

[0127] In this embodiment, the second preset comprehensive index difference is a preset value, which is preferably 20 here.

[0128] In this embodiment, the third comprehensive index difference between the next cut block to be updated and the cut block corresponding to the third comprehensive index value in the second fusion block or the first fusion block is obtained. Here, if the second comprehensive index value is added to the first fusion block, the third comprehensive index difference between the next cut block to be updated and the cut block corresponding to the third comprehensive index value in the first fusion block is obtained. If the second comprehensive index value is not added to the first fusion block, the first fusion block and the second fusion block are obtained, and the third comprehensive index difference between the next cut block to be updated and the cut block corresponding to the second fusion block is obtained. Continue to use the cut block corresponding to the fifth comprehensive index value as the next cut block to be updated, and continue to divide the comprehensive index value set into multiple fusion blocks according to the above judgment.

[0129] The beneficial effect of the above technical solution is that the present invention obtains multiple fusion blocks by analyzing the cut blocks, which can provide reliable data support for the construction of layered intervals and further ensure the precision and accuracy of document cutting.

[0130] S140: collecting the number of cut blocks in each fusion block, and calculating the hierarchical guidance factor of the corresponding fusion block according to the number of cut blocks;

[0131] In some embodiments of the present application, when collecting the number of cut blocks in each fusion block and calculating the hierarchical guidance factor of the corresponding fusion block according to the number of cut blocks, it includes:

[0132] A first weight is configured for the fusion block, and a second weight is configured for each cut block;

[0133] The hierarchical guidance factor of the corresponding fusion block is calculated according to the following formula:

[0134]

[0135] Among them, G is the hierarchical guidance factor, D is the first weight, n is the number of cut blocks in the fusion block, and f i is the comprehensive index value of the i-th cutting block, h i is the second weight of the i-th cutting block.

[0136] The beneficial effect of the above technical solution is that the present invention calculates the stratification guidance factor of the corresponding fusion block according to the number of cut blocks, and performs stratification according to the stratification guidance factor, thereby ensuring the reliability of stratification and avoiding stratification confusion.

[0137] S150: stratifying the fused block according to all stratification guiding factors to obtain a plurality of stratification intervals, and judging whether the stratification intervals need to be adjusted based on the stratification results, and when it is judged that the stratification intervals do not need to be adjusted, taking the stratification intervals as target document segmentation;

[0138] In some embodiments of the present application, when the fusion block is layered according to all the layered guiding factors to obtain multiple layer intervals, and whether the layer intervals need to be adjusted is determined based on the layered results, it includes:

[0139] Presetting a boundary range for each hierarchical interval, wherein the boundary range includes a left hierarchical guidance factor boundary value and a right hierarchical guidance factor boundary value;

[0140] Traversing all hierarchical guidance factors in the boundary range to determine corresponding hierarchical intervals, wherein the first layer of the hierarchical interval includes the smallest hierarchical guidance factor;

[0141] Determine the height of the hierarchical interval and the number of fusion blocks in each hierarchical interval, wherein the height of the hierarchical interval refers to the number of hierarchical intervals;

[0142] Obtaining a preset height and a preset number, and when the height is less than or equal to the preset height, determining that the level interval does not need to be adjusted;

[0143] When the height is greater than the preset height, a suspected adjustment mark is generated, and it is determined whether the number is less than or equal to the preset number. If so, it is determined that the level interval does not need to be adjusted;

[0144] If not, it is determined that the level interval needs to be adjusted.

[0145] In this embodiment, the boundary range of the first level interval is [0, 4], the boundary range of the second level interval is (4, 8], and the boundary range of the third level interval is (8, 12], and the rest are not shown one by one.

[0146] In this embodiment, the preset height and the preset number are preset values, the preset height is preferably 8, and the preset number is preferably 4.

[0147] The beneficial effect of the above technical solution is that the present invention obtains multiple hierarchical spaces by layering the fusion blocks, and can finally realize the cutting of the files according to the hierarchical intervals, which not only ensures the cutting efficiency but also avoids the phenomenon of incomplete documents after cutting.

[0148] S160: When it is determined that the hierarchical interval needs to be adjusted, an adjustment coefficient is set based on the fusion block, and the hierarchical interval is adjusted according to the adjustment coefficient, and the adjusted hierarchical interval is used as a target segmentation document.

[0149] In some embodiments of the present application, when it is determined that the hierarchical interval needs to be adjusted, an adjustment coefficient is set based on a fusion block, and the hierarchical interval is adjusted according to the adjustment coefficient, including:

[0150] Calculate the hierarchical guidance factors and values ​​within each hierarchical interval, extract them, and construct a hierarchical guidance factor and value set;

[0151] Sorting the stratified guiding factors and value sets by numerical value and determining the median;

[0152] Generate a first sub-stratification guidance factor and value set according to the stratification guidance factor and value between the head stratification guidance factor and value and the median in the stratification guidance factor and value set;

[0153] Generate a second sub-stratification guidance factor and value set according to the stratification guidance factor and value between the tail stratification guidance factor and value in the stratification guidance factor and value set and the median, wherein the first sub-stratification guidance factor and value set and the second sub-stratification guidance factor and value set do not include the median;

[0154] Calculating an adjustment coefficient based on the first sub-layer guidance factor and value set and the second sub-layer guidance factor and value;

[0155] Calculate the product value of the adjustment coefficient and all boundary ranges, and obtain a new boundary range;

[0156] All hierarchical guidance factors are traversed twice within the new boundary range to determine the adjusted hierarchical interval.

[0157] In this embodiment, as mentioned above, the boundary range of the first level interval is [0, 4], the boundary range of the second level interval is (4, 8], and the boundary range of the third level interval is (8, 12], if the calculated adjustment coefficient is 1.5, the adjusted level intervals are: the boundary range of the first level interval is [0, 6], the boundary range of the second level interval is (6, 12], and the boundary range of the third level interval is (12, 18], which is shown by way of example only and is not specifically limited.

[0158] The beneficial effect of the above technical solution is: the present invention calculates the adjustment coefficient based on the first sub-layer guidance factor and value set and the second sub-layer guidance factor and value, calculates the product value of the adjustment coefficient and all boundary ranges, and obtains a new boundary range, which can expand the boundary range of the layered interval, thereby ensuring that each layered interval can fall into more fusion blocks, further shortening the height of the hierarchical interval, and realizing accurate cutting of the document.

[0159] In some embodiments of the present application, when calculating the adjustment coefficient according to the first sub-layer guiding factor and value set and the second sub-layer guiding factor and value, it includes:

[0160] Calculate the adjustment factor according to the following formula:

[0161]

[0162] Where t is the adjustment coefficient, m1 is the number of stratified guiding factors and values ​​in the first sub-stratified guiding factor and value set, q is the median, and r j is the jth hierarchical guidance factor and value in the first sub-hierarchical guidance factor and value set, m2 is the number of hierarchical guidance factors and values ​​in the second sub-hierarchical guidance factor and value set, v w is the wth hierarchical guidance factor and value in the second sub-hierarchical guidance factor and value set.

[0163] In the description of the above embodiments, specific features, structures, materials or characteristics may be combined in a suitable manner in any one or more embodiments or examples.

[0164] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed by the present invention may be used in combination with each other in any manner, and the fact that these combinations are not fully described in this specification is only for the sake of omitting space and saving resources.

[0165] Those skilled in the art can understand that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions recorded in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An adaptive multi-level document cutting method, characterized in that: include: Receiving an original document, parsing the original document, determining whether the original document meets the cutting condition, and if so, determining that the original document is a document to be cut; Acquiring document information of the document to be cut, and performing a first cutting according to the document information to obtain a plurality of cut blocks; Determine the comprehensive index value corresponding to each cutting block, sort all the comprehensive index values ​​by numerical value, and fuse the cutting blocks that meet the conditions based on the sorting results to obtain multiple fused blocks; Collect the number of cut blocks in each fusion block, and calculate the hierarchical guidance factor of the corresponding fusion block according to the number of cut blocks; The fusion block is layered according to all the layering guiding factors to obtain a plurality of layer intervals, and whether the layer interval needs to be adjusted is determined based on the layering result. When it is determined that the layer interval does not need to be adjusted, the layer interval is used as a target document for segmentation; When it is determined that the hierarchical interval needs to be adjusted, an adjustment coefficient is set based on the fusion block, and the hierarchical interval is adjusted according to the adjustment coefficient, and the adjusted hierarchical interval is used as a target cutting document; When the document information of the document to be cut is obtained and the first cutting is performed according to the document information to obtain a plurality of cut blocks, it includes: Analyzing the document to be cut based on preset cutting points to determine a plurality of cutting units, wherein each cutting unit is a complete sentence; Parse each cutting unit to determine the document content and document structure corresponding to the cutting unit, analyze the document content based on a pre-trained document content index value model, and output the corresponding document content index value, and analyze the document structure based on a pre-trained document structure index value model, and output the corresponding document structure index value; Calculate a comprehensive index value of a cutting unit based on the document content index value and the document architecture index value; P=β1×(1+e1) e2 +β2×(1+e2) e1 ; Wherein, P is the comprehensive index value of the cutting unit, β1 is the first calculation coefficient, e1 is the document content index value, β2 is the second calculation coefficient, and e2 is the document architecture index value; All cutting units are bound in pairs based on the cutting order. If there are unbound cutting units, the unbound cutting units are used as a cutting block. Calculate the comprehensive index difference between every two bound cutting units to obtain a preset comprehensive index difference; Extract all comprehensive index difference values ​​that are less than or equal to the preset comprehensive index difference value and use them as a cutting block respectively; Unbinding all comprehensive indicator difference values ​​greater than the preset comprehensive indicator difference value to obtain a comprehensive indicator difference value series; All cutting units corresponding to the comprehensive index difference series are used as cutting blocks.

2. The adaptive multi-level document cutting method according to claim 1, characterized in that: When receiving an original document, parsing the original document to determine whether the original document meets the cutting condition, and if so, determining that the original document is a document to be cut, the method includes: Determining whether the original document is an accessed document based on the identification code carried by the original document, wherein the accessed document is stored in a document database, and the document in the document database is a document that has passed security detection; When the original document is a document that has been accessed, determining whether the original document is credible; If the original document is credible, determining that the original document meets the cutting condition; When the original document is a file that has not been accessed, a query read operation is performed and a security check is performed.

3. The adaptive multi-level document cutting method according to claim 1, characterized in that: When analyzing the document content based on a pre-trained document content index value model and outputting a corresponding document content index value, and analyzing the document architecture based on a pre-trained document architecture index value model and outputting a corresponding document architecture index value, the method includes: Performing word segmentation processing on the document content to obtain split words of the document content; Performing a first annotation on each split word based on a preset part of speech, performing word sense matching on the document content according to the first annotation, and obtaining a word sense segmentation set of the document content; Processing the document content set based on the document content index value model to obtain the document content index value; Splitting the document structure to obtain a split layout of the document structure; Performing a second annotation on each split layout based on the preset layout, performing layout matching on the document structure according to the second annotation, and obtaining a split layout set of the document structure; The split layout set is processed based on the document architecture index value model to obtain the document architecture index value.

4. The adaptive multi-level document cutting method according to claim 1, characterized in that: When the comprehensive index value corresponding to each cutting block is determined, and all the comprehensive index values ​​are sorted by numerical size, the cutting blocks that meet the conditions are merged based on the sorting results to obtain multiple fused blocks, including: Determine a comprehensive indicator value set based on the sorting result, fuse the cutting blocks corresponding to the first comprehensive indicator value and the second comprehensive indicator value in the comprehensive indicator value set to obtain a first fused block, and use the cutting block corresponding to the third comprehensive indicator value in the comprehensive indicator value set as the cutting block to be updated; Obtain a second comprehensive index difference between the cut block to be updated and the cut block corresponding to the second comprehensive index value, and obtain a preset second preset comprehensive index difference; Determine whether the second comprehensive index difference is less than or equal to the second preset comprehensive index difference; if so, add the cut block to be updated to the first fusion block to update the first fusion block; if not, use the cut block to be updated as the second fusion block; Taking the cutting block corresponding to the fourth comprehensive index value in the comprehensive index value set as the next cutting block to be updated; Obtaining a third comprehensive index difference between the next cutting block to be updated and the cutting block corresponding to the third comprehensive index value in the second fusion block or the first fusion block, and determining whether the third comprehensive index difference is less than or equal to the second preset comprehensive index difference; if so, adding the next cutting block to be updated to the second fusion block or the first fusion block; if not, using the next cutting block to be updated as the second fusion block or the third fusion block; In this way, updates and iterations are performed to divide the comprehensive indicator value set into multiple fusion blocks.

5. The adaptive multi-level document cutting method according to claim 4, characterized in that: When collecting the number of cut blocks in each fusion block and calculating the hierarchical guidance factor of the corresponding fusion block according to the number of cut blocks, it includes: A first weight is configured for the fusion block, and a second weight is configured for each cut block; The hierarchical guidance factor of the corresponding fusion block is calculated according to the following formula: Among them, G is the hierarchical guidance factor, D is the first weight, n is the number of cut blocks in the fusion block, and f i is the comprehensive index value of the i-th cutting block, h i is the second weight of the i-th cutting block.

6. The adaptive multi-level document cutting method according to claim 5, characterized in that: When the fusion block is layered according to all the layering guidance factors to obtain multiple layer intervals, and whether the layer interval needs to be adjusted is determined based on the layering results, including: Presetting a boundary range for each hierarchical interval, wherein the boundary range includes a left hierarchical guidance factor boundary value and a right hierarchical guidance factor boundary value; Traversing all hierarchical guidance factors in the boundary range to determine corresponding hierarchical intervals, wherein the first layer of the hierarchical interval includes the smallest hierarchical guidance factor; Determine the height of the hierarchical interval and the number of fusion blocks in each hierarchical interval, wherein the height of the hierarchical interval refers to the number of hierarchical intervals; Obtaining a preset height and a preset number, and when the height is less than or equal to the preset height, determining that the level interval does not need to be adjusted; When the height is greater than the preset height, a suspected adjustment mark is generated, and it is determined whether the number is less than or equal to the preset number. If so, it is determined that the level interval does not need to be adjusted; If not, it is determined that the level interval needs to be adjusted.

7. The adaptive multi-level document cutting method according to claim 6, characterized in that: When it is determined that the hierarchical interval needs to be adjusted, an adjustment coefficient is set based on the fusion block, and the hierarchical interval is adjusted according to the adjustment coefficient, including: Calculate the hierarchical guidance factors and values ​​within each hierarchical interval, extract them, and construct a hierarchical guidance factor and value set; Sorting the stratified guiding factors and value sets by numerical value and determining the median; Generate a first sub-stratification guidance factor and value set according to the stratification guidance factor and value between the head stratification guidance factor and value and the median in the stratification guidance factor and value set; Generate a second sub-stratification guidance factor and value set according to the stratification guidance factor and value between the tail stratification guidance factor and value in the stratification guidance factor and value set and the median, wherein the first sub-stratification guidance factor and value set and the second sub-stratification guidance factor and value set do not include the median; Calculating an adjustment coefficient based on the first sub-layer guidance factor and value set and the second sub-layer guidance factor and value; Calculate the product value of the adjustment coefficient and all boundary ranges, and obtain a new boundary range; All hierarchical guidance factors are traversed twice within the new boundary range to determine the adjusted hierarchical interval.

8. The adaptive multi-level document cutting method according to claim 7, characterized in that: When calculating the adjustment coefficient according to the first sub-layer guiding factor and value set and the second sub-layer guiding factor and value, it includes: Calculate the adjustment factor according to the following formula: Where t is the adjustment coefficient, m1 is the number of stratified guiding factors and values ​​in the first sub-stratified guiding factor and value set, q is the median, and r j is the jth hierarchical guidance factor and value in the first sub-hierarchical guidance factor and value set, m2 is the number of hierarchical guidance factors and values ​​in the second sub-hierarchical guidance factor and value set, v w is the wth hierarchical guidance factor and value in the second sub-hierarchical guidance factor and value set.

Images