Document splitting method and terminal

Abstract

The present invention discloses a document splitting method and a terminal. According to a received sentence splitting request, an original document is split according to a first preset splitting length and punctuation marks to obtain a plurality of sentences; the plurality of sentences are traversed until each of the sentences has been traversed; for a traversed target sentence, a previous sentence of the target sentence is determined, and the target sentence and the previous sentence are merged according to a preset sliding distance to obtain a splitting result, and the preset sliding distance is introduced to merge some of the split sentences. When there are overlapping parts in the contexts of the multiple clauses after the splitting, the overlapping parts can be merged into one split sentence, so as to avoid splitting the answer into two different small paragraphs during machine reading comprehension, thereby achieving finer-grained splitting, improving the accuracy of document splitting, and achieving cold start without a large amount of training data, thereby improving the document splitting effect.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a document splitting method and a terminal. Background Art

[0002] The Chinese document splitting of natural language question answering is to extract the document content from DOCX, PDF, and TXT and split it into small paragraphs for machine reading comprehension (MRC). Because the Transformer model (a neural network) needs to input a vector of a fixed length (for example, 512), each Chinese document needs to be split into small paragraphs of less than 512 words.

[0003] Currently, most methods for splitting documents into small paragraphs use rules or regular expressions. There are also methods that use deep learning models. For example, NLTK (Natural Language Toolkit) uses corpora and deep learning models to split document content into words or sentences. However, these methods have the following disadvantages:

[0004] 1. It only provides document splitting based on punctuation marks, but does not provide more fine-grained document splitting;

[0005] 2. The method of using corpus and deep learning model requires the organization of a large amount of training data, which requires a lot of manual work. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a document splitting method and a terminal, which can improve the document splitting effect.

[0007] In order to solve the above technical problems, a technical solution adopted by the present invention is:

[0008] A document splitting method comprises the steps of:

[0009] According to the received sentence splitting request, the original document is split according to a first preset splitting length and punctuation marks to obtain a plurality of sentences;

[0010] Traversing the plurality of sentences until each of the sentences has been traversed;

[0011] For the traversed target sentence, the previous sentence of the target sentence is determined, and the target sentence and the previous sentence are merged according to a preset sliding distance to obtain a split result.

[0012] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0013] A document splitting terminal comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0014] According to the received sentence splitting request, the original document is split according to a first preset splitting length and punctuation marks to obtain a plurality of sentences;

[0015] Traversing the plurality of sentences until each of the sentences has been traversed;

[0016] For the traversed target sentence, the previous sentence of the target sentence is determined, and the target sentence and the previous sentence are merged according to a preset sliding distance to obtain a split result.

[0017] The beneficial effects of the present invention are as follows: according to the received sentence splitting request, the original document is split according to the first preset splitting length and punctuation marks to obtain multiple sentences, the multiple sentences are traversed, and for the traversed target sentence, the previous sentence of the target sentence is determined, and the target sentence and the previous sentence are merged according to the preset sliding distance to obtain the splitting result, and the preset sliding distance is introduced to merge some of the split sentences. When there are overlapping parts in the contexts of multiple clauses after the split, the overlapping parts can be merged into one split sentence, which can avoid splitting the answer into two different small paragraphs during machine reading comprehension, thereby achieving more fine-grained splitting, improving the accuracy of document splitting, and without a large amount of training data, cold start can be achieved, thereby improving the document splitting effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of the steps of a document splitting method according to an embodiment of the present invention;

[0019] Figure 2 The present invention is a schematic diagram of the structure of a document splitting terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0021] Please refer to Figure 1 , an embodiment of the present invention provides a document splitting method, comprising the steps of:

[0022] According to the received sentence splitting request, the original document is split according to a first preset splitting length and punctuation marks to obtain a plurality of sentences;

[0023] Traversing the plurality of sentences until each of the sentences has been traversed;

[0024] For the traversed target sentence, the previous sentence of the target sentence is determined, and the target sentence and the previous sentence are merged according to a preset sliding distance to obtain a split result.

[0025] From the above description, it can be seen that the beneficial effects of the present invention are: according to the received sentence splitting request, the original document is split according to the first preset splitting length and punctuation marks to obtain multiple sentences, and the multiple sentences are traversed. For the traversed target sentence, the previous sentence of the target sentence is determined, and the target sentence and the previous sentence are merged according to the preset sliding distance to obtain the splitting result. By introducing the preset sliding distance, some of the split sentences are merged. When there are overlapping parts in the context of multiple clauses after the split, the overlapping parts can be merged into one split sentence, which can avoid splitting the answer into two different small paragraphs during machine reading comprehension, thereby achieving more fine-grained splitting, improving the accuracy of document splitting, and without a large amount of training data, cold start can be achieved, thereby improving the document splitting effect.

[0026] Furthermore, the method of splitting the original document according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences includes the following steps:

[0027] Using the first regular expression to identify blank lines in the original document, and removing the blank lines to obtain the original document after the removal;

[0028] Read each line of text in the original document after the removal;

[0029] After removing the blanks in each line of text, each line of text is merged to obtain a preprocessed original document;

[0030] The method of splitting the original document according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences includes:

[0031] According to the received sentence splitting request, the preprocessed original document is split according to the first preset splitting length and punctuation marks to obtain a plurality of sentences.

[0032] It can be seen from the above description that by removing blank spaces and empty lines in the original document, the document is preprocessed, and the preprocessed original document can avoid the situation where the inaccurate segmentation is caused by blank spaces and empty lines.

[0033] Furthermore, the preprocessed original document is split according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences including:

[0034] Using a second regular expression to identify punctuation marks in the preprocessed original document according to the received sentence splitting request;

[0035] Splitting the preprocessed original document according to the punctuation marks to obtain a plurality of initial sentences;

[0036] Determine whether the length of any of the multiple initial sentences is less than a first preset split length. If so, determine whether the length of the previous initial sentence of any of the initial sentences is less than the first preset split length. If so, merge the any initial sentence and the previous initial sentence to obtain multiple sentences. If not, determine whether the length of the subsequent initial sentence of any of the initial sentences is less than the first preset split length. If so, merge the any initial sentence and the subsequent initial sentence to obtain multiple sentences. Otherwise, do not merge.

[0037] It can be seen from the above description that the preprocessed original document is firstly split, and the multiple sentences obtained after the preliminary split according to punctuation marks not only meet the first preset split length, but also retain important semantic information in each sentence.

[0038] Furthermore, the target sentence and the previous sentence are merged according to the preset sliding distance to obtain a split result, which includes:

[0039] If the length of the previous sentence is greater than the preset sliding distance, the previous sentence is split according to punctuation marks to obtain multiple clauses;

[0040] Traversing the plurality of clauses in a direction from near to far from the target sentence, and initializing a history clause;

[0041] For the traversed target clause, determine whether the sum of the lengths of the target clause and the history clauses is greater than the preset sliding distance. If so, merge the target clause, the history clause and the target sentence in sequence to obtain a split result. If not, mark the target clause as the history clause, and after determining the next target clause from the multiple clauses, return to execute the step of determining whether the sum of the lengths of the target clause and the history clauses is greater than the preset sliding distance.

[0042] If the length of the previous sentence is less than or equal to the preset sliding distance, marking the previous sentence as a historical previous sentence;

[0043] The historical previous sentence and the previous sentence of the historical previous sentence are merged into a new previous sentence of the target sentence, and the step of returning to execute if the length of the previous sentence is greater than a preset sliding distance is performed according to the new previous sentence of the target sentence.

[0044] From the above description, it can be seen that since there will be partial overlap in the split sentences, the split sentences can be selectively merged by presetting the sliding distance, so that as much important semantic information as possible can be retained in each sentence of the split text, thereby improving the splitting effect and facilitating machine reading comprehension.

[0045] Furthermore, for the traversed target sentence, determining the previous sentence of the target sentence, and merging the target sentence and the previous sentence according to a preset sliding distance to obtain the splitting result includes the following steps:

[0046] Determine semantic boundaries and maximum preset split lengths;

[0047] The preset sliding distance is determined according to the semantic boundary and the maximum preset split length.

[0048] From the above description, it can be seen that determining the preset sliding distance according to the semantic boundary and the maximum preset split length can ensure that the determined sliding distance is not too large or too small. If it is too large, it will lead to too many overlapping parts, and if it is too small, it will not be able to retain the important semantic information of each sentence.

[0049] Please refer to Figure 2 , a document splitting terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0050] According to the received sentence splitting request, the original document is split according to a first preset splitting length and punctuation marks to obtain a plurality of sentences;

[0051] Traversing the plurality of sentences until each of the sentences has been traversed;

[0052] For the traversed target sentence, the previous sentence of the target sentence is determined, and the target sentence and the previous sentence are merged according to a preset sliding distance to obtain a split result.

[0053] From the above description, it can be seen that the beneficial effects of the present invention are: according to the received sentence splitting request, the original document is split according to the first preset splitting length and punctuation marks to obtain multiple sentences, and the multiple sentences are traversed. For the traversed target sentence, the previous sentence of the target sentence is determined, and the target sentence and the previous sentence are merged according to the preset sliding distance to obtain the splitting result. By introducing the preset sliding distance, some of the split sentences are merged. When there are overlapping parts in the context of multiple clauses after the split, the overlapping parts can be merged into one split sentence, which can avoid splitting the answer into two different small paragraphs during machine reading comprehension, thereby achieving more fine-grained splitting, improving the accuracy of document splitting, and without a large amount of training data, cold start can be achieved, thereby improving the document splitting effect.

[0054] Furthermore, the method of splitting the original document according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences includes the following steps:

[0055] Using the first regular expression to identify blank lines in the original document, and removing the blank lines to obtain the original document after the removal;

[0056] Read each line of text in the original document after the removal;

[0057] After removing the blanks in each line of text, each line of text is merged to obtain a preprocessed original document;

[0058] The method of splitting the original document according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences includes:

[0059] According to the received sentence splitting request, the preprocessed original document is split according to the first preset splitting length and punctuation marks to obtain a plurality of sentences.

[0060] It can be seen from the above description that by removing blank spaces and empty lines in the original document, the document is preprocessed, and the preprocessed original document can avoid the situation where the inaccurate segmentation is caused by blank spaces and empty lines.

[0061] Furthermore, the preprocessed original document is split according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences including:

[0062] Using a second regular expression to identify punctuation marks in the preprocessed original document according to the received sentence splitting request;

[0063] Splitting the preprocessed original document according to the punctuation marks to obtain a plurality of initial sentences;

[0064] Determine whether the length of any of the multiple initial sentences is less than a first preset split length. If so, determine whether the length of the previous initial sentence of any of the initial sentences is less than the first preset split length. If so, merge the any initial sentence and the previous initial sentence to obtain multiple sentences. If not, determine whether the length of the subsequent initial sentence of any of the initial sentences is less than the first preset split length. If so, merge the any initial sentence and the subsequent initial sentence to obtain multiple sentences. Otherwise, do not merge.

[0065] It can be seen from the above description that the preprocessed original document is firstly split, and the multiple sentences obtained after the preliminary split according to punctuation marks not only meet the first preset split length, but also retain important semantic information in each sentence.

[0066] Furthermore, the target sentence and the previous sentence are merged according to the preset sliding distance to obtain a split result, which includes:

[0067] If the length of the previous sentence is greater than the preset sliding distance, the previous sentence is split according to punctuation marks to obtain multiple clauses;

[0068] Traversing the plurality of clauses in a direction from near to far from the target sentence, and initializing a history clause;

[0069] For the traversed target clause, determine whether the sum of the lengths of the target clause and the history clauses is greater than the preset sliding distance. If so, merge the target clause, the history clause and the target sentence in sequence to obtain a split result. If not, mark the target clause as the history clause, and after determining the next target clause from the multiple clauses, return to execute the step of determining whether the sum of the lengths of the target clause and the history clauses is greater than the preset sliding distance.

[0070] If the length of the previous sentence is less than or equal to the preset sliding distance, marking the previous sentence as a historical previous sentence;

[0071] The historical previous sentence and the previous sentence of the historical previous sentence are merged into a new previous sentence of the target sentence, and the step of returning to execute if the length of the previous sentence is greater than a preset sliding distance is performed according to the new previous sentence of the target sentence.

[0072] From the above description, it can be seen that since there will be partial overlap in the split sentences, the split sentences can be selectively merged by presetting the sliding distance, so that as much important semantic information as possible can be retained in each sentence of the split text, thereby improving the splitting effect and facilitating machine reading comprehension.

[0073] Furthermore, for the traversed target sentence, determining the previous sentence of the target sentence, and merging the target sentence and the previous sentence according to a preset sliding distance to obtain the splitting result includes the following steps:

[0074] Determine semantic boundaries and maximum preset split lengths;

[0075] The preset sliding distance is determined according to the semantic boundary and the maximum preset split length.

[0076] From the above description, it can be seen that determining the preset sliding distance according to the semantic boundary and the maximum preset split length can ensure that the determined sliding distance is not too large or too small. If it is too large, it will lead to too many overlapping parts, and if it is too small, it will not be able to retain the important semantic information of each sentence.

[0077] The document splitting method and terminal described above in the present invention are applicable to a natural language question answering system, and are described below through specific implementation methods:

[0078] Embodiment 1

[0079] Please refer to Figure 1 , a document splitting method of this embodiment includes the steps of:

[0080] S0, receiving an original document and a split request corresponding to the original document, the split request including a sentence split request;

[0081] Wherein, the format of the original document is DOCX, PDF or TXT;

[0082] S1. Using a first regular expression to identify blank lines in an original document, and removing the blank lines to obtain the original document after the removal;

[0083] S2, reading each line of text in the original document after the removal;

[0084] S3, after removing the blanks in each line of text, merging each line of text to obtain a preprocessed original document;

[0085] In an optional implementation, after removing the blanks in each line of text, the header and footer in each line of text may be extracted and removed, and after removal, each line of text may be merged to obtain a preprocessed original document;

[0086] S4, splitting the original document according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain multiple sentences;

[0087] Specifically, according to the received sentence splitting request, the preprocessed original document is split according to the first preset splitting length and punctuation marks to obtain multiple sentences, specifically including:

[0088] S41, using a second regular expression to identify punctuation marks in the preprocessed original document according to the received sentence splitting request;

[0089] The punctuation marks include single-character punctuation marks, Chinese and English ellipsis punctuation marks, and Chinese double quotation mark punctuation marks; the single-character punctuation marks include period, exclamation mark, question mark, etc.;

[0090] S42, splitting the preprocessed original document according to the punctuation marks to obtain a plurality of initial sentences;

[0091] When a Chinese double quotation mark segmentation mark of a sentence contains a single-character segmentation mark, the sentence is not split according to the single-character segmentation mark in the Chinese double quotation mark segmentation mark;

[0092] S43, determining whether the length of any of the multiple initial sentences is less than the first preset split length, if so, executing S431; if not, not executing;

[0093] The first preset split length can be set according to actual conditions. In this embodiment, the first preset split length is 512;

[0094] S431, determining whether the length of the previous initial sentence of any of the initial sentences is less than the first preset split length, if so, executing S4311, if not, executing S4312;

[0095] S4311, merging any one of the initial sentences and the previous initial sentence to obtain a plurality of sentences;

[0096] S4312, determining whether the length of the subsequent initial sentence of any of the initial sentences is less than the first preset split length, if so, merging the any of the initial sentences and the subsequent initial sentence to obtain multiple sentences, otherwise, not merging;

[0097] S5, determining the semantic boundary and the maximum preset split length;

[0098] Wherein, the semantic boundary is the length of the last sentence among the multiple sentences or is set according to actual conditions;

[0099] Specifically, determine the semantic boundary s and the maximum preset split length L;

[0100] S6. Determine a preset sliding distance according to the semantic boundary and the maximum preset split length;

[0101] Specifically, the value range of the preset sliding distance m is s < m < L / 2. Such a set sliding distance will neither cause excessive overlapping parts nor can it retain important semantic information as much as possible;

[0102] S7. After sorting the multiple sentences in the order of the original document, traverse the multiple sentences until each sentence has been traversed;

[0103] S71. For the target sentence traversed, determine the previous sentence of the target sentence, and merge the target sentence and the previous sentence according to the preset sliding distance to obtain a splitting result, specifically including:

[0104] S710. For the target sentence traversed, determine the previous sentence of the target sentence;

[0105] For example, if the target sentence traversed is sentence n, determine that the previous sentence of sentence n is sentence n - 1;

[0106] S711. If the length of the previous sentence is greater than the preset sliding distance, split the previous sentence according to punctuation marks to obtain multiple clauses;

[0107] For example, if the length of sentence n - 1 is greater than m, split sentence n - 1 according to punctuation marks such as commas, semicolons, dashes, or colons to obtain multiple clauses n - 1 = k1, k2, k3,..., kn - 1;

[0108] S712. Traverse the multiple clauses in the direction from near to far from the target sentence, and initialize the historical clause;

[0109] Specifically, traverse k1, k2, k3,..., kn - 1 in the direction from near to far from sentence n, and initialize the historical clause to be empty;

[0110] S7121. For the target clause traversed, determine whether the sum of the lengths of the target clause and the historical clause is greater than the preset sliding distance. If so, execute S71211; if not, execute S71212;

[0111] S71211. Merge the target clause, the historical clause, and the target sentence in sequence to obtain a splitting result;

[0112] S71212. Mark the target clause as the historical clause, and after determining the next target clause from the multiple clauses, return to execute S7121;

[0113] For example, traverse to the target clause kn-1, determine whether the length of kn-1 is greater than m (the history clause is empty at this time, so its length is 0 and can be ignored). If so, merge [kn-1, n]. If not, mark kn-1 as a history clause, and determine the next target clause kn-2 from k1, k2, k3, ..., kn-1. Then determine whether the sum of the lengths of kn-2 and the history clause kn-1 is greater than m. If so, merge [kn-2, kn-1, n]. If not, mark kn-2 as a history clause as well, and return to execute S7121, and so on.

[0114] S713: if the length of the previous sentence is less than or equal to the preset sliding distance, marking the previous sentence as a historical previous sentence;

[0115] S714, merging the previous historical sentence and the previous sentence of the previous historical sentence into a new previous sentence of the target sentence, and returning to execute S711 according to the new previous sentence of the target sentence;

[0116] For example, if the length of sentence n-1 is less than or equal to m, sentence n-1 is marked as the previous historical sentence, sentence n-1 and sentence n-2 are merged into a new previous sentence [n-2, n-1] of the target sentence, and according to [n-2, n-1], execution of S711 is returned to determine whether the length of [n-2, n-1] is greater than m;

[0117] In another optional implementation, the split request further includes a word-by-word split request and a segment-by-segment split request;

[0118] When the split request is a word-by-word split request, S4 is:

[0119] According to the received word-by-word splitting request, the original document is split according to a second preset splitting length in units of single words to obtain a splitting result;

[0120] When the split request is a segment split request, S4 is:

[0121] The original document is split according to the received segment splitting request to obtain a splitting result.

[0122] Embodiment 2

[0123] Please refer to Figure 2 A document splitting terminal of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step of the document splitting method in the first embodiment is implemented.

[0124] In summary, the present invention provides a document splitting method and terminal, which splits the original document according to a first preset splitting length and punctuation marks according to a received sentence splitting request to obtain multiple sentences; traverses the multiple sentences until each of the sentences has been traversed; for the traversed target sentence, determines the previous sentence of the target sentence, and merges the target sentence and the previous sentence according to a preset sliding distance to obtain a splitting result, and merges some of the split sentences by introducing a preset sliding distance. When there are overlapping parts in the contexts of multiple clauses after the split, the overlapping parts can be merged into one split sentence, which can avoid splitting the answer into two different small paragraphs during machine reading comprehension, thereby achieving more fine-grained splitting, improving the accuracy of document splitting, and without the need for a large amount of training data, cold start can be achieved, thereby improving the document splitting effect.

[0125] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A document splitting method, characterized in that: Includes steps: According to the received sentence splitting request, the original document is split according to a first preset splitting length and punctuation marks to obtain a plurality of sentences; Traversing the plurality of sentences until each of the sentences has been traversed; For the traversed target sentence, determine the previous sentence of the target sentence, and merge the target sentence and the previous sentence according to a preset sliding distance to obtain a split result; The step of merging the target sentence and the previous sentence according to the preset sliding distance to obtain a split result includes: If the length of the previous sentence is greater than the preset sliding distance, the previous sentence is split according to punctuation marks to obtain multiple clauses; Traversing the plurality of clauses in a direction from near to far from the target sentence, and initializing a history clause; For the traversed target clause, determine whether the sum of the lengths of the target clause and the history clauses is greater than the preset sliding distance. If so, merge the target clause, the history clause and the target sentence in sequence to obtain a split result. If not, mark the target clause as the history clause, and after determining the next target clause from the multiple clauses, return to execute the step of determining whether the sum of the lengths of the target clause and the history clauses is greater than the preset sliding distance. If the length of the previous sentence is less than or equal to the preset sliding distance, marking the previous sentence as a historical previous sentence; Merging the previous historical sentence and the previous sentence of the previous historical sentence into a new previous sentence of the target sentence, and returning to execute the step of if the length of the previous sentence is greater than a preset sliding distance according to the new previous sentence of the target sentence; The method of determining the previous sentence of the target sentence for the traversed target sentence, and merging the target sentence and the previous sentence according to a preset sliding distance to obtain a split result includes the following steps: Determine semantic boundaries and maximum preset split lengths; The preset sliding distance is determined according to the semantic boundary and the maximum preset split length.

2. A document splitting method according to claim 1, characterized in that: The method of splitting the original document according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences includes the following steps: Using the first regular expression to identify blank lines in the original document, and removing the blank lines to obtain the original document after the removal; Read each line of text in the original document after the removal; After removing the blanks in each line of text, each line of text is merged to obtain a preprocessed original document; The step of splitting the original document according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences includes: According to the received sentence splitting request, the preprocessed original document is split according to the first preset splitting length and punctuation marks to obtain a plurality of sentences.

3. A document splitting method according to claim 2, characterized in that: The step of splitting the preprocessed original document according to the received sentence splitting request according to the first preset splitting length and punctuation marks to obtain a plurality of sentences includes: Using a second regular expression to identify punctuation marks in the preprocessed original document according to the received sentence splitting request; Splitting the preprocessed original document according to the punctuation marks to obtain a plurality of initial sentences; Determine whether the length of any of the multiple initial sentences is less than a first preset split length. If so, determine whether the length of the previous initial sentence of any of the initial sentences is less than the first preset split length. If so, merge the any initial sentence and the previous initial sentence to obtain multiple sentences. If not, determine whether the length of the subsequent initial sentence of any of the initial sentences is less than the first preset split length. If so, merge the any initial sentence and the subsequent initial sentence to obtain multiple sentences. Otherwise, do not merge.

4. A document splitting terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the following steps are implemented: According to the received sentence splitting request, the original document is split according to a first preset splitting length and punctuation marks to obtain a plurality of sentences; Traversing the plurality of sentences until each of the sentences has been traversed; For the traversed target sentence, determine the previous sentence of the target sentence, and merge the target sentence and the previous sentence according to a preset sliding distance to obtain a split result; The step of merging the target sentence and the previous sentence according to the preset sliding distance to obtain a split result includes: If the length of the previous sentence is greater than the preset sliding distance, the previous sentence is split according to punctuation marks to obtain multiple clauses; Traversing the plurality of clauses in a direction from near to far from the target sentence, and initializing a history clause; For the traversed target clause, determine whether the sum of the lengths of the target clause and the history clauses is greater than the preset sliding distance. If so, merge the target clause, the history clause and the target sentence in sequence to obtain a split result. If not, mark the target clause as the history clause, and after determining the next target clause from the multiple clauses, return to execute the step of determining whether the sum of the lengths of the target clause and the history clauses is greater than the preset sliding distance. If the length of the previous sentence is less than or equal to the preset sliding distance, marking the previous sentence as a historical previous sentence; Merging the previous historical sentence and the previous sentence of the previous historical sentence into a new previous sentence of the target sentence, and returning to execute the step of if the length of the previous sentence is greater than a preset sliding distance according to the new previous sentence of the target sentence; The method of determining the previous sentence of the target sentence for the traversed target sentence, and merging the target sentence and the previous sentence according to a preset sliding distance to obtain a split result includes the following steps: Determine semantic boundaries and maximum preset split lengths; The preset sliding distance is determined according to the semantic boundary and the maximum preset split length.

5. A document splitting terminal according to claim 4, characterized in that: The method of splitting the original document according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences includes the following steps: Using the first regular expression to identify blank lines in the original document, and removing the blank lines to obtain the original document after the removal; Read each line of text in the original document after the removal; After removing the blanks in each line of text, each line of text is merged to obtain a preprocessed original document; The method of splitting the original document according to the first preset splitting length and punctuation marks according to the received sentence splitting request to obtain a plurality of sentences includes: According to the received sentence splitting request, the preprocessed original document is split according to the first preset splitting length and punctuation marks to obtain a plurality of sentences.

6. A document splitting terminal according to claim 5, characterized in that: The step of splitting the preprocessed original document according to the received sentence splitting request according to the first preset splitting length and punctuation marks to obtain a plurality of sentences includes: Using a second regular expression to identify punctuation marks in the preprocessed original document according to the received sentence splitting request; Splitting the preprocessed original document according to the punctuation marks to obtain a plurality of initial sentences; Determine whether the length of any of the multiple initial sentences is less than a first preset split length. If so, determine whether the length of the previous initial sentence of any of the initial sentences is less than the first preset split length. If so, merge the any initial sentence and the previous initial sentence to obtain multiple sentences. If not, determine whether the length of the subsequent initial sentence of any of the initial sentences is less than the first preset split length. If so, merge the any initial sentence and the subsequent initial sentence to obtain multiple sentences. Otherwise, do not merge.

Images