Chinese long sentence segmentation method combining syntactic analysis and prompt learning

By combining syntactic analysis and prompt learning methods, the complex structure and multilingual processing challenges in Chinese long sentence segmentation are solved, segmentation accuracy and generalization capabilities are improved, and efficient automated processing and interpretability are achieved.

CN119026592APending Publication Date: 2024-11-26GUANGXI UNIV FOR NATITIES
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411227937.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Chinese long sentence segmentation faces the challenges of complex clause structure, implicit logical breakpoint recognition and sentence integrity maintenance. The existing technology needs to adjust and rely on large-scale parallel corpus in multilingual processing, and the deep learning model is weak in interpretability.

Method used

Combining syntactic analysis and prompt learning methods, sentences with boundary information are collected through the corpus, data augmentation and automated preprocessing are performed, and judgment and segmentation models are trained using BART pre-trained models to build adaptive models to improve segmentation accuracy and generalization capabilities.

Benefits of technology

It significantly improves the accuracy and generalization ability of Chinese long sentence segmentation, reduces error segmentation and miss segmentation, provides higher interpretability and automated processing capabilities, and is suitable for diverse sentences and fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026592A_ABST
    Figure CN119026592A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of natural language processing, and discloses a Chinese long sentence segmentation method combining syntactic analysis and prompt learning, which comprises the following steps: S1, collecting corpora, and obtaining sentences with boundary information from a corpus; s2, EDA data enhancement: cleaning and labeling the first corpus, and performing EDA data enhancement; s3, data preprocessing is conducted, each sentence in the second corpus is segmented, and segmentation is completed in three steps; s4, a Prompt template is constructed; and S5, model training: using a BART pre-training model to respectively train a model for judging whether the long sentence can be segmented or not and a long sentence segmentation model by applying the two Prompt templates constructed in the step S4. According to the method, the long sentences are separated by combining syntactic analysis and prompt learning, the syntactic analysis tree is utilized to guide the segmentation of the long sentences, and the prompt learning is combined to process complex language phenomena, so that the segmentation accuracy of the complex long sentences is improved, and each segmented short sentence can independently express complete meaning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a Chinese long sentence segmentation method combining syntactic analysis and prompt learning. Background Art

[0002] As a non-morphological language, Chinese has a sentence structure that is significantly different from morphological languages ​​such as English. The word order and context in Chinese sentences are crucial to understanding the meaning of the sentence, which increases the complexity of sentence segmentation.

[0003] Sentence segmentation is a basic task in natural language processing, which directly affects the results of subsequent information extraction, text summarization, machine translation, etc. Accurate segmentation can help improve the performance of these tasks.

[0004] Traditional Chinese sentence segmentation methods usually rely on rule-based systems, statistical-based machine learning methods, or a combination of the two. These methods work well when processing simple sentences, but often fail to perform well when faced with complex, nested, or long sentences containing multiple linguistic phenomena.

[0005] Existing literature XUE N, YANG Y. Chinese sentence segmentation as comma classification [J]. HLT'11: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: shortpapers-Volume 2, 2011. In this paper, Xue and Yang proposed a method to treat the long sentence segmentation problem as a comma disambiguation problem. The core idea of ​​this method is that sentences segmented by commas often have a loose structure, and each clause maintains relatively independent and complete semantics. This means that there may not be a close syntactic relationship between clauses, but they are connected by commas as separators. By effectively solving the comma disambiguation problem, that is, accurately identifying which commas should be used as sentence boundary markers, the accuracy of sentence segmentation can be significantly improved. However, in the existing corpus, there is no public dataset that clearly marks whether the comma is a sentence boundary, and it is very difficult to obtain corpus.

[0006] With the development of deep learning technology, sentence segmentation methods based on neural networks have begun to emerge. These methods usually use deep learning architectures such as sequence labeling and sequence-to-sequence models to automatically learn the complex features of sentence structure.

[0007] Reference [1] mentions that pre-trained language models have achieved great success in the field of natural language processing. These models learn rich language representations by pre-training on large-scale texts and can be used for various downstream tasks, including sentence segmentation.

[0008] In reference [2], BART provides powerful capabilities for text generation tasks through its denoising autoencoder pre-training framework; while the prompt learning mentioned in reference [3] enables the model to be effectively applied in different tasks and fields through flexible prompt design, especially in resource-constrained situations.

[0009] In order to improve the generalization ability and robustness of the model, the data augmentation technology mentioned in reference [4] is widely used in natural language processing tasks. Through synonym replacement, sentence interpolation, back translation and other technologies, new training samples can be generated from existing data to increase data diversity.

[0010] In reference [5], some studies have attempted to combine syntactic analysis technology with the sentence segmentation task, using syntactic analysis trees to guide sentence segmentation. This method can better understand the internal structure of sentences and thus improve the accuracy of segmentation.

[0011] The special challenges faced by Chinese long sentence segmentation include handling complex clause structures, identifying implicit logical breakpoints, maintaining sentence integrity, etc. These challenges require segmentation methods to have not only strong language understanding capabilities but also flexible decision-making mechanisms.

[0012] Prior art 1:

[0013] In reference [6], a unified approach using context-based modeling is presented, trained via semi-supervised learning on imperfectly labeled data, for sentence segmentation of punctuated text in multiple languages.

[0014] Defects of prior art 1:

[0015] 1) Although the model is designed to handle multiple languages, it may require additional tuning or specific training data for a specific language to perform best.

[0016] 2) This solution is mainly aimed at text with punctuation. For text without punctuation or with unclear punctuation, additional methods may be needed to process it.

[0017] Prior art 2:

[0018] In reference

[14] , the technical solution provided is to improve the accuracy of bilingual long sentence segmentation and machine translation performance by combining the monolingual segmentation model and sentence alignment model of deep learning and using the pre-trained language model BERT.

[0019] Defects of the second prior art:

[0020] 1) Rely on large-scale parallel corpora to train models.

[0021] 2) Insufficient segmentation accuracy when dealing with complex or irregular sentence structures.

[0022] 3) It relies on deep learning models and has poor interpretability. Summary of the invention

[0023] In order to overcome or alleviate one or more of the above technical problems, the present invention aims to provide a Chinese long sentence segmentation method combining syntactic analysis and prompt learning.

[0024] The present invention provides the following technical solutions:

[0025] A Chinese long sentence segmentation method combining syntactic analysis and prompt learning, comprising the following steps:

[0026] S1: Corpus collection, obtain sentences with boundary information from the corpus, and attach the component syntactic analysis tree of each sentence; collect a large number of long Chinese sentences in the target field from the Internet, and finally the set of long sentences with boundary information is called the first data set, and the set of long sentences without boundary information is called the second data set. The first data set and the second data set together constitute the first corpus;

[0027] S2: EDA data enhancement, cleaning and annotating the first corpus, and performing EDA data enhancement, including the following steps:

[0028] S21: first use the formula n=α*l to determine the number of words that need to be changed in each sentence in the first corpus, where l represents the length of the sentence and α is the proportional coefficient, and apply EDA data enhancement of synonym replacement, random insertion, random exchange and random deletion of n words to each sentence;

[0029] S22: generating n enhanced sentences for each original sentence in the first corpus, merging the newly generated sentences with the second data set to obtain a third data set, wherein the first data set and the third data set together constitute the second corpus;

[0030] S3: Data preprocessing, segmenting each sentence in the second corpus, and completing the segmentation in three steps. The queue used in the process is first-in-first-out, including the following steps:

[0031] S31: Segment each sentence in the second corpus according to the period, question mark, and exclamation mark to obtain a sentence set S, and then determine whether there is a comma in each sentence in the sentence set S. If there is a comma, the sentence is put into queue A; if not, the sentence is put into queue B;

[0032] S32: the sentence in queue A is removed from the queue, and the sentence is subjected to component syntactic analysis to obtain a component syntactic analysis tree. If the left and right nodes of the comma in the component syntactic analysis tree are both IP, the comma is replaced with a [TA] tag, otherwise the comma is replaced with a [NO] tag, and then the sentence is segmented according to the [TA] tag, and the segmented sentences are put into queue B; the above is repeated until queue A is empty; when obtaining the component syntactic analysis tree, if the sentence source is a long sentence with boundary information, that is, from the first data set, the component syntactic analysis tree attached thereto is directly used;

[0033] S33: remove the sentence from queue B, replace the [NO] tag in the sentence with a comma, add the sentence to the cut sentence set if the sentence length does not exceed l, perform dependency syntactic analysis on the sentence to obtain a dependency graph, determine whether there is an adverbial clause pattern according to the dependency graph, insert a [TA] tag before the adverbial clause guide word if it does, and manually insert a [TA] tag if it does not, and then segment the sentence according to the [TA] tag; determine again whether the sentence length exceeds l, if it does not exceed, add the sentence to the cut sentence set, if it exceeds, manually insert a [TA] tag, and then segment the sentence according to the [TA] tag, and then add it to the cut sentence set; repeat the above operations until queue B is empty; the cut sentence set will contain the input sentence and the corresponding segmented sentence list, and the cut sentence set is the fourth data set; based on the fourth data set, segmentation and analysis are performed to obtain another data set containing the input sentence and the corresponding binary tag, which is the fifth data set;

[0034] S4: Prompt template construction, setting two Prompt templates to train two models respectively, the two Prompts are the first Prompt template and the second Prompt template; the first Prompt template is used to determine whether the sentence can be segmented; the second Prompt template is used to segment long sentences;

[0035] S5: Model training, using the BART pre-trained model to apply the two Prompt templates constructed in step S4 to train the long sentence segmentation judgment model and the long sentence segmentation model respectively, including the following steps:

[0036] S51: In the training stage, the vector codes corresponding to the sentences of different templates are input to the encoding layer of the BART model, and the decoder receives the output of the encoder and generates a corresponding output; for the segmentation judgment model, that is, the model for judging whether the long sentence can be segmented, the fifth data set is used for model training;

[0037] S52: In the training phase, the first Prompt template is applied to each sentence to obtain the corresponding vector code, which is input into the encoding layer of the BART model. The decoder receives the output of the encoder, and a Softmax output layer is set for the output of the decoder to obtain two probability values, corresponding to the probabilities of yes and no respectively.

[0038] S53: For the segmentation execution model, that is, the long sentence segmentation model, use the fourth data set to perform model training;

[0039] S54: During the training phase, the second Prompt template is applied to each sentence to obtain the corresponding vector encoding which is input into the encoding layer of the BART model. The decoder receives the output of the encoder, and the output of the decoder is a list of segmented sentences.

[0040] According to some implementations, the corpus in step S1 is a phrase structure syntactic annotated corpus.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1) The present invention combines syntactic analysis and prompt learning to segment long sentences. The combined method uses a syntactic analysis tree to guide the segmentation of long sentences, and combines prompt learning to handle complex language phenomena to improve the segmentation accuracy of complex long sentences and ensure that each segmented short sentence can independently express a complete meaning. A Chinese long sentence segmentation model is obtained by combining syntactic analysis, prompt learning, data enhancement, and high-quality Chinese long sentence segmentation corpus for training. The model can more accurately identify logical breakpoints in long sentences, reduce the situation of incorrect segmentation or missed segmentation, and greatly improve the accuracy.

[0043] 2) This invention collects a large amount of corpus from different sources and performs data enhancement to improve the model's generalization ability for long sentences of various types and structures. It can handle diverse sentences. The model trained with data enhancement and diverse corpus can handle long sentences in various scenarios and fields, including unstructured and colloquial texts.

[0044] 3) The present invention automatically processes long sentences of various formats and sources through automated data preprocessing and model training processes, reduces manual participation, and improves the automation of segmentation tasks. The automated segmentation process greatly improves the speed of processing long sentences and reduces labor costs.

[0045] 4) The present invention constructs an adaptive model so that it can handle long sentence segmentation tasks in various scenarios and fields.

[0046] 5) The present invention combines syntactic analysis to provide a higher degree of interpretability, making the segmentation decisions of the model easier to understand and verify.

[0047] 6) The present invention is easy to integrate and deploy. Due to the efficiency and accuracy of the model, it can be conveniently integrated into various natural language processing applications, such as text summarization, information extraction, and machine translation.

[0048] 7) The present invention increases the user's trust in the model output through explainable segmentation decisions, thereby improving the end-user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flowchart of a Chinese long sentence segmentation method combining syntactic analysis and prompt learning provided by an embodiment of the present invention.

[0050] Figure 2 This is a flowchart of the first step of sentence segmentation in data preprocessing provided by an embodiment of the present invention.

[0051] Figure 3 A flowchart of sentence segmentation, the second step of data preprocessing, provided in an embodiment of the present invention.

[0052] Figure 4 A flowchart of sentence segmentation, the third step of data preprocessing, provided in an embodiment of the present invention.

[0053] Figure 5 A flowchart of an example of preprocessing before and after sentences provided in an embodiment of the present invention.

[0054] Figure 6 A schematic diagram of a model framework for determining whether a long sentence is segmentable provided in an embodiment of the present invention.

[0055] Figure 7 A schematic diagram of a long sentence segmentation model framework provided in an embodiment of the present invention.

[0056] Figure 8 This is a list of second corpus examples provided by the embodiment of the present invention.

[0057] Fig. 9 The first step of data preprocessing provided in the embodiment of the present invention is sentence segmentation result.

[0058] Fig.10 The sentence segmentation result of the second step of data preprocessing provided in the embodiment of the present invention.

[0059] Fig.11The sentence segmentation result of the third step of data preprocessing provided in the embodiment of the present invention. DETAILED DESCRIPTION

[0060] The present invention is described in detail below in conjunction with the embodiments and drawings, but it should be understood that the embodiments and drawings are only used to exemplify the present invention and do not constitute any limitation on the protection scope of the present invention. All reasonable changes and combinations within the scope of the inventive concept of the present invention fall within the protection scope of the present invention.

[0061] The present invention will be further described below in conjunction with the accompanying drawings.

[0062] Example 1

[0063] like Figure 1 As shown, the present embodiment provides a Chinese long sentence segmentation method combining syntactic analysis and prompt learning, comprising the following steps:

[0064] 1) Corpus collection: 12,258 sentences with boundary information were obtained from the Pennsylvania Chinese Treebank [8], along with the component syntactic analysis tree of each sentence [9]; a large number of long Chinese sentences in the target domain were collected from the Internet using a crawler

[10] . The final set of long sentences with boundary information is called the first dataset, and the set of long sentences without boundary information is called the second dataset. The first dataset and the second dataset together constitute the first corpus. In addition to the Pennsylvania Chinese Treebank, sentences with boundary information can also be obtained from other libraries, such as Sinica Treebank 3.0, the corpus of the Beijing Language and Culture University Language Resources Advanced Innovation Center, and other reliable phrase structure syntactic annotated corpora.

[0065] 2) EDA data augmentation [4]: ​​The first corpus is cleaned and annotated, and data augmentation is performed. First, the formula n = α*l is used to determine the number of words that need to be changed in each sentence in the first corpus, where l represents the length of the sentence and α is the proportional coefficient. EDA (Easy Data Augmentation) technology is applied to each sentence, including synonym replacement, random insertion, random exchange, and random deletion of n words. For each original sentence in the first corpus, n enhanced sentences are generated, and the newly generated sentences are merged with the second data set to obtain the third data set. The first data set and the third data set together constitute the second corpus.

[0066] 3) Data preprocessing: Segment each sentence in the second corpus in three steps, using the first-in-first-out queue. Figure 2 As shown in , each sentence in the second corpus is segmented according to the period, question mark, and exclamation mark to obtain a sentence set S, and then it is determined whether there is a comma in each sentence in S. If there is a comma, the sentence is put into queue A, and if not, it is put into queue B. The second step is as follows Figure 3As shown, the sentences in queue A are removed from the queue, and the sentence is subjected to component syntactic analysis to obtain a component syntactic analysis tree. If the left and right nodes of the comma in the tree are both IP (independent components), the comma is replaced with the [TA] tag, otherwise the comma is replaced with the [NO] tag, and then the sentence is segmented according to the [TA] tag, and the segmented sentences are put into queue B. Repeat the above until queue A is empty. When obtaining the component syntactic analysis tree, if the sentence source is a long sentence with boundary information, that is, from the first data set, the component syntactic analysis tree attached to it is used directly. The third step, as Figure 4 As shown, the sentence in queue B is removed from the queue, and the [NO] label in the sentence is replaced with a comma. If the sentence length does not exceed l, the sentence is added to the set of sentences after cutting. If the sentence length exceeds l, the sentence is subjected to dependency syntactic analysis to obtain a dependency graph. According to the dependency graph, it is determined whether there is an adverbial clause pattern. If so, a [TA] tag is inserted before the adverbial clause guide word. If not, a [TA] tag is manually inserted, and then the sentence is segmented according to the [TA] tag. It is determined again whether the sentence length exceeds l. If it does not exceed, the sentence is added to the set of sentences after cutting. If it exceeds, a [TA] tag is manually inserted, and then the sentence is segmented according to the [TA] tag and added to the set of sentences after cutting. The above operation is repeated until queue B is empty. The preprocessed data structure is shown as follows. Figure 5 As shown in the figure, if it is a separable sentence, it corresponds to several short sentences after segmentation, such as sentence A; if it is an inseparable sentence, it corresponds to the original sentence, such as sentence B. The segmented sentence set will contain the input sentence and the corresponding segmented sentence list, and the segmented sentence set is called the fourth data set. Based on the fourth data set, segmentation analysis is performed to obtain another data set containing the input sentence and the corresponding binary label (separable / inseparable), which is called the fifth data set.

[0067] 4) Prompt template construction

[12] : Set two prompts to train two models respectively. The first prompt template is used to determine whether a sentence can be segmented. For the input sentence X, a text string template "[CLS]" "[X]" is manually created. Can it be segmented? "[MASK][SEP]". In this template, [CLS] represents the beginning of the label, [X] represents the position of the input sentence, [MASK] is a placeholder, indicating the part that the model needs to judge, and [SEP] represents the end of the sentence separator. During the training process, the model will learn to predict the label at the [MASK] position, that is, whether the sentence can be segmented. The second prompt performs long sentence segmentation. For the input sentence X, a text string template "[CLS]" "[X]" is manually created. It can be segmented into these sentences "[MASK][SEP]". In this template, [CLS] and [X] have the same meaning as above, [MASK] now represents the list of segmented sentences, and [SEP] represents the sentence separator. The task of the model is to generate one or more sentences that will replace [MASK] and be separated from the original sentence by [SEP].

[0068] 5) Model training: Use the BART pre-trained model [2] to apply the two Prompt templates constructed in step 4) to train the "long sentence segmentation model" and "long sentence segmentation model" respectively. During the training phase, the vector encoding corresponding to the sentences of different templates is input into the encoding layer of the BART model. The decoder receives the output of the encoder and generates the corresponding output. For the segmentation judgment model, that is, the "long sentence segmentation model", the fifth data set is used for model training, such as Figure 6 As shown in the figure, in the training phase, the first Prompt template is applied to each sentence to obtain the corresponding vector encoding, which is input into the encoding layer of the BART model. The decoder receives the output of the encoder, and sets a Softmax output layer on the output of the decoder to obtain two probability values, corresponding to the probabilities of "yes" and "no", respectively. For the segmentation execution model, that is, the "long sentence segmentation model", the fourth data set is used for model training, as shown in Figure 7 As shown, during the training phase, the second Prompt template is applied to each sentence to obtain the corresponding vector encoding which is input into the encoding layer of the BART model. The decoder receives the output of the encoder, and the output of the decoder is a list of segmented sentences.

[0069] Example 2

[0070] According to the Chinese long sentence segmentation method combining syntactic analysis and prompt learning provided in Example 1, long sentences are segmented, and the data preprocessing example is as follows:

[0071] The second corpus obtained is Figure 8 :

[0072] It contains four sentences. Sentences 1 and 4 are from the third dataset and are sentences without boundary information. Sentences 2 and 3 are from the first dataset and are sentences with boundary information. Their component syntactic analysis trees can be directly queried. The first step of sentence segmentation results, such as Fig. 9 :

[0073] First, each sentence in the second corpus is segmented according to the period, question mark, and exclamation mark to obtain a sentence set S. All four sentences end with a period and do not contain a period, question mark, or exclamation mark, so the four sentences are removed from the period and then put into the sentence set S. After obtaining the sentence set S, determine whether there is a comma in each sentence in S. If there is, the sentence is put into queue A, and if not, it is put into queue B. Sentences 2 and 3 have commas in them and are put into queue A; sentences 1 and 4 do not have commas in them and are put into queue B. After judging and operating the four sentences, we get the following: Fig. 9 The sentence set S, queue A and queue B are shown.

[0074] The second step of sentence segmentation results, such as Fig.10 :

[0075] The second step is, Figure 3 As shown, the sentences in queue A are removed from the queue, and the sentence is subjected to component syntactic analysis to obtain a component syntactic analysis tree. If the left and right nodes of the comma in the tree are both IP (independent components), the comma is replaced with the [TA] tag, otherwise the comma is replaced with the [NO] tag, and then the sentence is segmented according to the [TA] tag, and the segmented sentences are put into queue B. Repeat the above until queue A is empty. When obtaining the component syntactic analysis tree, if the sentence source is a long sentence with boundary information, that is, from the first data set, the component syntactic analysis tree attached to it is used directly. Sentences 2 and 3 in queue A come from the first data set, and the component syntactic analysis trees attached to them are read directly. From the tree diagram, it is known that the left and right sides of the comma in sentence 2 are both independent components, so the comma is replaced with the [TA] tag; from the tree diagram, it is known that the left and right sides of the comma in sentence 3 are not both independent components, so the comma is replaced with the [NO] tag. Then the sentence is segmented according to the [TA] tag, and the segmented sentences are put into queue B. Fig. 9 After executing the above operations on queue A, the new queue B is obtained as follows Fig.10 shown.

[0076] The third step is sentence segmentation results, such as Fig.11 :

[0077] The third step is Figure 4As shown, the new sentence in queue B is removed from the queue, and the [NO] tag in the sentence is replaced with a comma. If the sentence length does not exceed l, the sentence is added to the set of sentences after segmentation. If the sentence length exceeds l, the sentence is subjected to dependency syntactic analysis to obtain a dependency graph. According to the dependency graph, it is determined whether there is an adverbial clause pattern. If so, a [TA] tag is inserted before the adverbial clause guide word. If not, a [TA] tag is manually inserted, and then the sentence is segmented according to the [TA] tag. It is determined again whether the sentence length exceeds l. If not, the sentence is added to the set of sentences after segmentation. If so, a [TA] tag is manually inserted, and then the sentence is segmented according to the [TA] tag and then added to the set of sentences after segmentation. Repeat the above operation until queue B is empty. When the length of the distinction between long and short sentences is set to l=16, in the new queue B, the lengths of sentences 1, 2-1, 2-2, and 3 are all less than l, so these sentences are directly added to the set of sentences after segmentation; while the length of sentence 4 exceeds l, and dependency syntactic analysis is performed on it, and it is found that there is an adverbial clause introduced by "so that" in the sentence, so the [TA] tag is inserted before "so that", and the sentence is segmented according to the [TA] tag. It is judged again whether the sentence length exceeds l, and it is found that the lengths of "encourage the adoption of all necessary measures, including the promotion of rapid certification" and "so that non-governmental organizations and representative sectors of civil society from all over the world can effectively participate in the preparation process of the World Conference" after segmentation are both longer than l, so an attempt is made to manually find the segmentation point. If there is one, the [TA] tag is inserted, and if not, the segmentation is terminated. In the segmentation of Fig.10 After executing the above operations in queue B, the sentence set after segmentation is obtained as follows Fig.11 Specifically, sentence 1 cannot be split; sentence 4 can be split into sentence 4-1 and sentence 4-2; sentence 2 can be split into sentence 2-1 and sentence 2-2; sentence 3 cannot be split.

[0078] The segmented sentence set is called the fourth data set, which is used for the segmentation execution model, that is, the training of the "long sentence segmentation model". Based on the segmentation analysis of the fourth data set, another data set containing input sentences and corresponding binary labels (separable / inseparable) is obtained, which is called the fifth data set and is used for the training of the segmentation judgment model, that is, the "long sentence segmentation judgment model".

[0079] It can be seen from this embodiment that the natural sentence segmentation punctuation marks, such as period, question mark and exclamation mark, are first used to roughly segment the sentence, and then the component syntactic analysis technology is used to accurately find the comma that can be used as the segmentation point. Then, the dependency syntactic analysis technology is used to find the logically segmentable position in the sentence, and the length l for distinguishing long and short sentences is set to control the degree of segmentation. The data is accurately and effectively preprocessed by combining automation and manual work, and the obtained corpus provides strong support for subsequent model training.

[0080] Definitions of abbreviations and key terms used in the present invention:

[0081] 1) Machine Learning (ML): A branch of artificial intelligence that enables computer systems to automatically improve their performance through experience.

[0082] 2) Deep Learning (DL): A subfield of machine learning that uses a neural network structure similar to the human brain to learn and recognize patterns.

[0083] 3) Neural Network: A computational model that simulates the structure of neurons in the human brain and is used to recognize complex patterns and relationships.

[0084] 4) Sequence Labeling: A task in natural language processing that involves assigning a label to each element in sequence data.

[0085] 5) Sequence-to-Sequence (Seq2Seq): A method for processing sequence data in which the input sequence is converted to an output sequence.

[0086] 6) Pre-trained Language Model (PLM)[1]: A model trained on a large-scale text dataset that can capture the common features of a language.

[0087] 7) Pre-training[1]: refers to the unsupervised learning process of a model before a specific task. The model learns the general features of language based on a large amount of data without focusing on any specific task.

[0088] 8) BART (Bidirectional and Auto-Regressive Transformers)[2]: A pre-trained language model based on the Transformer architecture that combines a bidirectional encoder and an autoregressive decoder for various text generation tasks.

[0089] 9) Denoising Autoencoder (DAE): A neural network that is trained by learning to reconstruct clean data from noisy data.

[0090] 10) Prompt Learning[3]: A method that leverages a pre-trained language model by designing specific prompts or templates to guide the model to complete a specific task without requiring a large number of parameter updates.

[0091] 11) Generalization Ability: refers to the ability of the model to make accurate predictions on unseen data.

[0092] 12) Robustness: refers to the ability of the model to maintain performance in the face of changes or interference in input data

[0093] 13) Data Augmentation Techniques[3]: refers to methods that increase the size and diversity of a dataset by creating data variants.

[0094] 14) Natural Language Processing (NLP): It is a branch of computer science and artificial intelligence that is dedicated to enabling computers to understand, interpret and generate human language.

[0095] 15) Synonym Replacement[4]: A data augmentation technique that creates new training samples by replacing words in a text with their synonyms.

[0096] 16) Sentence Interpolation[4]: A method for generating new sentences by interpolating between two or more existing sentences to create intermediate, synthetic sentences.

[0097] 17) Back-translation[4]: A data augmentation technique that involves translating text from one language to another and then translating it back to the original language to produce training samples that are similar to the original text but slightly different.

[0098] 18) Syntactic Analysis[5]: Syntactic analysis is the process of analyzing the structure and components of a sentence to determine the relationship between words and the grammatical structure of the sentence.

[0099] 19) Syntactic Analysis Tree (Syntax Tree)[5]: A graphical representation used to show the syntactic structure of a sentence. It is usually represented in the form of a tree diagram, where each node represents a sentence component.

[0100] 20) Context Modeling[6]: refers to considering the words and information surrounding a word when processing text to better understand the meaning and usage of the word.

[0101] 21) Semi-Supervised: A learning method that combines labeled and unlabeled data for model training.

[0102] 22) Sentence Alignment: refers to the process of pairing corresponding or related sentences in bilingual texts.

[0103] 23) BERT (Bidirectional Encoder Representations from Transformers)[7]: A pre-trained language model based on the Transformer architecture that learns deep representations of text through bidirectional context encoding.

[0104] 24) Parallel Corpus: It is a collection of texts corresponding to different languages, which are similar in structure and content.

[0105] 25) Interpretability: refers to the degree to which the model’s decision-making process and output results can be understood and explained by humans.

[0106] 26) Penn Chinese Treebank[8]: A large Chinese syntactic analysis treebank that provides detailed syntactic analysis results for research and development of Chinese NLP systems.

[0107] 27) Constituent Parsing[9]: It is a type of syntactic analysis that focuses on identifying the components of a sentence (such as noun phrases, verb phrases, etc.) and their relationships.

[0108] 28) Web Crawler

[10] : A program that automatically visits web pages and extracts information.

[0109] 29) EDA technology (Exploratory Data Analysis) [4]: ​​Exploratory data analysis is a method of analyzing data that aims to discover patterns, anomalies and relationships in the data.

[0110] 30) Dependency Parsing

[11] : It is the process of identifying the dependency relationships between words in a sentence. Unlike componential parsing, dependency parsing focuses on the direct relationship between words rather than their combined structure.

[0111] 31) Prompt Template

[12] : A tool for prompt learning that provides a structured way to design prompts to guide a pre-trained language model to complete a specific task.

[0112] The references used in this invention are listed as follows:

[0113] [1]RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving language understanding by generative pre training[J].URL https: / / s3-us-west-2amazonawscom / openai-assets / research-covers / language-unsupervised / language_understanding_paper pdf, 2018:12.

[0114] [2]MIKE L, YINHAN L, NAMAN G, et al. BART: Denoising Sequence-to-SequencePre-training for Natural Language Generation, Translation, and Comprehension[M]. 2020.

[0115] [3]LIU P,YUAN W,FU J,et al.Pre-train,Prompt,and Predict:A SystematicSurvey of Prompting Methods in Natural Language Processing[J].ACM ComputingSurveys,2021,55(9).

[0116] [4]WEI J,ZOU K.EDA:Easy Data Augmentation Techniques for BoostingPerformance on Text Classification Tasks[M].2019.

[0117] [5] Yang Hongxing, Gu Xin. A review of Chinese syntactic analysis[J]. Shanxi Youth, 2016, (23): 213.

[0118] [6]WICKS R,POST MA unified approach to sentence segmentation ofpunctuated text in many languages[J].Computer Science,2021,2021.acl-long:3995-4007.

[0119] [7]SUBAKAN C,RAVANELLI M,CORNELL S,et al.Attention is All You Need inSpeech Separation[M].2021.

[0120] [8]XIA F. the Penn Chinese Treebank(3.0)[J].mag, 2014.

[0121] [9] Chai Wei. A review of phrase structure syntactic analysis[J]. Computer Knowledge and Technology, 2020, 16(16): 26-7+30.

[0122]

[10] GUILLAUME W,MARIE-ANNE L,ALEXIS C,et al.CCNet:Extracting HighQuality Monolingual Datasets from Web Crawl Data[M].2020.

[0123]

[11] Tu Kewei, Li Jun. A review of the latest developments in syntactic analysis[J]. Journal of Chinese Information Processing, 2020, 34(07): 30-41.

[0124]

[12] COLIN R,NOAM S,ADAM R,et al.Exploring the Limits of TransferLearning with a Unified Text-to-Text Transformer[J].Journal of MachineLearning Research,2020,21(140):140:1-:67.

[0125]

[13]

[0126]

[14] Wei Xilin. Research on bilingual long sentence segmentation method based on deep learning[D], 2019.

[0127] The above embodiments are only preferred implementations of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A Chinese long sentence segmentation method combining syntactic analysis and prompt learning, characterized in that: The following steps are involved: S1: Corpus collection, obtain sentences with boundary information from the corpus, and attach the component syntactic analysis tree of each sentence; Collect a large number of Chinese long sentences in the target field from the Internet, and finally the set of long sentences with boundary information is called the first data set, and the set of long sentences without boundary information is called the second data set. The first data set and the second data set together constitute a first corpus; S2: EDA data enhancement, cleaning and annotating the first corpus, and performing EDA data enhancement, including the following steps: S21: first use the formula n=α*l to determine the number of words that need to be changed in each sentence in the first corpus, where l represents the length of the sentence and α is the proportional coefficient, and apply EDA data enhancement of synonym replacement, random insertion, random exchange and random deletion of n words to each sentence; S22: generating n enhanced sentences for each original sentence in the first corpus, merging the newly generated sentences with the second data set to obtain a third data set, wherein the first data set and the third data set together constitute the second corpus; S3: Data preprocessing, segmenting each sentence in the second corpus, and completing the segmentation in three steps. The queue used in the process is first-in-first-out, including the following steps: S31: Segment each sentence in the second corpus according to the period, question mark, and exclamation mark to obtain a sentence set S, and then determine whether there is a comma in each sentence in the sentence set S. If there is a comma, the sentence is put into queue A; if not, the sentence is put into queue B; S32: the sentence in queue A is removed from the queue, and the sentence is subjected to component syntactic analysis to obtain a component syntactic analysis tree. If the left and right nodes of the comma in the component syntactic analysis tree are both IP, the comma is replaced with a [TA] tag, otherwise the comma is replaced with a [NO] tag, and then the sentence is segmented according to the [TA] tag, and the segmented sentences are put into queue B; the above is repeated until queue A is empty; when obtaining the component syntactic analysis tree, if the sentence source is a long sentence with boundary information, that is, from the first data set, the component syntactic analysis tree attached thereto is directly used; S33: remove the sentence from queue B, replace the [NO] tag in the sentence with a comma, add the sentence to the cut sentence set if the sentence length does not exceed l, perform dependency syntactic analysis on the sentence to obtain a dependency graph, determine whether there is an adverbial clause pattern according to the dependency graph, insert a [TA] tag before the adverbial clause guide word if it does, and manually insert a [TA] tag if it does not, and then segment the sentence according to the [TA] tag; determine again whether the sentence length exceeds l, if it does not exceed, add the sentence to the cut sentence set, if it exceeds, manually insert a [TA] tag, and then segment the sentence according to the [TA] tag, and then add it to the cut sentence set; repeat the above operations until queue B is empty; the cut sentence set will contain the input sentence and the corresponding segmented sentence list, and the cut sentence set is the fourth data set; based on the fourth data set, segmentation and analysis are performed to obtain another data set containing the input sentence and the corresponding binary tag, which is the fifth data set; S4: Prompt template construction, setting two Prompt templates to train two models respectively, the two Prompts are the first Prompt template and the second Prompt template; the first Prompt template is used to determine whether the sentence can be segmented; the second Prompt template is used to segment long sentences; S5: Model training, using the BART pre-trained model to apply the two Prompt templates built in step S4 to train the long sentence segmentation judgment model and the long sentence segmentation model respectively, including the following steps: S51: In the training stage, the vector codes corresponding to the sentences of different templates are input to the encoding layer of the BART model, and the decoder receives the output of the encoder and generates a corresponding output; for the segmentation judgment model, that is, the model for judging whether the long sentence can be segmented, the fifth data set is used for model training; S52: In the training phase, the first Prompt template is applied to each sentence to obtain the corresponding vector code, which is input into the encoding layer of the BART model. The decoder receives the output of the encoder, and a Softmax output layer is set for the output of the decoder to obtain two probability values, corresponding to the probabilities of yes and no respectively. S53: For the segmentation execution model, that is, the long sentence segmentation model, use the fourth data set to perform model training; S54: During the training phase, the second Prompt template is applied to each sentence to obtain the corresponding vector encoding which is input into the encoding layer of the BART model. The decoder receives the output of the encoder, and the output of the decoder is a list of segmented sentences.

2. The Chinese long sentence segmentation method combining syntactic analysis and prompt learning according to claim 1 is characterized by: The corpus in step S1 is a phrase structure syntactic annotated corpus.