Chapter topic segmentation model training method, electronic device and storage medium

Abstract

The present invention discloses a chapter theme segmentation model training method, a chapter theme segmentation system, an electronic device and a storage medium, wherein a chapter theme segmentation model training method comprises: using a pointer network to replace the crf layer in the sequence annotation model in the segmentation task; and combining the index value generated by the pointer network to perform text classification in the classification task. By using a pointer network to replace the crf layer in the sequence annotation model in the segmentation task, and then combining the index value generated by the pointer network to perform text classification in the classification task, it can be applied to most fields of natural language processing, improve the chapter segmentation performance, and further, can also help relevant field personnel quickly understand the basic content in super large texts and generate relevant reports, and can detect the rationality of the overall structure of the document and automatically process and analyze the massive text information data derived in the big data era.

Description

Technical Field

[0001] The present invention belongs to the technical field of topic segmentation, and in particular relates to a chapter topic segmentation model training method, a chapter topic segmentation system, an electronic device and a storage medium. Background Art

[0002] There are generally three types of topic segmentation algorithms, including topic segmentation algorithms based on rule matching, topic segmentation algorithms based on unsupervised models such as LSA and HMM, and topic segmentation algorithms based on deep models.

[0003] Among them, the topic segmentation algorithm based on rule matching: The essence of chapter topic segmentation is to divide a document into multiple paragraphs with specific topics, so that readers can understand the structure of the article at a glance. The most common method to achieve this task is rule matching. The basic principle of this method is: first, manually design a rule matching table to build a mapping between different regular expressions and specific topics. Secondly, all sentences are matched through manually designed regular expressions. Assuming that the basic form in the rule is met in the sentence, it is mapped to a specific topic. Finally, according to the topics generated by the sentence mapping, the paragraphs are merged according to the continuity of the context topic. Assuming that multiple sentences with the same topic appear continuously in the document, they are merged and marked as paragraphs of this topic. After the merger is completed, the entire document can be divided into paragraphs containing multiple specific topics. The main defect is that the design of the rule topic matching table is difficult and requires professionals in this field. After the design is completed, it can only be used in this field. The secondary defect is that the paragraphs segmented according to the rules are relatively scattered, and simple documents are segmented into a large number of fragmented topics, which makes it difficult to distinguish and clarify the basic structure of the document.

[0004] Topic segmentation algorithms based on unsupervised models such as LSA and HMM: The basic idea of ​​this type of method is that sentences with the same topic satisfy similar vocabulary distributions. In short, when there is a place in the document where the vocabulary distribution of context sentences is inconsistent, it is a segmentation point. Based on the above idea, this type of method uses models such as LSA to vectorize sentences, calculates the similarity scores between adjacent sentences to obtain the segmentation position. At the same time, with the help of the sequential nature of models such as HMM, it can continuously iterate on the basis of a given initial state sequence to obtain the target topic segmentation sequence, and complete the chapter topic segmentation task according to the final segmentation sequence. The key problem of this type of method is that it can identify the paragraph segmentation position but it is difficult to distinguish the topic type of the paragraph. Of course, the common problems of unsupervised algorithm models: difficulty in setting thresholds and inability to introduce external resources are also the main contradictions that hinder its effectiveness.

[0005] Topic segmentation algorithm based on deep model: Chapter-level topic segmentation is essentially to classify each sentence and complete the task according to the similarity and continuity of the categories. Therefore, this problem is usually abstracted as the most classic sequence labeling task in NLP, and each sentence is labeled using the IOB labeling format. In order to distinguish the types of different sentences, a sentence type information will be spliced ​​after the IOB label, that is: B-LOCATION, I-LOCATION, etc. After completing the manual labeling, the BiLSTM+CRF, Bert+CRF and other architectures commonly used in sequence labeling are used for deep model training. After the network parameters fit the data rules, the topic distribution information in the new data can be predicted. The common problem of sequence labeling methods is that the effect is poor in multi-label classification tasks, and the model is difficult to converge in scenarios with a large number of topic types. Another problem is that the essence of sequence labeling is classification, and the model optimization goal is still the accuracy of each sentence category, which cannot optimize the effect of the segmentation process. Summary of the invention

[0006] The embodiments of the present invention provide a chapter topic segmentation model training method and a chapter topic segmentation system and device, which are used to solve at least one of the above-mentioned technical problems.

[0007] In a first aspect, an embodiment of the present invention provides a method for training a chapter topic segmentation model, comprising: using a pointer network to replace the crf layer in the sequence labeling model in the segmentation task; and performing text classification in the classification task in combination with the index value generated by the pointer network.

[0008] In the second aspect, an embodiment of the present invention provides a chapter topic segmentation system, including: a word segmentation and word vector generation model, used to perform word segmentation and word vector generation on training data; a sentence vector generation model, used to generate sentence vectors based on the generated word vectors; a segmentation model, used to segment the sentence vectors according to the method described in any one of claims 1-7; and a classification model, used to classify based on the segmentation results according to the method described in any one of claims 1-7.

[0009] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the chapter topic segmentation model training method and the chapter topic segmentation system of any embodiment of the present invention.

[0010] In a fourth aspect, an embodiment of the present invention further provides a computer program product, comprising a computer program stored on a non-volatile computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer executes the steps of the chapter topic segmentation model training method and the chapter topic segmentation system of any embodiment of the present invention.

[0011] The method, system, electronic device and storage medium of the present application adopt a pointer network to replace the CRF layer in the sequence labeling model in the segmentation task, and then perform text classification in combination with the index value generated by the pointer network in the classification task, so that it can be applied to most fields of natural language processing and improve the performance of chapter segmentation. Furthermore, it can also help people in related fields to quickly understand the basic content in ultra-large texts and generate relevant reports, and can detect the rationality of the overall structure of the document and automatically process and analyze the massive text information data derived in the big data era. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A flowchart of a chapter topic segmentation model training method provided by one embodiment of the present invention;

[0014] Figure 2 A flowchart of another chapter topic segmentation model training method provided by one embodiment of the present invention;

[0015] Figure 3 A flow chart of another chapter topic segmentation model training method provided by one embodiment of the present invention;

[0016] Figure 4 A flowchart of another method for training a chapter topic segmentation model provided by an embodiment of the present invention;

[0017] Figure 5 A block diagram of a chapter topic segmentation system provided by one embodiment of the present invention;

[0018] Figure 6 A basic algorithm flow chart of a specific example of a chapter theme segmentation model training method and a chapter theme segmentation system provided by an embodiment of the present invention;

[0019] Figure 7A model overall architecture diagram of a specific example of a chapter theme segmentation model training method and a chapter theme segmentation system provided in one embodiment of the present invention;

[0020] Figure 8 A comparison diagram of the effects of a specific example of a chapter theme segmentation model training method and a chapter theme segmentation system provided in one embodiment of the present invention;

[0021] Fig. 9 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Please refer to Figure 1 , which shows a flowchart of an embodiment of a chapter topic segmentation model training method of the present application, wherein the chapter topic segmentation model includes a segmentation task and a classification task, and the segmentation task includes using a sequence labeling model.

[0024] like Figure 1 As shown, in step 101, a pointer network is used in the segmentation task to replace the crf layer in the sequence labeling model;

[0025] In step 102, text classification is performed in the classification task in combination with the index value generated by the pointer network.

[0026] In this embodiment, for step 101, the chapter topic segmentation model training device uses a pointer network to replace the crf layer in the sequence labeling model in the segmentation task. For example, the encoder module in the Seq2Seq model is used as the basic architecture of the segmentation task. The model uses the generated sentence vector as input, and generates a hidden layer output consistent with the document length through the BiLSTM network. Then, with the help of the idea of ​​the pointer network, the hidden layer output of the encoder part and the output of the decoder part are processed by the attention mechanism, and the location of the document segmentation point is determined according to the position of the node with the highest score in the attention mechanism layer.

[0027] For step 102, the chapter topic segmentation model training device combines the index value generated by the pointer network to perform text classification in the classification task. For example, the corresponding elements in the original sentence vector matrix and the encoder hidden layer output are taken according to the segmented index value, and the input of the decoder module for the next time step is generated by splicing and pooling.

[0028] The method of this embodiment uses a pointer network to replace the CRF layer in the sequence labeling model in the segmentation task, and then combines the index value generated by the pointer network to perform text classification in the classification task. This can be applied to most fields of natural language processing and improve the performance of paragraph segmentation. Furthermore, it can also help people in related fields to quickly understand the basic content in ultra-large texts and generate relevant reports, and can detect the rationality of the overall structure of the document and automatically process and analyze the massive text information data derived in the big data era.

[0029] Further references Figure 2 , which shows a flowchart of another chapter theme segmentation model training method provided by an embodiment of the present application. Figure 1 A flowchart of the steps further defined in the process of "using a pointer network to replace the crf layer in the sequence labeling model in the segmentation task", wherein the sequence labeling model includes an encoder module and a decoder module, and the basic framework of the segmentation task is the encoder module in the sequence labeling model.

[0030] like Figure 2 As shown, in step 201, the hidden layer output of the encoder module and the output of the decoder module are processed by an attention mechanism based on a pointer network;

[0031] In step 202, the document segmentation point is determined according to the node position with the highest score in the attention mechanism layer.

[0032] In this embodiment, for step 201, the chapter topic segmentation model training device performs attention mechanism processing on the hidden layer output of the encoder module and the output of the decoder module based on the pointer network; then, for step 202, the chapter topic segmentation model training device determines the document segmentation point according to the node position with the highest score in the attention mechanism layer, for example, a hidden layer output consistent with the document length is generated through a BiLSTM network, and then, with the help of the idea of ​​the pointer network, the hidden layer output of the encoder part and the output of the decoder part are processed by the attention mechanism, and the location of the document segmentation point is determined according to the node position with the highest score in the attention mechanism layer.

[0033] The method of this embodiment processes the hidden layer output of the encoder module and the output of the decoder module through an attention mechanism based on a pointer network, and then determines the document segmentation point according to the node position with the highest score in the attention mechanism layer, thereby improving the chapter segmentation performance.

[0034] In the method of the above embodiment, the merging and classifying of texts in the classification task in combination with the index value generated by the pointer network includes:

[0035] According to the segmentation points of the segmentation task, the corresponding elements in the original sentence vector matrix and the hidden layer output of the encoder module are taken, and the input of the decoder module for the next time step is generated by splicing and pooling, and the input of the decoder module for each time step is continuously generated, wherein each input includes the feature information of each topic paragraph.

[0036] The method of this embodiment takes the corresponding elements in the original sentence vector matrix and the hidden layer output of the encoder module according to the segmentation points of the segmentation task, generates the input of the decoder module for the next time step by splicing and pooling, and continuously generates the input of the decoder module at each time step, thereby improving the chapter segmentation performance.

[0037] Further references Figure 3 , which shows a flowchart of another chapter topic segmentation model training method provided by an embodiment of the present application. This flowchart is mainly a flowchart of the steps further defining the process of the above embodiment.

[0038] like Figure 3 As shown, in step 301, the segmentation loss function of the segmentation task is obtained by comparing the document segmentation points determined based on the segmentation task with the original labels;

[0039] In step 302, a classification loss function generated in the classification process is calculated based on the probability distribution for each category determined by the classification task;

[0040] In step 303, a multi-task loss function is formed based on the segmentation loss function and the classification loss function;

[0041] In step 304, the parameters of the chapter topic segmentation model are updated based on the multi-task loss function.

[0042] The method of this embodiment can improve the accuracy of the model by forming a multi-task loss function.

[0043] Further references Figure 4 , which shows a flowchart of another chapter theme segmentation model training method provided by an embodiment of the present application. Figure 1A flowchart of the steps that are further defined by the process before the "Segmentation Task".

[0044] like Figure 4 As shown, in step 401, the training data is segmented and word vectors are generated;

[0045] In step 402, sentence vectors are generated based on the generated word vectors.

[0046] In this embodiment, for step 401, the chapter topic segmentation model training device performs word segmentation and word vector generation on the training data. For example, 1 million field-related dialogue texts are extracted, and the Jieba word segmentation method commonly used in the industry is adopted. After the word segmentation is completed, the GloVe model is used to self-train the Chinese word vector applicable in the field, and finally a matrix with a vocabulary size of 16w and a word vector dimension of 300 is generated, which is used as the initialization input of the model.

[0047] For step 402, the chapter topic segmentation model training device generates sentence vectors based on the generated word vectors. For example, the generated GloVe word vectors are used as the initialization representation of the sentence vectors, which are input into the BiLSTM model to generate sentence vectors.

[0048] The method of this embodiment performs word segmentation and word vector generation on the training data, and then generates sentence vectors based on the generated word vectors, thereby enhancing the robustness and adaptability of the sentence vector generation process.

[0049] In the method of the above embodiment, the sentence vector generation based on the generated word vector includes generating a sentence vector using a sentence vector generation model, and the sentence vector generation model includes a BiLSTM model. For example, all hidden layer outputs of the BiLSTM are subjected to maximum pooling, average pooling, and the output of the last time step is obtained, and the three types of results are finally spliced ​​together as the final sentence vector representation.

[0050] The method of this embodiment generates sentence vectors by using the BiLSTM model, thereby enhancing the robustness and adaptability of the sentence vector generation process.

[0051] In the method of the above embodiment, it also includes: jointly training the sentence vector generation model with the segmentation task and the classification task to simultaneously optimize the model parameters of each model.

[0052] The method of this embodiment jointly trains the sentence vector generation model with the segmentation task and the classification task, thereby optimizing the model parameters of each model at the same time.

[0053] Please refer to Figure 5 , which shows a block diagram of a chapter topic segmentation system provided by an embodiment of the present invention.

[0054] like Figure 5 As shown, the article topic segmentation system 500 includes a word segmentation and word vector generation model 510, a sentence vector generation model 520, a segmentation model 530 and a classification model 540.

[0055] Among them, the word segmentation and word vector generation model 510 is used to perform word segmentation and word vector generation on training data; the sentence vector generation model 520 is used to generate sentence vectors based on the generated word vectors; the segmentation model 530 is used to segment the sentence vectors according to the method described in any one of claims 1-7; and the classification model 540 is used to classify based on the segmentation results according to the method described in any one of claims 1-7.

[0056] It should be noted that the above method steps are not used to limit the execution order of each step. In fact, some steps may be executed simultaneously or in the opposite order to the specified step order, and this application has no limitation on this.

[0057] The following describes some problems encountered by the inventor in the process of implementing the present invention and a specific embodiment of the solution finally determined, so that those skilled in the art can better understand the solution of the present application.

[0058] The inventors found in the process of implementing this application that the defects in the prior art are mainly caused by the following reasons:

[0059] Topic segmentation algorithm based on rule matching: The difficulty in designing a topic rule matching table is because the topics represented by the text in different fields are different. Therefore, it is necessary to manually read a large amount of conversation text in this field to summarize what kind of text can represent a specific topic. Another reason for the scattered topic segmentation is that adjacent sentences often have multiple different keywords, and these keywords correspond to different topics, which makes it impossible to merge these adjacent sentences, resulting in a topic sequence similar to ABABAB. This scattered sequence is not the result desired by the segmentation model.

[0060] Topic segmentation algorithms based on unsupervised models such as LSA and HMM: The core idea of ​​this type of algorithm is that paragraphs with the same topic have similar vocabulary distribution characteristics. This characteristic can only be used to distinguish the differences in topics between paragraphs, but cannot identify which type of topic these different paragraphs represent. There is also the problem of threshold setting inherent in unsupervised learning, specifically the inability to accurately determine the impact of the degree of difference between adjacent sentences on the location of the segmentation point. At the same time, the optimization space of this type of algorithm is also limited, and it is impossible to integrate effective strategies into the model.

[0061] Topic segmentation algorithm based on deep model: According to the particularity of CRF layer, if there are 1000 types of labels, the model calculates the transfer matrix of dimension 1000*1000 at each time step. This process requires a huge cost, so the model is usually in a state of difficult convergence and high memory consumption. Another problem is that the effect of the segmentation process cannot be optimized, because the loss of sequence labeling focuses on the differences of each sentence category, and a small effect on the classification of each sentence does not mean a good effect of article segmentation.

[0062] The inventors also found that for the defects in the topic segmentation algorithm based on rule matching, the common method is to add and optimize rules and optimize the merging strategy (for example, merge the trivial topics in the middle according to the length of the sentence); for the defects in the topic segmentation algorithm based on unsupervised models such as LSA and HMM, the common method is to integrate the supervised classification model on the basis of unsupervised to solve the recognition problem; the problem of the topic segmentation algorithm based on the deep model is usually solved by reducing the size of the label space or using softmax to replace the crf layer. The first difficult point is that the traditional method does not consider the problem of chapter topic segmentation from the perspective of multi-task. In essence, this task needs to go through two stages: segmentation + classification, and two tasks need to be optimized simultaneously during the modeling process. How to elegantly combine these two tasks is also difficult to think of in the existing methods. Because the loss calculation methods of these two tasks are completely inconsistent, it is also difficult to integrate the two tasks in the same model. The third difficult point to consider is to introduce a new network structure (pointer network) to replace the crf layer to complete the segmentation task. The pointer network is a network structure proposed by the top artificial intelligence conference NIPS 2015. At present, there are few related works introduced in topic segmentation. The last difficult point to consider is the topic segmentation task at the paragraph level. How to effectively organize sentence vectors and how to reasonably design the Pipline are also difficult to think of.

[0063] The solution of this application is designed and optimized mainly from the following aspects:

[0064] In order to solve the trivial topic segmentation problem in the rule matching strategy, the training data of this algorithm is manually labeled. Since manually labeled data is more in line with the intuitive segmentation method, the segmented paragraphs are mainly large paragraphs, and the learning results based on this data will not be scattered or fragmented.

[0065] To address the problem that unsupervised learning cannot identify the topic type of a paragraph, this algorithm adds a special classification network to identify the paragraphs after segmentation.

[0066] To address the problems of high computational cost and poor segmentation effect of the third sequence labeling model CRF, we use a pointer network to replace the original CRF layer and use the idea of ​​multi-task learning to simultaneously optimize both segmentation and classification objectives, which can solve the above two problems well.

[0067] The main goal of the chapter-level topic segmentation algorithm is to divide a given article into multiple paragraphs with different topics, and the sentences contained in the paragraphs have similar topic distributions. The current SOTA method for topic segmentation tasks is a sequence labeling algorithm based on a deep model. However, the problems of high CRF layer cost and poor segmentation effect in sequence labeling are inherent and difficult to solve defects of the model. Based on this, the embodiment of this application proposes a new chapter-level topic segmentation method. Please refer to the process. Figure 6 , which shows a basic algorithm flow chart of a specific example of a chapter theme segmentation model training method and a chapter theme segmentation system provided by an embodiment of the present invention, such as Figure 6 As shown in Figure 1, the algorithm mainly consists of five steps. The word segmentation and word vector generation parts adopt the industry-recognized methods, which are not very innovative. Therefore, we will only briefly introduce the process here without further elaboration. The sentence vector generation model is jointly trained with the subsequent segmentation and classification tasks to form a complete pipeline (please refer to Figure 7 , which shows a model overall architecture diagram of a specific example of a chapter topic segmentation model training method and a chapter topic segmentation system provided by an embodiment of the present invention). The segmentation and classification tasks each constitute the Encoder and Decoder modules of Seq2Seq, where the main innovation is that the segmentation task uses a pointer network to replace the crf layer in the sequence labeling, and the classification task combines the index generated by the pointer network and the output of the Encoder part to merge and classify the text. Finally, in order to promote the joint training of multiple tasks, the embodiment of the present application designs a multi-objective loss function as an optimization indicator of the model.

[0068] Step 1: Word segmentation and word vector generation: This algorithm extracts 1 million domain-related conversation texts and adopts the Jieba word segmentation method commonly used in the industry. After word segmentation, the GloVe model is used to self-train Chinese word vectors applicable in the field. Finally, a matrix with a vocabulary size of 16w and a word vector dimension of 300 is generated, which is used as the initialization input of the model.

[0069] Step 2: Sentence vector generation: Based on the GloVe word vector generated above, the sentence vector is initialized and input into the BiLSTM model to generate the sentence vector (e.g. Figure 7 As shown in the Sentence Embedding Generation module, W iIn order to fully capture the information inside the sentence, this application performs maximum pooling, average pooling and the output of the last time step on all the hidden layer outputs of BiLSTM, and finally concatenates the three types of results together as the final sentence vector representation (such as Figure 6 As shown, S i Represents sentence vector). Due to the importance of sentence vectors to downstream tasks, the embodiment of the present application combines the sentence vector generation model with the two subsequent tasks and optimizes the parameters in the three models at the same time, so as to enhance the robustness and adaptability of the sentence vector generation process.

[0070] Step 3: Segmentation tasks: Figure 7 As shown in the Task 1 section, this application uses the Encoder module in the Seq2Seq model as the basic architecture for the segmentation task. i As input, the BiLSTM network generates a hidden layer output that is consistent with the document length (the number of sentences in the document). After that, with the help of the idea of ​​pointer network, the hidden layer output of the Encoder part and the output of the Decoder part are processed by Attention (attention mechanism), and the location of the document segmentation point is determined according to the position of the node with the highest score in the Attention layer. For example, suppose there are five sentences in the article, which are represented by sentence vectors [S1, S2, S3, S4, S5] as Encoder input, and the hidden layer outputs are [E1, E2, E3, E4, E5]. The Decoder part uses random values. <start>The output value of the last time step of the Encoder is used as input to obtain the initial output D1. D1 and [E1, E2, E3, E4, E5] are processed as Attention, and the softmax function is used to calculate the effect of D1 on each E i The highest score is taken as the position of the split point, such as Figure 7 Assume that the red node E2 has the highest score. After obtaining the first segment point position, it can be inferred that E3 is the beginning of the next segment. Similarly, the output of the second node of the Decoder is processed with [E3, E4, E5] for Attention, as shown below: Figure 7 Assume that E5 is the segmentation point. Similarly, we finally get <2, 5> as the position of the two segmentation points. By comparing with the original label, we can calculate the loss of the segmentation process. seg .

[0071] Step 4 Classification Task: After completing the segmentation task, take the corresponding elements in the original sentence vector matrix and the encoder hidden layer output according to the segment index value, and generate the input of the next time step of the decoder module by concatenation and pooling. It can be seen that the entire training and testing process is completed in the Teacher Forcing way. Figure 7 For example, suppose the segmentation result is obtained according to the first time step input of Decoder: <2> , it can be inferred that the two sentences S1 and S2 constitute the first part of the topic paragraph. Therefore, the input of the Decoder in the second time step is the aggregation of the sentence vectors (S1, S2) of the first part of the topic paragraph. The specific method is to splice and pool the four groups of values ​​of the original sentence vectors S1, S2 (that is, the first BiLSTM output) and the Encoder hidden layer output vector (E1, E2). Similarly, the inputs of the Decoder at each time step are continuously generated. These inputs contain the feature information of each topic paragraph. Finally, a fully connected layer is connected to the outermost edge of the Decoder, and the final vector is generated through the softmax function to generate the probability distribution for each category, so as to calculate the loss generated in the classification process. label .

[0072] The multi-task loss function is designed based on the two losses generated by the above segmentation and classification processes. The update of the model parameters aims to optimize the following loss, where changing the value α can adjust the importance of the classification and segmentation tasks, and θ represents all the parameters in the entire model:

[0073] L(θ)=α·loss seg (y seg |x;θ)+(1-α)·loss label (y label |x;θ)

[0074] Figure 8 A comparison diagram of the effects of a specific example of a chapter topic segmentation model training method and a chapter topic segmentation system provided in one embodiment of the present invention.

[0075] like Figure 8 As shown in the figure, this application implements the topic segmentation algorithm based on rule matching, the topic segmentation algorithm based on unsupervised models such as LSA and HMM, and the topic segmentation algorithm based on deep model respectively. Here, two evaluation indicators are used to measure the effect of each model, where P k The value (the smaller the value, the better the effect) is used to measure the effect of model segmentation, while the F1 value (the larger the value, the better the effect) measures the effect of model classification. As can be seen from the table, the method proposed in the embodiment of the present application exceeds the current SOTA sequence labeling model and is optimal because of the multi-task learning idea and pointer network strategy.

[0076] The solution to this chapter-level topic segmentation task can be applied to most areas of natural language processing. The direct effect that can be achieved in the process of optimizing this model is the improvement of the F1 value of topic identification and the performance of chapter segmentation. On a deeper level, it can help personnel in related fields (medical, literature, etc.) quickly understand the basic content of ultra-large texts and generate relevant reports, detect the rationality of the overall structure of the document, and automatically process and analyze the massive text information data derived from the big data era.

[0077] Beta version formed by the inventor in the process of realizing the present invention:

[0078] In the actual development process, we considered isolating the two tasks of segmentation and classification into two models for training. After training, the sentences are segmented first and then the segmented results are sent to classification. The advantage of this solution is that the two are separated, and the model can be used in both classification and segmentation scenarios, which is convenient for transplantation to other similar tasks. However, due to the separate optimization of network parameters, the two models can only achieve the optimal value of their respective optimization goals, and both cannot achieve the best at the same time. Moreover, when the segmentation model obtains relatively poor results, it has a greater impact on subsequent downstream tasks (classification), and the dependency between the two cannot be eliminated.

[0079] One feasible beta version is to remove the pointer network and simply use the output of BiLSTM in the Encoder module to generate segmentation results. The advantage of this process is that it does not need to use the output of the Decoder as the basis for segmentation, and the time performance is improved to a certain extent. However, the main disadvantage is that the previous and subsequent segmentation results cannot be directly linked, the network convergence speed is slow, and the final effect is not as good as the pointer network.

[0080] In some other embodiments, the present invention also provides a non-volatile computer storage medium, the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the chapter topic segmentation model training method and chapter topic segmentation system in any of the above method embodiments;

[0081] As an implementation mode, the non-volatile computer storage medium of the present invention stores computer executable instructions, and the computer executable instructions are configured as follows:

[0082] In the segmentation task, a pointer network is used to replace the crf layer in the sequence labeling model;

[0083] In the classification task, the index value generated by the pointer network is combined to perform text classification.

[0084] The non-volatile computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the chapter theme segmentation model training device and the chapter theme segmentation system, etc. In addition, the non-volatile computer-readable storage medium may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the chapter theme segmentation model training device and the chapter theme segmentation system via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0085] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any one of the above-mentioned chapter topic segmentation model training methods and chapter topic segmentation systems.

[0086] Fig. 9 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Fig. 9 As shown, the device includes: one or more processors 910 and a memory 920, Fig. 9 A processor 910 is used as an example. The apparatus of the article theme segmentation model training method and the article theme segmentation system may also include: an input device 930 and an output device 940. The processor 910, the memory 920, the input device 930 and the output device 940 may be connected via a bus or other means. Fig. 9 The example of connecting through a bus is taken. The memory 920 is the above-mentioned non-volatile computer-readable storage medium. The processor 910 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 920, that is, the chapter theme segmentation model training method and the chapter theme segmentation system of the above-mentioned method embodiment are realized. The input device 930 can receive input digital or character information, and generate key signal input related to the user setting and function control of the communication compensation device. The output device 940 may include display devices such as display screens.

[0087] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0088] As an implementation mode, the electronic device is applied to a text topic segmentation model training device and a text topic segmentation system, and includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can:

[0089] In the segmentation task, a pointer network is used to replace the crf layer in the sequence labeling model;

[0090] In the classification task, the index value generated by the pointer network is combined to perform text classification.

[0091] The electronic device of the embodiment of the present application exists in various forms, including but not limited to:

[0092] (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0093] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0094] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0095] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0096] (5) Other electronic devices with data interaction functions.

[0097] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.< / start>

Claims

1. A method for training a chapter topic segmentation model, wherein: The chapter topic segmentation model includes a segmentation task and a classification task, the segmentation task includes using a sequence labeling model, and the method includes: Using a pointer network to replace the crf layer in the sequence labeling model in the segmentation task includes: performing attention mechanism processing on the hidden layer output of the encoder module and the output of the decoder module based on the pointer network; determining the document segmentation point according to the node position with the highest score in the attention mechanism layer; In the classification task, text classification is performed in combination with the index value generated by the pointer network, including: taking the corresponding elements in the original sentence vector matrix and the hidden layer output of the encoder module according to the segmentation points of the segmentation task, generating the input of the decoder module for the next time step by splicing and pooling, and continuously generating the input of the decoder module at each time step, each of which includes the feature information of each topic paragraph, and finally, connecting a fully connected layer to the end of the decoder, and generating the probability distribution for each category through the softmax function of the final vector.

2. The method according to claim 1, further comprising: Comparing the document segmentation points determined based on the segmentation task with the original labels to obtain a segmentation loss function of the segmentation task; Calculating a classification loss function generated in the classification process based on the probability distribution of each category determined by the classification task; Forming a multi-task loss function based on the segmentation loss function and the classification loss function; The parameters of the chapter topic segmentation model are updated based on the multi-task loss function.

3. The method according to claim 1, before the segmentation task, the method further comprises: Perform word segmentation and word vector generation on training data; Generate sentence vectors based on the generated word vectors.

4. The method according to claim 3, wherein: The generating sentence vectors based on the generated word vectors includes generating sentence vectors using a sentence vector generation model, and the sentence vector generation model includes a BiLSTM model.

5. The method according to claim 4, further comprising: The sentence vector generation model is jointly trained with the segmentation task and the classification task to simultaneously optimize model parameters of each model.

6. A system for segmenting a chapter theme, comprising: Word segmentation and word vector generation model, used to segment and generate word vectors for training data; Sentence vector generation model, used to generate sentence vectors based on generated word vectors; A segmentation model, used for segmenting the sentence vector according to the method described in any one of claims 1 to 5; as well as A classification model, used for classifying based on segmentation results according to the method of any one of claims 1-5.

7. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method described in any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images