A text summarization method, a text summarization device, and a computer-readable storage medium

Abstract

The application discloses a text summarization method, a text summarization device and a computer readable storage medium. The method comprises the following steps: obtaining first text information and a text summarization model, wherein the text summarization model comprises a key information extraction network, an embedding layer and an extraction generation network; processing the first text information by using the key information extraction network to obtain first key information; mapping the first key information into a first key vector sequence by using the embedding layer; and processing the first key vector sequence by using the extraction generation network to obtain first summary information. In this way, the effect of text summarization can be optimized.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a text summarization method, a text summarization device, and a computer-readable storage medium. Background Technology

[0002] Customer service is the primary form of communication between companies / organizations and users. Each call is recorded as a work order, and the analysis and mining of work order information is crucial for improving efficiency. Through long-term research, the inventors of this application have discovered that Natural Language Processing (NLP) technology can significantly improve efficiency and help customers automate the analysis of work orders, such as categorizing, summarizing, or abstracting them. However, in practical use, many technical challenges remain, such as the lengthy dialogue content in work order information, which drastically increases the difficulty of NLP analysis and mining. Summary of the Invention

[0003] This application provides a text summarization method, a text summarization device, and a computer-readable storage medium, which can optimize the effect of text summarization.

[0004] To solve the above-mentioned technical problems, the technical solution adopted in this application is: to provide a text summarization method, the method comprising: acquiring first text information and a text summarization model, the text summarization model comprising a key information extraction network, an embedding layer and an extraction generation network; processing the first text information using the key information extraction network to obtain first key information; mapping the first key information into a first key vector sequence using the embedding layer; and processing the first key vector sequence using the extraction generation network to obtain first summary information.

[0005] To solve the above-mentioned technical problems, the technical solution adopted in this application is as follows: a text summarization device is provided, which includes an acquisition module and a processing module. The acquisition module is used for first text information and a text summarization model. The text summarization model includes a key information extraction network, an embedding layer, and an extraction generation network. The processing module is used to process the first text information using the key information extraction network to obtain first key information; to map the first key information into a first key vector sequence using the embedding layer; and to process the first key vector sequence using the extraction generation network to obtain first summary information.

[0006] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a text summarization device, which includes a memory and a processor connected to each other. The memory is used to store a computer program, and when the computer program is executed by the processor, it is used to implement the text summarization method in the above-mentioned technical solution.

[0007] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium for storing a computer program, which, when executed by a processor, is used to implement the text summarization method in the above-mentioned technical solution.

[0008] The beneficial effects of this application through the above scheme are as follows: First, first text information and a text summary model are obtained. This text summary model includes a key information extraction network, an embedding layer, and an extraction and generation network. Then, the first text information is input into the key information extraction network to generate first key information. Next, the first key information is input into the embedding layer to generate a first key vector sequence. Finally, the first key vector sequence is input into the extraction and generation network to generate first summary information. Because a key information extraction network is added before the embedding layer, and this network has the function of locating the more critical information in the input text, it can effectively solve the problem that long input text leads to more interference, resulting in higher learning costs and poorer summary effects. This effectively shortens the time spent on summarizing and improves the accuracy of the summary, thus optimizing the text summary. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0010] Figure 1 This is a flowchart illustrating an embodiment of the text summarization method provided in this application;

[0011] Figure 2 This is a structural diagram of the text summarization model provided in this application;

[0012] Figure 3 This is a schematic diagram of the first text information and the first key information provided in this application;

[0013] Figure 4 This is a flowchart illustrating another embodiment of the text summarization method provided in this application;

[0014] Figure 5 This is a schematic diagram of the key information extraction network provided in this application;

[0015] Figure 6 This is a schematic diagram of the structure of an embodiment of the text summarization device provided in this application;

[0016] Figure 7 This is a schematic diagram of another embodiment of the text summarization device provided in this application;

[0017] Figure 8 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0019] Technical solutions for work order summaries can be divided into two categories: extraction-based models, which extract paragraphs from the original input text as the summary result, and generative models, which automatically generate the summary result based on network models such as Transformer. For long text processing, to avoid noise interference and ensure summary effectiveness, a long text truncation solution is used, assuming that key customer call information is at the beginning of the work order information, thus uniformly truncating the text. This approach simply and effectively solves problems in some specific domains and achieves good results. However, for customer service work orders in certain domains, the work order content is often particularly lengthy, and key information is not necessarily at the beginning. If a simple and crude long text truncation method is used, it is easy to lose key information and introduce more invalid and distracting information, thus resulting in poor analysis and mining effects. Therefore, these solutions cannot effectively locate key information in the input work order text and often easily lose key information.

[0020] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the text summarization method provided in this application. The method includes:

[0021] Step 11: Obtain the first text information and the text summary model.

[0022] The text summarization model is a pre-trained learning model, such as... Figure 2 As shown, the text summary model includes a key information extraction network, an embedding layer, and an extraction and generation network. The first text information can be input into the text summary model, which processes the first text information to obtain the first summary information. The first text information can be work order information or article information.

[0023] Step 12: Use a key information extraction network to process the first text information to obtain the first key information.

[0024] After obtaining the first text information, it can be input into the key information extraction network in the text summarization model. This key information extraction network can process the first text information and extract the more critical information (i.e., the first key information). For example, as shown in Figure 3, the first text information is "Hello, I would like to inquire about how to apply for a bank loan? Okay, just a moment. Have you ever done business at our bank before? No, this is my first time here." The first key information is "Inquire about how to apply for a bank loan."

[0025] Step 13: Map the first key information into a first key vector sequence using an embedding layer.

[0026] After extracting the first key information from the first text information, the first key information can be input into the embedding layer. The embedding layer processes the first key information to convert it into a vector, thus obtaining a first key vector sequence.

[0027] Step 14: Use an extraction-generating network to process the first key vector sequence to obtain the first summary information.

[0028] After obtaining the first key vector sequence, the first key vector sequence can be input into the extraction and generation network so that the extraction and generation network processes the first key vector sequence to generate the first summary information. The first text information can be a part of the first summary information; for example, suppose the first text information is: "How much is the cost of the mobile phone card package", and the first summary information is: "Mobile phone package cost".

[0029] Furthermore, the extraction-generative network can be an extraction-based text summarization model, a generative text summarization model, or a fusion of these two models (i.e., a fusion model).

[0030] This embodiment proposes an optimized text summary scheme for long texts, specifically for work order summary technology. By introducing a key information extraction network into the network structure of the text summary model, the efficiency and accuracy of key information location and extraction are improved. This reduces invalid information in the input text, minimizes the interference of invalid information on the summary effect, and allows the text summary model to automatically adapt to text information of various lengths, thereby improving the problem of poor summary effect for long texts.

[0031] Please see Figure 4 , Figure 4 This is a flowchart illustrating another embodiment of the text summarization method provided in this application, the method comprising:

[0032] Step 41: Obtain audio data and convert it into second text information.

[0033] In practical applications, the raw data obtained may be audio data. The audio data is transcribed into text data (i.e., second text information) through speech-to-text technology to meet the usage requirements of the text summary model.

[0034] Step 42: Preprocess the second text information to obtain the first text information.

[0035] After obtaining the transcribed second text information, the second text information can be preprocessed to generate the first text information. Specifically, considering issues such as data transcription quality and the quality of the data itself, the format and quality of the second text information can be verified and preprocessed to ensure data reliability.

[0036] Furthermore, the format of the second text information is checked to determine if it conforms to a preset format. If it does, the second text information is converted to the preset format to obtain the first text information. Alternatively, since audio data converted to text data may be inaccurate, manual verification can be used to validate the second text information, thereby verifying the quality of the text data and ensuring that the text data input into the text summarization model is accurate.

[0037] Step 43: Use a key information extraction network to process the first text information to obtain the first key information.

[0038] The first text information is input into a key information extraction network to obtain the first key information. This key information extraction network can be a feedforward network. Specifically, for example... Figure 5 As shown, the key information extraction network includes an input layer, a nonlinear transformation layer, and an output layer. The input layer is used to convert the first text information into a word feature sequence, the nonlinear transformation layer is used to process the word feature sequence to obtain key features, and the output layer is used to process the key features to obtain the first key information.

[0039] Furthermore, the nonlinear transformation layer includes multiple neurons, and the word feature sequence includes a predetermined number of word features. The i-th neuron is used to process the m-th word feature. i The features of the word up to the (m)th word i +n i The features of each word are processed to obtain key features, m i Greater than or equal to 1 and less than the preset number, n i Greater than or equal to 0 and less than a preset quantity. Specifically, the i-th neuron is used to transfer the m-th neuron... i The features of the word up to the (m)th word i +n i Inputting the features of 1 digit word into a preset activation function yields key features. If the parameters of the i-th neuron are n... iA value of 0 indicates that this neuron does not extract key features.

[0040] In one specific embodiment, assume the nonlinear transformation layer comprises K neurons, where K is typically set by the user, for example, 5, 10, or 15. Generally, a larger K yields better extraction results, but training the model becomes more expensive, requiring a larger number of samples. Each neuron has two parameters: m i With n i If the activation function is the sigmoid function, the relationship between the input and output of the nonlinear transformation layer is as follows:

[0041]

[0042] Where f is the output of the nonlinear transformation layer, x is the word feature sequence, and x[m i :m i +n i ] is the m-th i The features of the word up to the (m)th word i +n i ) word features.

[0043] Information can be extracted from long input text data (i.e., the first text information) through nonlinear transformation, with parameter m... i and n i The text summarization model is updated and adjusted during training to achieve optimal performance.

[0044] Step 44: Map the first key information into a first key vector sequence using an embedding layer.

[0045] For the first key information extracted by the key information extraction network, before the input extraction and generation network performs result prediction, it is vectorized through the embedding layer to generate the first key vector sequence.

[0046] Step 45: Input the first key vector sequence into the extraction and generation network to obtain the probability distribution of words.

[0047] The first key vector sequence is input into the extraction-generating network for text summarization processing to obtain the summarization result (i.e., the probability distribution of words). The extraction-generating network can be a Transformer model or a fusion model of Transformer and copy. The first key information includes at least one word, and the probability distribution includes the word and the probability of the word. For example, if the first key information includes three letters: AC, the probability of letter A is 0.72, the probability of letter B is 0.8, and the probability of letter C is 0.95, then the probability distribution is {(A, 0.72), (B, 0.8), (C, 0.95)}.

[0048] Step 46: Obtain the first summary information based on the probability distribution.

[0049] The process involves determining whether the probability of each word in the probability distribution is greater than a preset probability. This preset probability can be set in advance based on experience or application needs, such as 0.5, 0.55, or 0.6. If the probability of a word is greater than the preset probability, it is marked as a word to be output. This process is repeated until the probability of each word is determined. Then, the words to be output are arranged in a preset order to obtain the first summary information. This preset order can be the order in which the words in the first key information are arranged. For example, suppose the first text information is "Hello, I would like to inquire about how to open a bank deposit? Okay, please wait a moment. Do you need to open a credit card? Yes, you need to open a credit card." The first key information is "inquire about how to open a bank deposit, need to open a credit card." Through the processing of the embedding layer and the extraction generation network, the first summary information is obtained: "open a deposit, open a credit card."

[0050] Understandably, in other embodiments, the text summarization model may also include a network output layer connected to an extraction-generating network, which receives a probability distribution output by the extraction-generating network, i.e., the output layer performs the task of obtaining first summary information based on the probability distribution.

[0051] By incorporating a key information extraction network, precise key information can be located and extracted from the input long text, generating the first key information. This first key information is then input into the subsequent embedding layer and extraction generation network for feature vectorization and feature extraction. Since the model parameters can be continuously updated and adjusted through loss calculation and back-inference, it can adaptively handle text data of different lengths, thus effectively solving the problems of poor summarization efficiency and effectiveness for long texts.

[0052] In one specific embodiment, in order to summarize the input text data, a text summarization model is first trained. This text summarization model includes a key information extraction network, an embedding layer, and an extraction and generation network, and is trained using the following steps:

[0053] (1) Obtain the training sample set.

[0054] The training sample set includes multiple training samples, which include text information and corresponding labels (i.e., real summary information).

[0055] In one specific embodiment, audio data can be acquired first, and then the audio data can be converted into text data (i.e., text information) using a speech-to-text method. In order to improve the summarization effect of the text summary model, basic data augmentation and data denoising processing can be performed on the text information to form a dataset, which includes training data and test data.

[0056] (2) Select the text information of a training sample from the training sample set and input it to obtain the second summary information.

[0057] After obtaining the preprocessed text information, it is input into a key information extraction network to obtain second key information. This allows for the location and extraction of key information from the text, accurately grasping the core content and removing invalid or distracting information, thereby improving the summarization effect. The second key information is then input into an embedding layer, which performs feature vectorization to generate a second key vector sequence. This second key vector is then input into an extraction and generation network to obtain the second summary information.

[0058] (3) Compare the second summary information with the labels and adjust the model parameters until the preset stopping conditions are met.

[0059] The second summary information is compared with the corresponding labels to calculate the current loss value and determine whether the preset stopping condition is met. If the preset stopping condition is not met, training continues using training samples from the training sample set, and the model parameters are adjusted until training is complete, resulting in a trained text summary model.

[0060] Furthermore, the preset stopping conditions may include: loss value convergence, that is, the difference between the previous loss value and the current loss value is less than a set value; determining whether the current loss value is less than a preset loss value, which is a pre-set loss threshold; the number of training iterations reaching a set value (e.g., 10,000 training iterations); or the accuracy obtained when testing with a test set reaching a set condition, etc.

[0061] The model parameters include the parameters of the key information extraction network, the parameters of the embedding layer, or the parameters of the extraction and generation network. That is, during the training process, the parameters of the embedding layer that converts the second key information into the second key vector sequence, the parameters of the key information extraction network, and the parameters of the extraction and generation network are learned to update and optimize the model parameters in real time, adaptively learn the optimal text summarization network structure, and obtain the text summarization model for optimizing the location of key information in long texts after the model training is completed.

[0062] This embodiment provides a method for summarizing long text work orders based on a key information location mechanism. To address the problem of poor summary performance of long text work orders, a key information extraction network is introduced into the fusion model. The optimal extraction parameters are adaptively learned, which improves the efficiency and accuracy of key information location and extraction, and optimizes the summary effect of long text work orders.

[0063] Please see Figure 6 , Figure 6This is a schematic diagram of an embodiment of the text summarization device provided in this application. The text summarization device 60 includes: an acquisition module 61 and a processing module 62.

[0064] The acquisition module 61 is used for the first text information and text summary model, which includes a key information extraction network, an embedding layer, and an extraction generation network.

[0065] The processing module 62 is connected to the acquisition module 61. It is used to process the first text information using a key information extraction network to obtain the first key information; to map the first key information into a first key vector sequence using an embedding layer; and to process the first key vector sequence using an extraction generation network to obtain the first summary information.

[0066] This embodiment proposes a solution to optimize the technology of summarizing long text work orders. By adding a key information extraction network with key information location function to the front of the network structure of the text summary, it can automatically and adaptively process work order information of various lengths, thereby improving the problem of poor performance of long text summaries.

[0067] Please see Figure 7 , Figure 7 This is a schematic diagram of another embodiment of the text summarization device provided in this application. The text summarization device 70 includes a memory 71 and a processor 72 connected to each other. The memory 71 is used to store computer programs. When the computer programs are executed by the processor 72, they are used to implement the text summarization method in the above embodiment. The text summarization device 70 can be a call center assistant.

[0068] This embodiment employs a novel key information extraction network that can adaptively extract key information and input it into the embedding layer. It is suitable for long text analysis and mining, such as work order summary business, and can improve the analysis and mining effect of long text work orders.

[0069] Please see Figure 8 , Figure 8 This is a schematic diagram of an embodiment of a computer-readable storage medium provided in this application. The computer-readable storage medium 80 is used to store a computer program 81. When the computer program 81 is executed by a processor, it is used to implement the text summarization method in the above embodiment.

[0070] The computer-readable storage medium 80 can be any medium capable of storing program code, such as a server, USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0073] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0074] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A text summarization method, characterized in that, include: Acquire first text information and a text summary model, wherein the text summary model includes a key information extraction network, an embedding layer, and an extraction and generation network; The first text information is processed using the key information extraction network to obtain the first key information; The first key information is mapped into a first key vector sequence using the embedding layer; The extraction and generation network is used to process the first key vector sequence to obtain the first summary information; The key information extraction network includes an input layer, a nonlinear transformation layer, and an output layer. The nonlinear transformation layer includes K neurons, each with parameters mi and ni, where K is set by the user and can take values ​​of 5, 10, or 15, and m... i Greater than or equal to 1 and less than the preset number, n i If the output f of the nonlinear transformation layer is greater than or equal to 0 and less than the preset quantity, it is calculated in the following way: , where x is the word feature sequence, x[mi:mi+ni] represents the sequence from the mi-th word feature to the (mi+ni)-th word feature, and sigmoid is the activation function; If the parameter ni of the i-th neuron is 0, then the neuron does not extract key features; Information is extracted from the first text input through nonlinear transformation, and the parameters mi and ni are updated and adjusted during the training process of the text summarization model to achieve optimality.

2. The text summarization method according to claim 1, characterized in that, The input layer is used to convert the first text information into a word feature sequence, the nonlinear transformation layer is used to process the word feature sequence to obtain key features, and the output layer is used to process the key features to obtain the first key information.

3. The text summarization method according to claim 2, characterized in that, The nonlinear transform layer includes multiple neurons, and the word feature sequence includes a preset number of word features. The i-th neuron is used to process the m-th word feature. i The features of the word up to the (m)th word i +n i The key features are obtained by processing the features of each word.

4. The text summarization method according to claim 3, characterized in that, The i-th neuron is used to transfer the m-th neuron. i The word features up to the (m)th word feature i +n i The key features are obtained by inputting the features of each word into a preset activation function.

5. The text summarization method according to claim 2, characterized in that, The first key information includes at least one word, and the step of processing the first key vector sequence using the extraction and generation network to obtain the first summary information includes: The first key vector sequence is input into the extraction and generation network to obtain the probability distribution of the word; The first summary information is obtained based on the probability distribution.

6. The text summarization method according to claim 5, characterized in that, The probability distribution includes the word and the probability of the word. The step of obtaining the first summary information based on the probability distribution includes: Determine whether the probability of the word is greater than a preset probability; If so, then record the word as the word to be output; The words to be output are arranged in a preset order to obtain the first summary information.

7. The text summarization method according to claim 1, characterized in that, The step of obtaining the first text information includes: Acquire audio data and convert the audio data into second text information; The second text information is preprocessed to obtain the first text information.

8. The text summarization method according to claim 7, characterized in that, The step of preprocessing the second text information to obtain the first text information includes: The format of the second text information is detected to determine whether the format of the second text information is a preset format; If not, the second text information is formatted and converted to obtain the first text information.

9. The text summarization method according to claim 1, characterized in that, The method further includes: Obtain a training sample set, which includes multiple training samples, and each training sample includes text information and labels corresponding to the text information; Text information of one training sample is selected from the training sample set and input into the text summarization model to obtain second summary information; The second summary information is compared with the calculated label, and the model parameters are adjusted until a preset stopping condition is met to obtain the text summary model. The model parameters include the parameters of the key information extraction network, the parameters of the embedding layer, and the parameters of the extraction generation network.

10. A text summarization device, characterized in that, include: The acquisition module is used for the first text information and the text summary model, wherein the text summary model includes a key information extraction network, an embedding layer, and an extraction and generation network. The processing module is configured to process the first text information using the key information extraction network to obtain first key information; map the first key information into a first key vector sequence using the embedding layer; and process the first key vector sequence using the extraction generation network to obtain first summary information. The key information extraction network includes an input layer, a nonlinear transformation layer, and an output layer. The nonlinear transformation layer includes K neurons, each with parameters mi and ni, where K is set by the user and can take values ​​of 5, 10, or 15, and m... i Greater than or equal to 1 and less than the preset number, n i If the output f of the nonlinear transformation layer is greater than or equal to 0 and less than the preset quantity, it is calculated in the following way: , where x is the word feature sequence, x[mi:mi+ni] represents the sequence from the mi-th word feature to the (mi+ni)-th word feature, and sigmoid is the activation function; If the parameter ni of the i-th neuron is 0, then the neuron does not extract key features; Information is extracted from the first text input through nonlinear transformation, and the parameters mi and ni are updated and adjusted during the training process of the text summarization model to achieve optimality.

11. A text summarization device, characterized in that, It includes an interconnected memory and a processor, wherein the memory is used to store a computer program, which, when executed by the processor, is used to implement the text summarization method of any one of claims 1-9.

12. A computer-readable storage medium for storing a computer program, characterized in that, When executed by a processor, the computer program is used to implement the text summarization method according to any one of claims 1-9.

Images