Method for generating text summary, method and device for training summary generation model

Abstract

The embodiment of the application discloses a method for generating a text summary, a method and device for training a summary generation model, and relates to the technical field of artificial intelligence. The main technical scheme comprises: obtaining a to-be-processed text; extracting a key text unit from the to-be-processed text; inputting information of the to-be-processed text and the key text unit into a summary generation model to obtain a summary of the to-be-processed text generated by the summary generation model using the information of the to-be-processed text and the key text unit; and the summary generation model is obtained by pre-training based on a second deep learning model. The application provides guidance for the generation of the summary through the extraction and introduction of the key text unit, so that the summary generation model can focus on the key content in the to-be-processed text, the influence of noise is reduced, and the accuracy of generating a summary for a long text is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a method for generating a text summary, a method and device for training a summary generation model. BACKGROUND

[0002] The process of using artificial intelligence technology to process a large amount of text to produce concise and refined content is summary generation. People can grasp the main content of the text by reading the summary, saving time and improving reading efficiency. However, in many scenarios, it is necessary to generate a summary for a long text, such as generating a summary for a video conference, a lecture, an interview, etc. The long text in these scenarios has characteristics such as long duration and sparse distribution of key information, and existing academic research solutions and industrial solutions are difficult to accurately generate summaries for long texts. SUMMARY

[0003] Therefore, the present application provides a method for generating a text summary, a method and device for training a summary generation model, to accurately generate summaries for long texts.

[0004] The present application provides the following solutions:

[0005] In a first aspect, a method for generating a text summary is provided, the method comprising:

[0006] obtaining a text to be processed;

[0007] extracting a key text unit from the text to be processed;

[0008] inputting information of the text to be processed and the key text unit into a summary generation model, and obtaining a summary of the text to be processed generated by the summary generation model using information of the text to be processed and the key text unit;

[0009] wherein the summary generation model is obtained by pre-training based on a second deep learning model.

[0010] According to an implementable manner in the embodiments of the present application, extracting a key text unit from the text to be processed comprises:

[0011] inputting the text to be processed into a key text extraction model to obtain a key text unit extracted from the text to be processed by the key text extraction model, wherein the key text extraction model is obtained by pre-training based on a first deep learning model; or

[0012] extracting a key text unit from the text to be processed using a preset feature rule; or

[0013] The key text extraction model is input with the text to be processed, first key text units extracted from the text to be processed by the key text extraction model are obtained, second key text units are extracted from the text to be processed by using a preset feature rule, the first key text units and the second key text units are fused, and key text units are obtained.

[0014] According to an implementable manner in embodiments of the present application, the key text extraction model comprises a first encoding network and a classification network.

[0015] The first encoding network encodes the text to be processed to obtain feature representations of each text unit in the text to be processed.

[0016] The classification network classifies each text unit by using the feature representations of each text unit to obtain a classification result of whether each text unit is a key text unit.

[0017] According to an implementable manner in embodiments of the present application, the summary generation model comprises a second encoding network and a decoding network.

[0018] The second encoding network encodes the text to be processed by using information of the key text units to obtain feature representations of each Token in the text to be processed.

[0019] The decoding network decodes by using the feature representations of each Token in the text to be processed to generate a summary of the text to be processed.

[0020] According to an implementable manner in embodiments of the present application, the second encoding network encodes the text to be processed by using information of the key text units to obtain feature representations of each Token in the text to be processed, which comprises:

[0021] The second encoding network obtains embedding features of each Token in the text to be processed after embedding processing of the text to be processed.

[0022] The embedding features of each Token are processed by using information of the key text units to obtain feature representations of each Token, wherein the processing of the attention mechanism comprises: when attention processing is performed on each Token belonging to a key text unit, embedding features of all Tokens in the text to be processed are used; and when attention processing is performed on each Token not belonging to a key text unit, embedding features of each Token within a preset window distance from the Token are used.

[0023] According to an implementable manner in the embodiments of the present application, the method is applied to an online meeting scenario, the to-be-processed text is a meeting record of an online meeting, the key text unit is a key sentence, and the summary is a meeting summary of the meeting record.

[0024] In a second aspect, a method for training a summary generation model is provided, and the method comprises:

[0025] obtaining a plurality of second training samples, wherein each second training sample comprises a second text sample, information of a key text unit of the second text sample, and a summary sample of the second text sample;

[0026] training the summary generation model by using the plurality of second training samples, wherein the summary generation model comprises a second encoding network and a decoding network;

[0027] the second encoding network encodes the second text sample by using the information of the key text unit of the second text sample, to obtain a feature representation of each token in the second text sample;

[0028] the decoding network decodes by using the feature representation of each token in the second text sample, to generate a summary of the second text sample;

[0029] the training target comprises minimizing a difference between the summary of the second text sample generated by the decoding network and the summary sample of the second text.

[0030] According to an implementable manner in the embodiments of the present application, the second encoding network encodes the second text sample by using the information of the key text unit of the second text sample, to obtain a feature representation of each token in the second text sample, which comprises:

[0031] the second encoding network obtains an embedding feature of each token in the second text sample after embedding processing of the second text sample;

[0032] the embedding feature of each token is processed by using the information of the key text unit of the second text sample, to obtain the feature representation of each token, wherein the processing by using the attention mechanism comprises: when each token belonging to the key text unit is processed by using the attention mechanism, embedding features of all tokens in the second text sample are used; and when each token not belonging to the key text unit is processed by using the attention mechanism, embedding features of tokens within a preset window distance from the token are used.

[0033] In a third aspect, a method for training a key text extraction model is provided, and the method comprises:

[0034] obtaining a plurality of first training samples, the first training samples comprising first text samples and key text unit labels annotated for the first text samples;

[0035] training a key text extraction model using the plurality of first training samples, wherein the key text extraction model comprises a first encoding network and a classification network;

[0036] the first encoding network performs encoding processing on the first text samples to obtain feature representations of text units in the first text samples;

[0037] the classification network classifies the text units using the feature representations of the text units to obtain classification results of whether the text units are key text units;

[0038] the training target comprises minimizing the difference between the classification results of the classification network and the key text unit labels annotated for the first text samples.

[0039] In a fourth aspect, a text summary generation apparatus is provided, and the apparatus comprises:

[0040] a text obtaining unit configured to obtain a to-be-processed text;

[0041] a key extraction unit configured to extract key text units from the to-be-processed text;

[0042] a summary generation unit configured to input information of the to-be-processed text and the key text units into a summary generation model to obtain a summary of the to-be-processed text generated by the summary generation model using the information of the to-be-processed text and the key text units; wherein the summary generation model is obtained by pre-training a second deep learning model.

[0043] In a fifth aspect, a device for training a summary generation model is provided, and the device comprises:

[0044] a second sample obtaining unit configured to obtain a plurality of second training samples, the second training samples comprising second text samples, information of key text units of the second text samples, and summary samples of the second text samples;

[0045] a second model training unit configured to train the summary generation model using the plurality of second training samples, wherein the summary generation model comprises a second encoding network and a decoding network; the second encoding network encodes the second text sample using information of the key text units of the second text sample to obtain feature representations of tokens in the second text sample; the decoding network decodes the feature representations of the tokens in the second text sample to generate a summary of the second text sample; and a target of the training comprises minimizing a difference between the summary of the second text sample generated by the decoding network and a summary sample of the second text.

[0046] In a sixth aspect, a device for training a key text extraction model is provided, and the device comprises:

[0047] a first sample obtaining unit configured to obtain a plurality of first training samples, wherein the first training samples comprise first text samples and key text unit labels annotated for the first text samples;

[0048] a first model training unit configured to train a key text extraction model using the plurality of first training samples, wherein the key text extraction model comprises a first encoding network and a classification network; the first encoding network encodes the first text samples to obtain feature representations of text units in the first text samples; the classification network classifies the text units using the feature representations of the text units to obtain classification results of whether the text units are key text units; and a target of the training comprises minimizing a difference between the classification results of the classification network and the key text unit labels annotated for the first text samples.

[0049] According to a seventh aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement steps of the method in any one of the first aspect to the third aspect.

[0050] According to an eighth aspect, an electronic device is provided, and the electronic device comprises:

[0051] one or more processors; and

[0052] a memory associated with the one or more processors, the memory being configured to store program instructions, which, when executed by the one or more processors, perform steps of the method in any one of the first aspect to the third aspect.

[0053] According to the embodiments provided in the present application, the following technical effects are disclosed:

[0054] 1) The present application first extracts key text units from the text to be processed, and then uses the information of the text to be processed and the key text units to generate a summary of the text to be processed by a summary generation model based on a deep learning model. The extraction and introduction of key text units provide guidance for the generation of summaries, enabling the summary generation model to focus on the key content in the text to be processed, reducing the influence of noise, and thus improving the accuracy of generating summaries for long texts.

[0055] 2) In the present application, not only can the supervised method (i.e. through the key text extraction model) be used to extract the key text units of the text to be processed, but also the unsupervised method (i.e. using pre-set feature rules) can be further combined to enhance the information of the extracted key text units.

[0056] 3) In the present application, when the summary generation model uses the information of the key text units to generate a summary of the text to be processed, only the Tokens in the key text units are processed with global attention, while other Tokens are processed with local attention, thereby enhancing the ability of the summary generation model to focus on key information and resist noise interference, reducing the computational overhead, and enabling the summary generation model to successfully generate summaries for long texts.

[0057] Of course, implementing any product of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0059] Figure 1 is a system architecture diagram suitable for the embodiments of the present application;

[0060] Figure 2 is the main flowchart of the text summary generation method provided by the embodiments of the present application;

[0061] Figure 3 is the structure principle diagram of the key text extraction model provided by the embodiments of the present application;

[0062] Figure 4 is the training method flowchart of the key text extraction model provided by the embodiments of the present application;

[0063] Figure 5 is the structure principle diagram of the summary generation model provided by the embodiments of the present application;

[0064] Figure 6 A flowchart of a training method of a summary generation model provided in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of summary extraction of an online conference recording provided in an embodiment of the present application;

[0066] Figure 8 A schematic block diagram of a text summary generation apparatus provided in an embodiment of the present application;

[0067] Figure 9 A schematic block diagram of an apparatus for training a key text extraction model provided in an embodiment of the present application;

[0068] Figure 10 A schematic block diagram of an apparatus for training a summary generation model provided in an embodiment of the present application;

[0069] Figure 11 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0071] The terms used in the embodiments of the present application are merely for the purpose of describing the specific embodiments, and are not intended to limit the present application. The singular forms “a”, “an” and “the” used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0072] It should be understood that the term “and / or” used herein is merely to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character “ / ” herein generally represents an “or” relationship between the front and rear associated objects.

[0073] Depending on the context, the word “if” as used herein can be interpreted as meaning “when” or “upon” or “in response to determining” or “in response to detecting”. Similarly, depending on the context, the phrase “if it is determined” or “if (a stated condition or event) is detected” can be interpreted as meaning “when it is determined” or “in response to determining” or “when (a stated condition or event) is detected” or “in response to detecting (a stated condition or event)”.

[0074] Currently, the model based on the Transformer network has achieved excellent performance in generating summaries for short texts. The research on long text summaries is only in the academic field, and the academic field has given various solutions, but the effect is not good.

[0075] Therefore, the present application provides a brand-new summary extraction framework. In order to facilitate the understanding of the present application, first, the system architecture to which the present application is applied is simply described. Figure 1 An exemplary system architecture to which the embodiments of the present application can be applied is shown as follows: Figure 1 As shown in FIG. 1, the system architecture can include a first model training device, a second model training device, and a text summary generation device.

[0076] The first model training device can use the method provided by the embodiments of the present application to train the model after obtaining the first training data, and obtain a key text extraction model.

[0077] The second model training device can use the method provided by the embodiments of the present application to train the model after obtaining the second training data, and obtain a summary generation model.

[0078] The first model training device and the second model training device can establish the summary generation model in an offline manner.

[0079] The text summary generation device generates a summary for the input to-be-processed text using the summary generation model, and can further extract key text units in the summary generation process using the key text extraction model, and then generate a summary for the to-be-processed text using the extracted key text units. The text summary generation device can generate summaries online or offline. The first model training device and the key text extraction model are not necessarily used, and other methods can be used to extract key text units, which will be described in detail in subsequent embodiments.

[0080] The first model training device, the second model training device, and the text summary generation device can be set as independent servers, or can be set in the same server or server group, or can be set in independent or same cloud servers. The cloud server is also called a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of large management difficulty and weak service scalability in traditional physical hosts and virtual private server (VPs, Virtual Private Server) services. The first model training device, the second model training device, and the text summary generation device can also be set in a computer terminal with strong computing power.

[0081] It should be understood that Figure 1The number of the first model training apparatus, the second model training apparatus, the text summary generation apparatus, the key text extraction model and the summary generation model in the system is merely illustrative. According to the implementation needs, there can be any number of the first model training apparatus, the second model training apparatus, the text summary generation apparatus, the key text extraction model and the summary generation model.

[0082] Figure 2 The main flowchart of the text summary generation method provided by the embodiment of the present application can be executed by the text summary generation apparatus in the system shown in Figure 1 As shown in the method shown in Figure 2 The method can include the following steps:

[0083] Step 202: Obtain the text to be processed.

[0084] Step 204: Extract the key text unit from the text to be processed.

[0085] Step 206: Input the information of the text to be processed and the key text unit into the summary generation model to obtain the summary of the text to be processed generated by the summary generation model using the information of the text to be processed and the key text unit, wherein the summary generation model is pre-trained based on the second deep learning model.

[0086] As can be seen from the above flow, the present application first extracts the key text unit from the text to be processed, and then generates the summary of the text to be processed using the information of the text to be processed and the key text unit by the summary generation model based on the deep learning model. The extraction and introduction of the key text unit provide guidance for the generation of the summary, thereby improving the accuracy of the generated summary.

[0087] It should be noted that the "first", "second" and the like involved in the present disclosure do not have the limitation of size, order and quantity, and are only used to distinguish the names. For example, "first deep learning model" and "second deep learning model" are used to distinguish two deep learning models, for example, "first key text unit" and "second key text unit" are used to distinguish two key text units, and for example, "first encoding network" and "second encoding network" are used to distinguish two encoding networks.

[0088] The steps in the above flow will be described in detail below. First, the above step 202, i.e. "obtaining the text to be processed", will be described in detail.

[0089] The to-be-processed text involved in the embodiments of the present application refers to a text from which an abstract needs to be extracted. In the embodiments of the present application, the to-be-processed text can be a long text or a short text. That is, the manner provided by the embodiments of the present application has no limitation on the length of the to-be-processed text, and can realize extraction of an abstract from a short text as well as extraction of an abstract from a long text. Wherein, “long text” and “short text” are a relative concept, and the specific standard can be self-defined. For example, a text with less than 512 characters can be referred to as a short text, and a text with more than or equal to 512 characters can be referred to as a long text.

[0090] The to-be-processed text can be obtained from a database storing the text or from a system generating the to-be-processed text. For example, an online conference service can generate a conference record for the conference content of a user, and the text abstract generation apparatus in the embodiments of the present application can obtain the conference record from the online conference service side. The online conference service can persist the conference record in a database, and the text abstract generation apparatus in the embodiments of the present application can obtain the conference record from the database as the to-be-processed text after obtaining an abstract generation instruction.

[0091] The above step 204, i.e., “extracting a key text unit from the to-be-processed text” is described in detail below.

[0092] The key text unit involved in this step can be a paragraph, a key sentence, a key phrase, or the like. As one of the more preferred manners, the subsequent embodiments are described by taking a key sentence as an example of a key text unit. The key text unit extracted from the to-be-processed text can be one or more. For example, m key sentences are extracted from the to-be-processed text after executing this step, and m is a positive integer.

[0093] The manner of extracting a key text unit in this step can adopt, but is not limited to, the following three manners:

[0094] The first manner: extracting a key text unit by using a deep learning model. That is, inputting the to-be-processed text into a key text extraction model to obtain the key text unit extracted from the to-be-processed text by the key text extraction model.

[0095] The key text extraction model is obtained by pre-training based on a first deep learning model. The structure of the key text extraction model can be as shown in FIG. 1, which includes a first encoding network (Encoder) and a classification network. Figure 3

[0096] The first encoding network encodes the to-be-processed text to obtain a feature representation of each text unit in the to-be-processed text.

[0097] ​The first encoding network can be implemented by a Transformer network, for example, can be implemented based on a pre-trained language model, and the pre-trained language model can be implemented by a BERT (Bidirectional Encoder Representation from Transformers) model, a GPT (Generative Pre-Training) model, an XLNet (a self-recurrent model that realizes bidirectional context information by arranging a language model), etc.

[0098] In the first encoding network, the Token in the to-be-processed text can be first subjected to an Embedding process, and then encoded based on the result of the Embedding to obtain a feature representation of each Token. Then, the feature representation of each Token contained in each text unit is used to obtain a feature representation of each text unit. For example, as shown in FIG. 1, the to-be-processed text includes n sentences, and for each sentence, the feature representations of the Tokens in the sentence are spliced to obtain a sentence-level feature representation. The Token can include a character, a start symbol, a separator, etc. Figure 3

[0099] The above-mentioned Embedding process can include word Embedding, position Embedding, and sentence Embedding. The word Embedding is to encode each Token by a word vector to obtain a word vector representation. The position Embedding is to encode the position of each Token in the to-be-predicted text sequence to obtain a position representation. For example, the positions of the Tokens in the text sequence can be sequentially numbered as 0, 1, 2, 3, 4, 5, and 6, etc. The sentence Embedding is to encode the sentence information of each Token to obtain a representation of the belonging sentence. For example, the Tokens in the first sentence are encoded as 0, the Tokens in the second sentence are encoded as 1, and so on.

[0100] The classification network classifies each text unit by using the feature representation of each text unit to obtain a classification result of whether each text unit is a key text unit. The classification network is actually a binary classification network. For example, the feature representation of each sentence is used to classify the sentence, and the classification result is a key sentence or a non-key sentence. After the classification network classifies each sentence, the information of the key sentence in the to-be-processed text can be obtained, that is, which sentence or which sentences are the key sentence. For example, as shown in FIG. 1, the output key sentences i, j, …, k are obtained. Figure 3

[0101] ​​The second mode is to extract the key text unit in an unsupervised manner, that is, to extract the key text unit from the text to be processed according to the preset feature rule.

[0102] As one of the modes, each sentence in the text to be processed can be scored based on the preset feature rule, and the score of each sentence is obtained. The sentence whose score meets the preset requirement is regarded as a key sentence.

[0103] For example, the TF (term frequency)-IDF (inverse document frequency) of each word in the sentence can be counted, and the sentence can be scored based on the TF-IDF of each word. For another example, the TextRank algorithm can be used to score each sentence based on the similarity between sentences, in which the TextRank algorithm regards the sentences in the text as the web pages in the PageRank algorithm, constructs the graph relationship between the sentences, and obtains the importance ranking of the sentences through the iterative calculation similar to the PageRank algorithm. Since the TextRank algorithm is an existing algorithm, it will not be described in detail here.

[0104] The third mode is to combine the deep learning model and the unsupervised mode to extract the key text unit, that is, to combine the first mode and the second mode, which not only utilizes the recognition ability of the deep learning model for the key information, but also introduces the unsupervised method to enhance the key information.

[0105] As one of the modes, the text to be processed can be input into the key text extraction model, the first key text unit extracted from the text to be processed by the key text extraction model can be obtained, the second key text unit can be extracted from the text to be processed according to the preset feature rule, and the first key text unit and the second key text unit can be fused to obtain the key text unit.

[0106] That is, the key text unit extracted by the first mode is regarded as the first key text unit, and the key text unit extracted by the second mode is regarded as the second key text unit. When the first key text unit and the second key text unit are fused, the key text units extracted by the two modes can be taken as an intersection or a union, etc., to obtain the final key text unit. For example, the first mode obtains four key sentences: sentence 3, sentence 20, and sentence 32, and the second mode obtains three key sentences: sentence 5, sentence 20, and sentence 32. Then, the union processing can be performed to obtain sentence 3, sentence 5, sentence 20, and sentence 32 as the key sentences. Alternatively, the intersection processing can be performed to obtain sentence 20 and sentence 32 as the key sentences.

[0107] It has been mentioned above that the key text extraction model involved in the first mode and the third mode is pre-trained based on a first deep learning model, and the model structure is as shown in Figure 3 The training process of the key text extraction model will be introduced below. As shown in Figure 4 The key text extraction model can be trained by the following steps:

[0108] Step 402: Obtain a plurality of first training samples, and each first training sample includes a first text sample and a key text unit label annotated for the first text sample.

[0109] When training the text extraction model, some texts can be selected as the first text samples. When annotating the key text unit label for the first text sample, manual annotation can be used. Since the manual annotation method is low in efficiency, some methods can be used to automatically determine the information of the key text unit of the first text sample.

[0110] As one of the implementable methods, the text with known abstract can be used as the first text sample, that is, some texts with abstract can be used as the first text sample. Then, the similarity between each text unit in the first text sample and the abstract of the first text sample is calculated to determine the key text unit. For example, by using the GreedySearch method, m sentences with the highest similarity to the abstract are searched from the first text sample as the key sentences, and m is a preset positive integer.

[0111] As another implementable method, an unsupervised method can be used to extract the key text unit, for example, based on a preset feature rule, each sentence in the text to be processed is scored to obtain the score value of each sentence, and the sentence with a score value meeting a preset requirement is used as the key sentence. For example, the TF-IDF of each word in the sentence can be counted, and the sentence is scored based on the TF-IDF of each word. For another example, the TextRank algorithm can be used to score each sentence based on the similarity between the sentences.

[0112] Step 404: Train the key text extraction model using the plurality of first training samples, wherein the key text extraction model includes a first encoding network and a classification network, and the training target includes minimizing the difference between the classification result of the classification network and the key text unit label annotated for the first text sample.

[0113] The structure of the key text extraction model is also as shown in Figure 3 The first training sample includes a first text sample and a key text unit label annotated for the first text sample. The first text sample in the first training sample is used as the input of the key text extraction model, and the key text unit label annotated for the first text sample is used as the target output of the key text extraction model.

[0114] The first encoding network encodes the first text sample to obtain a feature representation of each text unit in the first text sample.

[0115] The first encoding network can first perform Embedding processing on each Token in the first text sample, and then encode based on the result of Embedding to obtain a feature representation of each Token. Then, the feature representation of each Token contained in each text unit is used to obtain a feature representation of each text unit, respectively. For example, the first text sample includes n sentences. For each sentence, the feature representations of the Tokens in the sentence are spliced to obtain a sentence-level feature representation. The Token can include a character, a start symbol, a separator, and the like.

[0116] The classification network classifies each text unit using the feature representation of each text unit to obtain a classification result of whether each text unit is a key text unit. The classification network is actually a binary classification network. For example, the classification network classifies a sentence using the feature representation of the sentence, and the classification result is: key sentence or non-key sentence. After classifying each sentence using the classification network, the information of the key sentence in the first text sample can be obtained.

[0117] The training target used when training the key text extraction model includes: minimizing the difference between the classification result of the classification network and the key text unit label annotated by the first text sample. A loss function can be constructed according to the training target. In each iteration, the value of the loss function is used to update the model parameters using methods such as gradient descent until a preset training end condition is met. The training end condition can include, for example, the value of the loss function is less than or equal to a preset loss function threshold, the number of iterations reaches a preset number threshold, and the like.

[0118] Referring back to Figure 2 , the step 206 in Figure 2 , i.e., "inputting the information of the text to be processed and the key text unit into an abstract generation model to obtain an abstract of the text to be processed generated by the abstract generation model using the information of the text to be processed and the key text unit", will be described in detail below in conjunction with an embodiment.

[0119] In this step, the abstract generation model generates an abstract of the text to be processed based on the information of the key text unit. The structure of the abstract generation model can be as shown in Figure 5 , which includes a second encoding network (Encoder) and a decoding network (Decoder), i.e., an Encoder-Decoder framework is used to realize sequence-to-sequence prediction.

[0120] The second encoding network encodes the to-be-processed text by using information of the key text unit, to obtain a feature representation of each element Token in the to-be-processed text.

[0121] Specifically, the second encoding network performs Embedding processing on the to-be-processed text, to obtain an embedding feature of each Token in the to-be-processed text. Then, the embedding feature of each Token is processed by using an attention mechanism according to information of the key text unit, to obtain a feature representation of each Token. The processing by using the attention mechanism can include: performing global attention (Global Attention) processing on each Token belonging to the key text unit, and performing local attention (Local Attention) processing on each Token not belonging to the key text unit.

[0122] The Embedding processing can include: word Embedding, position Embedding, and sentence Embedding. The word Embedding is to encode each Token by using a word vector, to obtain a word vector representation. The position Embedding is to encode a position of each Token in the to-be-processed text sequence, to obtain a position representation. For example, each Token in the text sequence can be sequentially numbered as 0, 1, 2, 3, 4, 5, and 6, etc. The sentence Embedding is to encode sentence information of each Token, to obtain a representation of a sentence to which the Token belongs. For example, each Token in a first sentence is encoded as 0, each Token in a second sentence is encoded as 1, and so on. It should be noted that the Embedding processing can also include only the word Embedding and the position Embedding, or only the word Embedding and the sentence Embedding.

[0123] In addition, the traditional self-attention processing is to perform global attention processing on all Tokens, to consider all hidden states, to combine them into an attention matrix, and then to multiply the attention matrix by a weight matrix. An increase in the input quantity will also cause an increase in the size of the attention matrix, and thus a large amount of calculation will be generated and the memory will be limited, so that the length of the input sequence is usually limited to 512 characters, and the extraction of a summary of a long text cannot be realized.

[0124] In the embodiments of the present application, when the embedding features of each Token are processed by the attention mechanism, the traditional self-attention processing is no longer used, but only the global attention processing is performed on the Tokens belonging to the key text units. When the Tokens belonging to the key text units are processed by the attention mechanism, the embedding features of all Tokens in the text to be processed are used, that is, for the Token belonging to the key text unit, the attention information of all Tokens in the text to be processed on the Token is calculated. The local attention processing is performed on other Tokens not belonging to the key text units. When the Tokens not belonging to the key text units are processed by the attention mechanism, the embedding features of the Tokens within a preset window distance from the Token are used, that is, for the Token not belonging to the key text unit, only the attention information of the Tokens within a preset window distance from the Token in the text to be processed on the Token is calculated. This way can help the model to obtain the ability to focus on the key information and reduce the influence of noise in the long text caused by the sparsity of the key information. Compared with the traditional global attention processing on all Tokens, the computational cost is greatly reduced, which provides strong support for the application landing of the summary generation service.

[0125] The global attention processing refers to that when the feature representation of a Token is calculated, the hidden states of all Tokens in the input sequence (i.e., the entire text to be processed) need to be considered. The local attention processing refers to that when the feature representation of a Token is calculated, only a subset of the hidden states of the input sequence needs to be considered, which is usually the hidden states of the corresponding subsequence in a preset length window centered on the current time step position. The preset length of the window can be an empirical value or an experimental value, and is usually a positive integer within 512 characters.

[0126] Suppose that the key sentences i, j, …, k are extracted in step 204, the global attention processing is performed on the Tokens in the key sentences i, j, …, k, and the local attention processing is performed on the Tokens in other sentences, and finally the feature representations of the Tokens in the text to be processed are obtained.

[0127] The decoding network performs decoding processing on the feature representations of the Tokens in the text to be processed to generate the summary of the text to be processed.

[0128] The training process of the summary generation model will be introduced below. As shown in Figure 6 The summary generation model can be trained by the following steps:

[0129] Step 602: Obtain a plurality of second training samples, and the second training sample includes a second text sample, information of a key text unit of the second text sample, and a summary sample of the second text sample.

[0130] In training the summary generation model, some texts with summaries can be selected as the second text samples, and the summaries thereof can be selected as the summary samples. Alternatively, some texts can be selected as the second text samples, and summaries of the second text samples can be generated manually as the summary samples.

[0131] The information of the key text units of the second text samples can be labeled manually. Since the manual labeling is inefficient, some manners can be used to automatically determine the information of the key text units of the second text samples.

[0132] As one of the manners, the key text units can be determined by calculating the similarity between each text unit of the second text sample and the summary of the second text sample. For example, the m sentences with the highest similarity to the summary can be searched from the second text sample as the key sentences by using the GreedySearch manner, where m is a preset positive integer.

[0133] As another manner, the key text units can be extracted from the second text sample by using a trained key text extraction model.

[0134] As still another manner, the key text units can be extracted from the second text sample in an unsupervised manner. For example, each sentence in the text to be processed can be scored based on a preset feature rule to obtain a score value of each sentence, and the sentence with a score value satisfying a preset requirement can be selected as the key sentence. For example, the TF-IDF of each word in the sentence can be counted, and the sentence can be scored based on the TF-IDF of each word. For another example, the TextRank algorithm can be used to score each sentence based on the similarity between the sentences.

[0135] As still another manner, the key text units extracted by the above at least two manners can be fused to obtain the final result of the extracted key text units. The fusion processing can be an intersection or a union, etc.

[0136] In step 604, the summary generation model is trained by using the second training samples, where the summary generation model includes a second encoding network and a decoding network, and the training target includes minimizing the difference between the summary of the second text sample generated by the decoding network and the summary sample of the second text.

[0137] The structure of the summary generation model can be referred to as shown in Figure 5 The second encoding network encodes the second text sample by using the information of the key text units of the second text sample to obtain the feature representation of each Token in the second text sample.

[0138] Specifically, the second encoding network acquires the embedding features of each token in the second text sample after embedding the second text sample; it then uses the information of the key text units in the second text sample to perform attention mechanism processing on the embedding features of each token to obtain the feature representation of each token. The attention mechanism processing includes: performing global attention processing on tokens belonging to key text units, i.e., using the embedding features of all tokens in the second text sample when performing attention processing on tokens belonging to key text units; and performing local attention processing on tokens not belonging to key text units, i.e., using the embedding features of tokens within a preset window distance of the token in the second text sample when performing attention processing on tokens not belonging to key text units.

[0139] The decoding network uses the feature representations of each token in the second text sample to perform decoding processing and generate a summary of the second text sample.

[0140] The training objectives for the summary generation model include minimizing the difference between the summary generated by the decoding network and the summary sample of the second text sample. A loss function can be constructed based on this training objective. In each iteration, the model parameters are updated using methods such as gradient descent based on the value of the loss function, until a preset training termination condition is met. This termination condition may include, for example, the value of the loss function being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0141] The methods provided in this application embodiment can be applied to various application scenarios, including but not limited to:

[0142] Application Scenario 1: Extracting summaries from online meeting minutes

[0143] With the continued growth of the global market, online conferencing services are being used more and more. Users can use online conferencing software to conduct web conferences, participate in classroom teaching, conduct remote interviews without leaving home, and create lectures and forums. Users can also record and transmit information while conducting video conferences.

[0144] Online conferencing services generate meeting minutes after performing speech recognition on the audio during a meeting. These minutes can include not only the speech recognition results but also text and document information entered by the user in the meeting interface. To help users understand the main content of the meeting, a meeting summary needs to be generated from the minutes. Because meetings, lectures, interviews, and similar scenarios are characterized by long durations, sparse distribution of key information, and a high degree of colloquialism, therefore... Figure 7As shown in the middle, the method provided by the embodiment of the application can be used to obtain the conference record from the server end of the online conference service as the to-be-processed text. Then, on the one hand, the key sentence is extracted from the to-be-processed text by using the key text extraction model, and on the other hand, the key sentence is extracted from the to-be-processed text by using the unsupervised manner, that is, by using the preset feature rule. The key sentences extracted in the two aspects are fused to obtain the key sentence of the to-be-processed text. Then, the information of the to-be-processed text and the key sentence is input into the summary generation model to obtain the summary of the to-be-processed text, that is, the conference record.

[0145] The summary of the conference record can be automatically sent to each participant of the online conference. The conference record can also be stored in association with the summary in a database, so as to subsequently send the summary of the conference record to a user in response to a request of the user, and the like.

[0146] By the manner provided by the embodiment of the application, the information of the key sentence extracted from the conference record is introduced when the summary is generated for the conference record, thereby providing guidance for the generation of the summary, enabling the summary generation model to focus on the key content in the conference record and reducing the influence of noise. Through experiments, it is demonstrated that the manner provided by the embodiment of the application can generate an accurate summary on a long document such as an online conference, which has a long duration, sparse distribution of key information, and oralization, and exhibits a relatively optimal performance.

[0147] When the key sentence is extracted, the supervised and unsupervised manners are combined to enhance the information of the extracted key sentence. In addition, when the summary generation model generates the summary for the to-be-processed text by using the information of the key sentence, only the global attention of each Token in the key sentence is processed, and the local attention of other Tokens is processed, thereby reducing the computational overhead and enabling the model to smoothly generate the summary for the long text.

[0148] Application scenario 2, summary extraction of papers, news, and the like

[0149] With the extensive use of the Internet, the document data on the Internet grows explosively, and a large number of papers, news, and the like appear, and the reading of these documents such as papers and news is provided on some data platforms. In order to facilitate the user to quickly understand the content of the documents, it is usually necessary to generate a summary for the documents and provide the summary on the platform for the user to read and refer. In this application scenario, the manner provided in the embodiment of the application can also be used to generate a summary for the documents such as papers and news as the to-be-processed text. The specific process is not described herein.

[0150] The above describes particular embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than those in the embodiments and still achieve desirable results. Additionally, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or possible.

[0151] According to another aspect, embodiments of a text summary generation apparatus are provided. Figure 8 A schematic block diagram of the text summary generation apparatus according to one embodiment is shown. As shown, the apparatus 800 comprises a text acquisition unit 801, a key extraction unit 802 and a summary generation unit 803, wherein the main functions of each component unit are as follows: Figure 8 The text acquisition unit 801 is configured to acquire a text to be processed.

[0152] The key extraction unit 802 is configured to extract a key text unit from the text to be processed.

[0153] The summary generation unit 803 is configured to input information of the text to be processed and the key text unit into a summary generation model, acquire a summary of the text to be processed generated by the summary generation model using the information of the text to be processed and the key text unit, wherein the summary generation model is pre-trained based on a second deep learning model.

[0154] As one of the implementable ways, the key extraction unit 802 can be specifically configured to input the text to be processed into a key text extraction model, and acquire the key text unit extracted from the text to be processed by the key text extraction model, wherein the key text extraction model is pre-trained based on a first deep learning model.

[0155] As another implementable way, the key extraction unit 802 can be specifically configured to extract the key text unit from the text to be processed using a preset feature rule.

[0156] As still another implementable way, the key extraction unit 802 can be specifically configured to input the text to be processed into a key text extraction model, acquire a first key text unit extracted from the text to be processed by the key text extraction model, extract a second key text unit from the text to be processed using a preset feature rule, and fuse the first key text unit and the second key text unit to obtain the key text unit.

[0157] As one of the implementable ways, the key text extraction model can include a first encoding network and a classification network.

[0158] ​

[0159] The first encoding network encodes the to-be-processed text to obtain feature representations of each text unit in the to-be-processed text.

[0160] The first encoding network can first perform Embedding processing on each Token in the to-be-processed text, and then encode based on the result of the Embedding to obtain the feature representation of each Token. Then, the feature representation of each Token contained in each text unit is used to obtain the feature representation of each text unit, respectively. For example, the to-be-processed text includes n sentences, and for each sentence, the feature representations of the Tokens in a sentence are spliced to obtain a sentence-level feature representation. Each Token can include a character, a start symbol, a separator, and the like.

[0161] The classification network classifies each text unit using the feature representation of each text unit to obtain a classification result of whether each text unit is a key text unit.

[0162] The structure and principle of the key text extraction model described above can be referred to the related description in the method embodiments, which will not be repeated here.

[0163] As one of the implementable manners, the abstract generation model can include a second encoding network and a decoding network.

[0164] The second encoding network encodes the to-be-processed text using the information of the key text unit to obtain feature representations of each Token in the to-be-processed text.

[0165] The second encoding network performs embedding processing on the to-be-processed text to obtain embedding features of each Token in the to-be-processed text; and performs attention mechanism processing on the embedding features of each Token using the information of the key text unit, to obtain the feature representation of each Token, wherein the attention mechanism processing includes: using embedding features of all Tokens in the to-be-processed text when performing attention processing on each Token belonging to the key text unit, and using embedding features of Tokens within a preset window distance from the Token when performing attention processing on each Token not belonging to the key text unit.

[0166] The decoding network decodes the feature representations of each Token in the to-be-processed text to generate an abstract of the to-be-processed text.

[0167] The structure and principle of the abstract generation model described above can be referred to the related description in the method embodiments, which will not be repeated here.

[0168] Figure 9 A schematic block diagram of an apparatus for training a key text extraction model according to one embodiment is shown. As shown in FIG. 7, the apparatus includes a processor 710, a memory 720, and a communication interface 730. Figure 9As shown, the apparatus 900 includes a first sample obtaining unit 901 and a first model training unit 902, and main functions of constituent units are as follows:

[0169] The first sample obtaining unit 901 is configured to obtain a plurality of first training samples, and each first training sample includes a first text sample and a key text unit label annotated for the first text sample.

[0170] When training the text extraction model, some texts can be selected as the first text samples. When annotating the key text unit label for the first text sample, the annotation can be performed in an artificial manner. Since the artificial annotation manner is low in efficiency, some manners can be used to automatically determine the information of the key text unit of the first text sample.

[0171] As one of the implementable manners, the text with a known abstract can be used as the first text sample, that is, some texts with abstracts are used as the first text samples. Then, the similarity between each text unit in the first text sample and the abstract of the first text sample is calculated to determine the key text unit. For example, the GreedySearch manner is used to search m sentences with the highest similarity to the abstract from the first text sample as the key sentences, and m is a preset positive integer.

[0172] As another implementable manner, the unsupervised manner can be used to extract the key text unit, for example, each sentence in the to-be-processed text is scored based on a preset feature rule to obtain a score value of each sentence, and the sentence with a score value satisfying a preset requirement is used as the key sentence. For example, the TF-IDF of each word in the sentence can be counted, and the sentence is scored based on the TF-IDF of each word. For another example, the TextRank algorithm can be used to score each sentence based on the similarity between the sentences.

[0173] The first model training unit 902 is configured to train a key text extraction model using the plurality of first training samples, wherein the key text extraction model includes a first encoding network and a classification network; the first encoding network performs encoding processing on the first text sample to obtain a feature representation of each text unit in the first text sample; the classification network classifies each text unit using the feature representation of each text unit to obtain a classification result of whether each text unit is a key text unit; and a training target includes minimizing the difference between the classification result of the classification network and the key text unit label annotated for the first text sample.

[0174] The first encoding network can first perform embedding processing on each token in the first text sample, and then encode based on the embedding result to obtain the feature representation of each token. Then, the feature representation of each token contained in each text unit is used to obtain the feature representation of each text unit. For example, the first text sample includes n sentences. For each sentence, the feature representations of the tokens in the sentence are spliced to obtain the sentence-level feature representation. Each token can include characters, start symbols, and delimiters, etc.

[0175] The classification network is actually a binary classification network. For example, the classification network classifies each sentence based on the feature representation of the sentence, and the classification result is a key sentence or a non-key sentence. After the classification network classifies each sentence, the information of the key sentence in the first text sample can be obtained.

[0176] The training target used by the first model training unit 902 when training the key text extraction model includes minimizing the difference between the classification result of the classification network and the key text unit label of the first text sample. A loss function can be constructed according to the training target. In each iteration, the value of the loss function is used to update the model parameters by using methods such as gradient descent until the preset training end condition is met. The training end condition can include, for example, the value of the loss function being less than or equal to a preset loss function threshold, the number of iterations reaching a preset number threshold, etc.

[0177] Figure 10 A schematic block diagram of an apparatus for training an abstract generation model according to an embodiment is shown. As shown, the apparatus 1000 includes a second sample obtaining unit 1001 and a second model training unit 1002. The main functions of each component unit are as follows: Figure 9

[0178] The second sample obtaining unit 1001 is configured to obtain a plurality of second training samples. The second training sample includes a second text sample, information of a key text unit of the second text sample, and an abstract sample of the second text sample.

[0179] When training the abstract generation model, some texts with abstracts can be selected as the second text sample, and the abstracts can be selected as the abstract sample. Some texts can also be selected as the second text sample, and then the abstracts of the second text sample can be generated manually as the abstract sample.

[0180] The information of the key text unit of the second text sample can be manually labeled. Since the manual labeling method is low in efficiency, some methods can be used to automatically determine the information of the key text unit of the second text sample.

[0181] ​As one of the implementable manners, the key text units can be determined by calculating the similarity between each text unit in the second text sample and the summary of the second text sample. For example, m sentences with the highest similarity to the summary are searched from the second text sample as key sentences by means of GreedySearch, where m is a preset positive integer.

[0182] As another implementable manner, the key text units can be extracted from the second text sample by using a trained key text extraction model.

[0183] As still another implementable manner, the key text units can be extracted from the second text sample in an unsupervised manner. For example, each sentence in the text to be processed is scored based on a preset feature rule to obtain a score value of each sentence, and the sentence with a score value meeting a preset requirement is taken as a key sentence. For example, the TF-IDF of each word in the sentence can be counted, and the sentence is scored based on the TF-IDF of each word. For another example, the TextRank algorithm can be used to score each sentence based on the similarity between sentences.

[0184] As still another implementable manner, the key text units extracted by the above at least two manners can be fused to obtain the result of the finally extracted key text units. The fusion processing can be an intersection or a union, etc.

[0185] The second model training unit 1002 is configured to train a summary generation model by using a plurality of second training samples, where the summary generation model includes a second encoding network and a decoding network; the second encoding network encodes the second text sample by using the information of the key text units of the second text sample to obtain the feature representation of each Token in the second text sample; the decoding network decodes by using the feature representation of each Token in the second text sample to generate the summary of the second text sample; and the training target includes minimizing the difference between the summary of the second text sample generated by the decoding network and the summary sample of the second text.

[0186] The second encoding network obtains the embedding features of each Token in the second text sample after embedding processing of the second text sample; and the embedding features of each Token are processed by using the information of the key text units of the second text sample to obtain the feature representation of each Token, where the attention mechanism processing includes: when attention processing is performed on each Token belonging to the key text units, the embedding features of all Tokens in the second text sample are used; and when attention processing is performed on each Token not belonging to the key text units, the embedding features of each Token within a preset window distance from the Token are used.

[0187] The objective of the training employed by the second model training unit 1002 when training the summary generation model includes minimizing the difference between the summary generated by the decoding network and the summary sample of the second text sample. A loss function can be constructed according to the training objective, and the value of the loss function is used in each iteration to update the model parameters using methods such as gradient descent until a preset training end condition is met. The training end condition can include, for example, the value of the loss function being less than or equal to a preset loss function threshold, the number of iterations reaching a preset number threshold, and the like.

[0188] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, they are described more simply, and the relevant parts can be referred to the part of the method embodiments. The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. Those skilled in the art can understand and implement without creative labor.

[0189] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0190] In addition, the embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the steps of the method in any one of the preceding method embodiments.

[0191] An electronic device, comprising:

[0192] one or more processors; and

[0193] a memory associated with the one or more processors, the memory configured to store program instructions that, when executed by the one or more processors, perform the steps of the method of any one of the preceding method embodiments.

[0194] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method according to any one of the preceding method embodiments.

[0195] wherein, Figure 11 An exemplary architecture of the electronic device is shown, which can specifically include a processor 1110, a video display adapter 1111, a disk drive 1112, an input / output interface 1113, a network interface 1114, and a memory 1120. The processor 1110, the video display adapter 1111, the disk drive 1112, the input / output interface 1113, the network interface 1114, and the memory 1120 can be communicatively connected through a communication bus 1130.

[0196] The processor 1110 can be implemented in the form of a general-purpose CPU, a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the application.

[0197] The memory 1120 can be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1120 can store an operating system 1121 for controlling the operation of the electronic device 1100, a basic input / output system (BIOS) 1122 for controlling the low-level operation of the electronic device 1100. In addition, a web browser 1123, a data storage management system 1124, and a text summary generation apparatus 1125, etc. can also be stored. The text summary generation apparatus 1125 can be an application program that implements the above steps in the embodiments of the application. In summary, when the technical solutions provided by the application are implemented by software or firmware, the related program codes are stored in the memory 1120 and executed by the processor 1110.

[0198] The input / output interface 1113 is configured to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input devices can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices can include a display, a speaker, a vibrator, an indicator light, etc.

[0199] The network interface 1114 is configured to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through wired mode (such as USB, network cable, etc.), or realize communication through wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0200] The bus 1130 includes a path for transmitting information between various components (such as the processor 1110, the video display adapter 1111, the disk drive 1112, the input / output interface 1113, the network interface 1114, and the memory 1120) of the device.

[0201] It should be noted that although the above device only shows the processor 1110, the video display adapter 1111, the disk drive 1112, the input / output interface 1113, the network interface 1114, the memory 1120, the bus 1130, etc., in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary to implement the scheme of the present application, and does not have to contain all the components shown in the figure.

[0202] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and the necessary general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer program product, which can be stored in a storage medium such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions for making a computer device (which can be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of the present application.

[0203] The above describes the technical solutions provided by the present application in detail, and the principles and implementation manners of the present application are described by applying specific examples; the above description of the embodiments is only to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for generating text summaries, characterized in that, The method includes: Get the text to be processed; Extract key text units from the text to be processed; The information of the text to be processed and the key text units is input into the summary generation model to obtain a summary of the text to be processed generated by the summary generation model using the information of the text to be processed and the key text units; The summary generation model mentioned above is pre-trained based on a second deep learning model; The summary generation model encodes the text to be processed using the information of the key text units to obtain the feature representation of each token in the text to be processed. The feature representation of each token is obtained by processing the embedding features of each token using the information of the key text units through an attention mechanism. The attention mechanism includes: when performing attention processing on each token belonging to the key text units, using the embedding features of all tokens in the text to be processed; when performing attention processing on each token not belonging to the key text units, using the embedding features of each token within a preset window distance of the token.

2. The method according to claim 1, characterized in that, Extracting key text units from the text to be processed includes: The text to be processed is input into a key text extraction model to obtain key text units extracted from the text, wherein the key text extraction model is pre-trained based on a first deep learning model; or... Extract key text units from the text to be processed using preset feature rules; or, The text to be processed is input into the key text extraction model to obtain the first key text unit extracted from the text to be processed by the key text extraction model. The second key text unit is extracted from the text to be processed using preset feature rules. The first key text unit and the second key text unit are fused to obtain the key text unit.

3. The method according to claim 2, characterized in that, The key text extraction model includes a first encoding network and a classification network; The first encoding network encodes the text to be processed to obtain the feature representation of each text unit in the text to be processed. The classification network uses the feature representations of each text unit to classify each text unit, and obtains the classification result of whether each text unit is a key text unit.

4. The method according to claim 1, characterized in that, The summary generation model includes a second encoding network and a decoding network; The second encoding network uses the information of the key text units to encode the text to be processed, thereby obtaining the feature representation of each token in the text to be processed; The decoding network uses the feature representations of each token in the text to be processed to perform decoding processing and generate a summary of the text to be processed.

5. The method according to any one of claims 1 to 4, characterized in that, The method is applied to online meeting scenarios, where the text to be processed is the meeting record of the online meeting, the key text unit is the key sentence, and the summary is the meeting summary of the meeting record.

6. A method for training a summary generation model, characterized in that, The method includes: Obtain multiple second training samples, which include second text samples, information on key text units of the second text samples, and summary samples of the second text samples; The summarization generation model is trained using the plurality of second training samples, wherein the summarization generation model includes a second encoding network and a decoding network; The second encoding network uses information from key text units of the second text sample to encode the second text sample, thereby obtaining feature representations of each token in the second text sample; The decoding network uses the feature representations of each token in the second text sample to perform decoding processing and generate a summary of the second text sample; The training objective includes minimizing the difference between the summary of the second text sample generated by the decoding network and the summary sample of the second text. The summary generation model uses the information of the key text units to encode the text to be processed, thereby obtaining the feature representation of each token in the text to be processed. The feature representation of each token is obtained by using the information of the key text units to process the embedding features of each token through an attention mechanism. The attention mechanism includes the following processing methods: when performing attention processing on each token belonging to a key text unit, the embedding features of all tokens in the text to be processed are utilized; when performing attention processing on each token not belonging to a key text unit, the embedding features of each token within a preset window distance from the token are utilized.

7. A method for training a key text extraction model, characterized in that, The method includes: Multiple first training samples are obtained, including first text samples and key text unit labels annotated in the first text samples; A key text extraction model is trained using the multiple first training samples, wherein the key text extraction model includes a first encoding network and a classification network; The first encoding network encodes the first text sample to obtain the feature representation of each text unit in the first text sample; The classification network uses the feature representation of each text unit to classify each text unit, and obtains the classification result of whether each text unit is a key text unit; The training objective includes minimizing the difference between the classification result of the classification network and the key text unit labels annotated in the first text sample; The key text units are extracted from the text to be processed. The feature representation of each token in the text to be processed is obtained by encoding the text to be processed using the information of the key text units by the summary generation model. The feature representation of each token is obtained by processing the embedding features of each token using the information of the key text units through an attention mechanism. The attention mechanism includes: when performing attention processing on each token belonging to the key text units, the embedding features of all tokens in the text to be processed are used; when performing attention processing on each token not belonging to the key text units, the embedding features of each token within a preset window distance of the token are used.

8. A text summarization generation apparatus, characterized in that, The device includes: The text acquisition unit is configured to acquire the text to be processed. The key extraction unit is configured to extract key text units from the text to be processed. A summary generation unit is configured to input the text to be processed and the information of the key text units into a summary generation model, and obtain a summary of the text to be processed generated by the summary generation model using the information of the text to be processed and the key text units; wherein the summary generation model is pre-trained based on a second deep learning model, wherein the summary generation model encodes the text to be processed using the information of the key text units to obtain the feature representation of each token in the text to be processed, and the feature representation of each token is obtained by processing the embedding features of each token using the information of the key text units through an attention mechanism, wherein the attention mechanism processing includes: when performing attention processing on each token belonging to the key text units, using the embedding features of all tokens in the text to be processed, and when performing attention processing on each token not belonging to the key text units, using the embedding features of each token within a preset window distance of the token.

9. An apparatus for training a summary generation model, characterized in that, The device includes: The second sample acquisition unit is configured to acquire multiple second training samples, the second training samples including second text samples, information of key text units of the second text samples, and summary samples of the second text samples; The second model training unit is configured to train the summary generation model using the plurality of second training samples. The summary generation model includes a second encoding network and a decoding network. The second encoding network encodes the second text sample using information from key text units to obtain feature representations of each token in the second text sample. The decoding network decodes the second text sample using the feature representations of each token to generate a summary of the second text sample. The training objective includes minimizing the difference between the summary of the second text sample generated by the decoding network and the summary sample of the second text. The summary generation model encodes the text to be processed using information from the key text units to obtain feature representations of each token in the text to be processed. The feature representations of each token are obtained by processing the embedding features of each token using an attention mechanism based on information from the key text units. The attention mechanism processing includes: when performing attention processing on tokens belonging to key text units, using the embedding features of all tokens in the text to be processed; when performing attention processing on tokens not belonging to key text units, using the embedding features of tokens within a preset window distance of the token.

10. An apparatus for training a key text extraction model, characterized in that, The device includes: The first sample acquisition unit is configured to acquire multiple first training samples, the first training samples including first text samples and key text unit labels annotated on the first text samples; The first model training unit is configured to train a key text extraction model using the plurality of first training samples, wherein the key text extraction model includes a first encoding network and a classification network; the first encoding network encodes the first text samples to obtain feature representations of each text unit in the first text samples; the classification network classifies each text unit using the feature representations of each text unit to obtain a classification result of whether each text unit is a key text unit; the training objective includes minimizing the difference between the classification result of the classification network and the key text unit labels labeled on the first text samples, wherein the key text units are extracted from the text to be processed, the feature representations of each token in the text to be processed are obtained by the summarization generation model using the information of the key text units to encode the text to be processed, and the feature representations of each token are obtained by using the information of the key text units to process the embedding features of each token using an attention mechanism, wherein the attention mechanism processing includes: when performing attention processing on each token belonging to a key text unit, using the embedding features of all tokens in the text to be processed, and when performing attention processing on each token not belonging to a key text unit, using the embedding features of each token within a preset window distance of the token.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.

12. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 7.

Images