Summary extraction model training method, text summary generation method and related equipment

Abstract

The present application belongs to the field of computer technology, and specifically relates to a training method for a summary extraction model, a text summary generation method and related equipment. Embodiments of the present invention can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving. The method includes: obtaining a first loss function for training a summary extraction model; periodically updating the training parameters of an auxiliary encoder according to the training parameters of the encoder; calculating a first similarity between a text encoding of a text sample and a summary encoding of a corresponding summary sample, calculating a second similarity between the text encoding of a text sample and summary encodings of other summary samples, and determining a second loss function according to the first similarity and the second similarity; and training the summary extraction model together through the first loss function and the second loss function. Based on the present application, the accuracy of text summary generation of the summary extraction model can be improved, so that the generated text summary is more consistent with the semantics of the original text to be processed.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a training method for a summary extraction model, a text summary generation method, device, medium and equipment. Background Art

[0002] With the continuous development of the Internet, the text data in the network has exploded. If people want to search for the information they need from the Internet, they need to spend a lot of energy and time. How to quickly obtain effective information from the text requires automatic text summary generation technology.

[0003] Existing text summary generation methods may lose information when extracting features from the original text, resulting in inaccurate text summaries. Inaccurate text summaries can cause readers to have difficulty reading or receive erroneous information, which directly limits the application scenarios and effects of machine text summary extraction. Therefore, how to improve the accuracy of generated summary text is a technical problem that needs to be solved urgently.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of this application is to provide a training method, device, medium and electronic device for a summary extraction model, which at least to a certain extent overcomes the technical problem of how to improve the accuracy of the generated summary text in the related art.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0007] According to one aspect of an embodiment of the present application, a training method for a summary extraction model is provided, wherein the summary extraction model includes an encoder and a decoder, and the summary extraction model is used to generate a text code by semantically encoding a text or a text sample to be processed through the encoder, and decode the text code through the decoder to obtain a text summary, and the method includes:

[0008] The summary extraction model is trained according to a training data set to obtain a first loss function and training parameters of the encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples;

[0009] Initializing parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically updating the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model;

[0010] Inputting each of the summary samples into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample;

[0011] Calculating a first similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample, calculating a second similarity between the text encoding of the text sample and the summary encodings of the summary samples respectively corresponding to a plurality of other text samples in the training data set, and determining a second loss function according to the first similarity and the second similarity;

[0012] The summary extraction model is trained by using the first loss function and the second loss function together.

[0013] According to one aspect of an embodiment of the present application, a training device for a summary extraction model is provided, wherein the summary extraction model includes an encoder and a decoder, wherein the summary extraction model is used to generate a text code by semantically encoding a text or a text sample to be processed through the encoder, and to decode the text code through the decoder to obtain a text summary, wherein the device includes:

[0014] A first loss function determination module is configured to train the summary extraction model according to a training data set to obtain a first loss function and training parameters of the encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples;

[0015] a training parameter updating module, configured to initialize parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically update the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model;

[0016] an auxiliary encoder encoding module, configured to input each of the summary samples into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample;

[0017] A second loss function determination module is configured to calculate a first similarity between the text encoding of the text sample and a summary encoding of a summary sample corresponding to the text sample, calculate a second similarity between the text encoding of the text sample and summary encodings of summary samples respectively corresponding to a plurality of other text samples in the training data set, and determine a second loss function according to the first similarity and the second similarity;

[0018] The model training module is configured to train the summary extraction model by using the first loss function and the second loss function.

[0019] In some embodiments of the present application, based on the above technical solution, the auxiliary encoder encoding module includes:

[0020] The auxiliary encoding unit is configured to input the summary sample of the current training round into the auxiliary encoder to obtain the sample code corresponding to the summary sample of the current training round.

[0021] In some embodiments of the present application, based on the above technical solution, the second loss function determination module includes:

[0022] A first similarity calculation unit is configured to calculate a first similarity between a text encoding of a text sample of a current training round and a summary encoding of a summary sample of the current training round;

[0023] The summary code storage unit is configured to store the summary code of the summary sample of the current training round into the storage queue.

[0024] In some embodiments of the present application, based on the above technical solution, the second loss function determination module further includes:

[0025] a summary code fetching unit configured to select from the storage queue a plurality of summary codes stored in the storage queue in a training round before a current training round;

[0026] The second similarity calculation unit is configured to calculate second similarities between the text encoding of the text sample of the current training round and a plurality of summary encodings stored in the storage queue in the training round before the current training round.

[0027] In some embodiments of the present application, based on the above technical solution, the training parameter updating module includes:

[0028] A training parameter reading unit, configured to periodically read the training parameters of the encoder;

[0029] A parameter update value acquisition unit, configured to perform weighted summation of the training parameters of the encoder and the training parameters of the auxiliary encoder to obtain a parameter update value;

[0030] A parameter updating unit is configured to update the training parameters of the auxiliary encoder according to the parameter update value.

[0031] In some embodiments of the present application, based on the above technical solution, the model training module includes:

[0032] a comprehensive loss function acquisition unit, configured to perform a weighted summation of the first loss function and the second loss function to obtain a comprehensive loss function;

[0033] The model training unit is configured to train the summary extraction model through the comprehensive loss function.

[0034] In some embodiments of the present application, based on the above technical solution, the second loss function determination module further includes:

[0035] a sum value obtaining unit, configured to perform a sum operation on the first similarity and a plurality of the second similarities to obtain a sum value;

[0036] a ratio obtaining unit, configured to obtain a ratio of the first similarity to the sum value;

[0037] A second loss function determining unit is configured to determine the second loss function according to the ratio, wherein the second loss function has a negative correlation with the ratio.

[0038] According to one aspect of an embodiment of the present application, a method for generating a summary is provided, the method comprising:

[0039] Initializing a summary extraction model, the summary extraction model comprising an encoder and a decoder, the summary extraction model being used to generate a text code by semantically encoding the text or text sample to be processed through the encoder, and to decode the text code through the decoder to obtain a text summary;

[0040] The summary extraction model is trained according to a training data set to obtain a first loss function and training parameters of the encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples;

[0041] Initializing parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically updating the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model;

[0042] Inputting each of the summary samples into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample;

[0043] Calculating a first similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample, calculating a second similarity between the text encoding of the text sample and the summary encodings of the summary samples respectively corresponding to a plurality of other text samples in the training data set, and determining a second loss function according to the first similarity and the second similarity;

[0044] Training the summary extraction model by using the first loss function and the second loss function together;

[0045] The text to be processed is input into the trained summary extraction model to obtain a text summary corresponding to the text to be processed.

[0046] According to one aspect of an embodiment of the present application, a summary generation device is provided, the device comprising:

[0047] A model initialization module is configured to initialize a summary extraction model, wherein the summary extraction model includes an encoder and a decoder, wherein the summary extraction model is used to generate a text code by semantically encoding a text or a text sample to be processed through the encoder, and to decode the text code through the decoder to obtain a text summary;

[0048] A first loss function determination module is configured to train the summary extraction model according to a training data set to obtain a first loss function and training parameters of the encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples;

[0049] a training parameter updating module, configured to initialize parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically update the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model;

[0050] an auxiliary encoder encoding module, configured to input each of the summary samples into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample;

[0051] A second loss function determination module is configured to calculate a first similarity between the text encoding of the text sample and a summary encoding of a summary sample corresponding to the text sample, calculate a second similarity between the text encoding of the text sample and summary encodings of summary samples respectively corresponding to a plurality of other text samples in the training data set, and determine a second loss function according to the first similarity and the second similarity;

[0052] A model training module, configured to train the summary extraction model by using the first loss function and the second loss function;

[0053] The text summary generation module is configured to input the text to be processed into the trained summary extraction model to obtain the text summary corresponding to the text to be processed.

[0054] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the training method of the summary extraction model or the summary generation method in the above technical solution is implemented.

[0055] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the training method of the summary extraction model or the summary generation method in the above technical solution by executing the executable instructions.

[0056] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the training method of the summary extraction model in the above technical solution.

[0057] In the technical solution provided in the embodiment of the present application, the summary extraction model is trained according to the training data set to obtain the training parameters of the first loss function and the encoder; the first similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample is calculated, and the second similarity between the text encoding of the text sample and the summary encoding of the summary samples corresponding to multiple other text samples in the training data set is calculated, and the second loss function is determined according to the first similarity and the second similarity; the summary extraction model is trained together with the first loss function and the second loss function; the present application can achieve the effect of reducing the semantic distance between the text and the corresponding summary text and increasing the semantic distance between the text and other summary texts by calculating the first similarity and the second similarity between the samples and determining the second loss function without adding additional training samples, thereby improving the encoding accuracy of the summary extraction model, and then improving the text summary generation accuracy of the summary extraction model, so that the generated text summary is more consistent with the semantics of the original text to be processed.

[0058] Furthermore, the present application periodically updates the training parameters of the auxiliary encoder according to the training parameters of the encoder, which can make the representation of the text by the auxiliary encoder and the encoder similar, thereby improving the training effect of the model and further improving the encoding accuracy of the summary extraction model.

[0059] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 The following schematically shows an exemplary device architecture block diagram applying the technical solution of the present application.

[0062] Figure 2 The flowchart schematically shows the steps of the training method of the summary extraction model in some embodiments of the present application.

[0063] Figure 3 The flowchart schematically shows the steps of periodically updating the training parameters of the auxiliary encoder according to the training parameters of the encoder in a certain embodiment of the present application.

[0064] Figure 4 The diagram schematically shows a training process of a training method for a summary extraction model according to a certain embodiment of the present application.

[0065] Figure 5 The flowchart schematically shows the steps of calculating the first similarity between the text encoding of a text sample and the summary encoding of a summary sample corresponding to the text sample in a certain embodiment of the present application.

[0066] Figure 6 The flowchart schematically shows the steps of calculating the second similarity between the text encoding of a text sample and the summary encodings of summary samples corresponding to multiple other text samples in the training data set in a certain embodiment of the present application.

[0067] Figure 7 The flowchart schematically shows the steps of determining the second loss function according to the first similarity and the second similarity in a certain embodiment of the present application.

[0068] Figure 8 The flowchart schematically shows the auxiliary steps between calculating the first similarity and calculating the second similarity in a certain embodiment of the present application.

[0069] Fig. 9 The flowchart schematically shows the steps of training the summary extraction model by using the first loss function and the second loss function in an embodiment of the present application.

[0070] Fig.10 The flowchart schematically shows the steps of the training method of the summary extraction model in some embodiments of the present application.

[0071] Fig.11The structural block diagram of the training device for the summary extraction model provided in an embodiment of the present application is schematically shown.

[0072] Fig.12 The structural block diagram of the summary generation device provided in an embodiment of the present application is schematically shown.

[0073] Fig.13 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0074] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0075] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0076] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0077] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0078] Before describing in detail the technical solutions such as the training method of the summary extraction model and the training device of the summary extraction model provided in the embodiments of the present application, a brief introduction to the artificial intelligence technology involved in some embodiments of the present application is first given.

[0079] Artificial Intelligence (AI) is the theory, method, technology and application device that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0080] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction devices, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0081] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0082] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0083] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0084] The system involved in the embodiments of the present application may be a distributed system formed by connecting a client and multiple nodes (any form of computing devices in an access network, such as a server and a user terminal) through network communication.

[0085] The following is a detailed description of the training method of the summary extraction model, the summary generation method, and the corresponding device provided by the present application in conjunction with the specific implementation methods.

[0086] Figure 1 The following schematically shows an exemplary device architecture block diagram applying the technical solution of the present application.

[0087] like Figure 1 As shown, the device architecture 100 may include a terminal device 110, a network 120 and a server 130. The terminal device 110 may include various electronic devices such as a smart phone, a tablet computer, a laptop computer, a desktop computer, etc. The server 130 may be an independent physical server, or a server cluster or a distributed device composed of multiple physical servers, or a cloud server providing cloud computing services. The network 120 may be a communication medium of various connection types that can provide a communication link between the terminal device 110 and the server 130, such as a wired communication link or a wireless communication link.

[0088] According to the implementation requirements, the device architecture in the embodiment of the present application can have any number of terminal devices, networks and servers. For example, the server 130 can be a server group composed of multiple server devices. In addition, the technical solution provided in the embodiment of the present application can be applied to the terminal device 110, can also be applied to the server 130, or can be implemented by the terminal device 110 and the server 130 together, and the present application does not make any special restrictions on this.

[0089] For example, the server 130 can execute the training method of the summary extraction model provided by the present application, train the summary extraction model according to the training data set, and obtain the training parameters of the first loss function and the encoder; calculate the first similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample, calculate the second similarity between the text encoding of the text sample and the summary encoding of the summary samples corresponding to multiple other text samples in the training data set, and determine the second loss function according to the first similarity and the second similarity; train the summary extraction model together with the first loss function and the second loss function; enable the present application to achieve the effect of reducing the semantic distance between the text and the corresponding summary text and increasing the semantic distance between the text and other summary texts by calculating the first similarity and the second similarity between the samples and determining the second loss function without adding additional training samples, thereby improving the encoding accuracy of the summary extraction model, and then improving the text summary generation accuracy of the summary extraction model, so that the generated text summary is more consistent with the semantics of the original text to be processed.

[0090] Furthermore, the present application periodically updates the training parameters of the auxiliary encoder according to the training parameters of the encoder, which can make the representation of the text by the auxiliary encoder and the encoder similar, thereby improving the training effect of the model and further improving the encoding accuracy of the summary extraction model.

[0091] The following is a detailed description of the training method of the summary extraction model provided in this application in conjunction with the specific implementation methods.

[0092] Figure 2 The flowchart schematically shows the steps of the training method of the summary extraction model of certain embodiments of the present application. The execution subject of the training method of the summary extraction model can be a terminal device or a server, etc., and the present application does not limit this. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. The embodiments of the present invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0093] like Figure 2 As shown, the training method of the summary extraction model may mainly include the following steps S210 to S250.

[0094] S210. Train the summary extraction model according to the training data set to obtain the first loss function and the training parameters of the encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples.

[0095] Specifically, the text sample may include original news, original article, web page text, etc. The summary sample is a concise text that summarizes the text sample.

[0096] For example, a text sample may be the original news text: "Last night, many people were found smoking on a China United Airlines flight from Chengdu to Beijing. Later, due to weather reasons, the plane made an emergency landing at Taiyuan Airport. Several passengers were found smoking near the cabin door. Some passengers requested re-security check, and the captain decided to continue the flight, which caused a conflict between the crew and non-smoking passengers. China United Airlines is currently contacting the crew for verification." A summary sample may be a news summary: "Many people were smoking on the flight from Chengdu to Beijing, and the crew had a conflict with non-smoking passengers."

[0097] In some implementations, a cross entropy loss function may be used as the first loss function obtained by training the summary extraction model. Alternatively, a triplet loss function (Triplet Loss) may be used as the first loss function obtained by training the summary extraction model.

[0098] S220. Initialize the parameters of the auxiliary encoder according to the training parameters of the encoder, and periodically update the training parameters of the auxiliary encoder according to the training parameters of the encoder. The auxiliary encoder is used to assist in the training of the summary extraction model.

[0099] Specifically, the auxiliary encoder may be a momentum encoder. In some implementations, the auxiliary encoder may also be other encoders, which is not limited in the present application.

[0100] Specifically, in the model initialization stage, the auxiliary encoder can be initialized according to the training parameters of the encoder of the summary extraction model during initialization, so that the initialization of the auxiliary encoder is the same as the initialization of the encoder. Then, in the model training stage, the training parameters of the auxiliary encoder can be periodically updated according to the training parameters of the encoder. In this way, the representation of the text by the auxiliary encoder and the encoder can be similar, so that the semantic encoding process of the text sample can be similar to the processing of the semantic encoding process of the summary sample, which can better shorten the spatial distance between the semantic vectors of the text encoding and the corresponding summary sample, and increase the spatial distance between the semantic vectors of the text encoding and other summary samples, thereby improving the training effect of the model.

[0101] Figure 3 The flowchart schematically shows the steps of periodically updating the training parameters of the auxiliary encoder according to the training parameters of the encoder in a certain embodiment of the present application. Figure 3 As shown, based on the above embodiment, step S220 of periodically updating the training parameters of the auxiliary encoder according to the training parameters of the encoder may further include the following steps S310 to S330.

[0102] S310. Periodically read the training parameters of the encoder;

[0103] S320. Perform a weighted summation of the training parameters of the encoder and the training parameters of the auxiliary encoder to obtain a parameter update value;

[0104] S330. Update the training parameters of the auxiliary encoder according to the parameter update value.

[0105] S230. Input each summary sample into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample.

[0106] Specifically, the periodicity may be every preset training rounds, or every preset time length.

[0107] The training parameters of the encoder and the training parameters of the auxiliary encoder are weighted and summed to obtain parameter update values, and the training parameters of the auxiliary encoder are updated according to the parameter update values. In some embodiments, it can be expressed as the following formula:

[0108] (1):θ k ←mθ k +(1-m)θ q

[0109] Among them, θ k are the training parameters of the auxiliary encoder, θ q is the training parameter of the encoder. m can be a preset constant less than 1 such as 0.999, 0.998, 0.990, etc. k +(1-m)θ q For θ k The assignment update can make the auxiliary encoder updated in each training round, make the parameters of the auxiliary encoder tend to be similar to the parameters of the encoder during training, make the representation of the text by the auxiliary encoder and the encoder similar, so that the semantic encoding process of the text sample can be similar to the semantic encoding process of the summary sample, and can better shorten the spatial distance between the semantic vectors of the text encoding and the corresponding summary sample, and increase the spatial distance between the semantic vectors of the text encoding and other summary samples, thereby improving the training effect of the model.

[0110] In some implementations, m in equation (1) may be a preset constant that is close to 1 but less than 1. Thus, the parameter update iteration process of the auxiliary encoder can be slow, so that the semantic encoding process of the auxiliary encoder changes less in adjacent training rounds, thereby ensuring the stability of the model.

[0111] In some implementations, based on the above embodiments, step S230 of inputting each summary sample into the auxiliary encoder for semantic encoding to obtain the summary encoding of the summary sample may further include the following steps:

[0112] Input the summary sample of the current training round into the auxiliary encoder to obtain the sample code corresponding to the summary sample of the current training round;

[0113] Figure 4 The schematic diagram schematically shows the training process of the training method of the summary extraction model of a certain embodiment of the present application. Figure 4 In each training round, the text sample Y in the "text-summary" sample pair Y-Y' in the training data set is input into the encoder to semantically encode the text sample Y to obtain the text encoding y. Then, the text encoding y is input into the decoder to decode the text sample encoding to obtain the actual text output of the decoder. The summary text Y' in the sample pair Y-Y' is used as the target text output of the decoder. The actual text output and the target text output are calculated to obtain the first loss function L1. In addition, during the training process, the first loss function performs parameter backpropagation on the encoder and decoder in the model, so that the parameters of the encoder and the parameters of the decoder are continuously updated during the model training. Then, with one training round as a cycle, or with a preset time length as a cycle, the training parameters of the encoder are pushed to the auxiliary encoder to update the training parameters of the auxiliary encoder.

[0114] In each training round, the summary sample Y' in the "text-summary" sample pair Y-Y' in the training data set is input into the auxiliary encoder, the summary text Y' is semantically encoded to obtain the summary code y', and the summary code y' is pushed to the calculation unit B. Then, multiple summary codes stored in the storage queue in the training round before the current training round are selected from the storage queue and pushed to the calculation unit B. Then, the summary code y' of the summary sample of the current training round is stored in the storage queue. It should be noted that the storage space of the storage queue can be a preset size. When the storage space of the preset size in the storage queue is exhausted, when the summary code y' is stored in the storage queue, the earliest summary code of the training round in the storage queue can be popped out of the queue and discarded. In addition, the text code y of the text sample Y is also pushed to the calculation unit B. Therefore, the calculation unit B calculates the second loss function L2 based on the text code y, the summary code y' and the multiple summary codes pushed by the storage queue.

[0115] Then, the comprehensive loss function L can be determined based on the first loss function L1 and the second loss function L2. 总 , and train the summary extraction model through a comprehensive loss function.

[0116] S240. Calculate a first similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample, calculate a second similarity between the text encoding of the text sample and the summary encodings of the summary samples corresponding to multiple other text samples in the training data set, and determine a second loss function based on the first similarity and the second similarity.

[0117] Specifically, when the first loss function adopts the cross entropy loss function, the function form of the cross entropy loss function can be used, and the second loss function obtained by training the summary extraction model can be determined according to the first similarity and the second similarity. Alternatively, when the first loss function adopts the triplet loss function, the function form of the triplet loss function (TripletLoss) can also be used, and the second loss function obtained by training the summary extraction model can be determined according to the first similarity and the second similarity. In this way, the training effect of the model can be improved.

[0118] Figure 5 The flowchart schematically shows the steps of calculating the first similarity between the text encoding of a text sample and the summary encoding of a summary sample corresponding to the text sample in a certain embodiment of the present application. Figure 5 As shown, based on the above embodiment, the step S240 of calculating the first similarity between the text code of the text sample and the summary code of the summary sample corresponding to the text sample may further include the following steps S510 and S520.

[0119] S510. Calculate the first similarity between the text encoding of the text sample of the current training round and the summary encoding of the summary sample of the current training round;

[0120] S520. Store the summary code of the summary sample of the current training round into a storage queue.

[0121] The first similarity may be semantic similarity, or the first similarity may be a mathematical expression of similarity after mathematical calculation of the semantic similarity.

[0122] In some embodiments, the storage space of the storage queue can be much larger than the data size of the summary code, so that a large number of summary codes can be stored in the storage queue. Combined with using summary samples other than the summary sample corresponding to the text sample as negative samples corresponding to the text sample, the number of negative samples can be large enough, and the spatial distance between the vector representations of the text sample and the positive sample can be shortened, and the spatial distance between the vector representations of the text sample and the negative sample can be shortened. This can make the text encoding space distribution more uniform and the semantic encoding more accurate, thereby improving the encoding accuracy of the summary extraction model and even the accuracy of the generated summary.

[0123] Figure 6The flowchart schematically shows the steps of calculating the second similarity between the text encoding of a text sample and the summary encodings of the summary samples corresponding to multiple other text samples in the training data set in a certain embodiment of the present application. Figure 6 As shown, based on the above embodiment, the step S240 of calculating the second similarity between the text encoding of the text sample and the summary encodings of the summary samples corresponding to multiple other text samples in the training data set may further include the following steps S610 and S620.

[0124] S610. Select multiple summary codes stored in the storage queue in the training round before the current training round from the storage queue;

[0125] S620. Calculate the second similarity between the text encoding of the text sample in the current training round and the plurality of summary encodings stored in the storage queue in the training round before the current training round.

[0126] The second similarity may be semantic similarity, or the second similarity may be a mathematical expression of similarity after mathematical calculation of the semantic similarity.

[0127] In some embodiments, more than a preset number of summary codes stored in a storage queue in a training round before the current training round can be selected from the storage queue, thereby making the number of summary codes greater than the preset number, and combining the summary samples other than the summary sample corresponding to the text sample as the negative samples corresponding to the text sample, so that the number of negative samples can be sufficient, and at the same time, the summary sample corresponding to the text sample is used as the positive sample of the text sample, which can make the spatial distribution of the text encoding more uniform and the semantic encoding more accurate by shortening the spatial distance between the vector representation of the text sample and the positive sample and increasing the spatial distance between the vector representation of the text sample and the negative sample, thereby improving the encoding accuracy of the summary extraction model and even the accuracy of the generated summary.

[0128] Figure 7 The flowchart of the steps of determining the second loss function according to the first similarity and the second similarity in a certain embodiment of the present application is schematically shown. Figure 7 As shown, based on the above embodiment, step S240 of determining the second loss function according to the first similarity and the second similarity may further include the following steps S710 to S730.

[0129] S710. Perform a sum operation on the first similarity and the plurality of second similarities to obtain a sum value;

[0130] S720. Obtaining a ratio of the first similarity to the sum value;

[0131] S730. Determine a second loss function according to the ratio, wherein the second loss function has a negative correlation with the ratio.

[0132] It can be understood that the second loss function has a negative correlation with the ratio, that is, when the ratio of the first similarity to the summed value increases, the second loss function will decrease accordingly; when the ratio of the first similarity to the summed value decreases, the second loss function will increase accordingly. In some embodiments, the model and the auxiliary encoder can be trained with the goal of reducing the value of the second loss function. Thus, it is possible to improve the ratio of the first similarity to the summed value as the model is trained. In other words, the distance between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample in the vector space can be shortened, and the distance between the text encoding of the text sample and the such encoding of other summary samples in the vector space can be shortened; the semantic similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample is increased, and the semantic similarity between the text encoding of the text sample and the summary encoding of other summary samples is reduced; the spatial distance between the vector representation of the text sample and the positive sample is shortened, and the spatial distance between the vector representation of the text sample and the negative sample is shortened.

[0133] In some embodiments, the second loss function L2 can be calculated according to the following formula (2):

[0134] (2):

[0135] Among them, exp(m i / T) can be used to represent the semantic similarity m between the text encoding and the corresponding summary encoding i The first similarity obtained after mathematical calculation, exp(n ij / T) can be used to represent the semantic similarity n obtained for multiple summary codes other than the text code and the corresponding summary code. ij The second similarity n obtained after mathematical calculation ij , N is used to represent the number of introduced negative samples, that is, the total number of the second similarity, and T is a preset constant.

[0136] Figure 8 The flowchart schematically shows the auxiliary steps between calculating the first similarity and calculating the second similarity in a certain embodiment of the present application. Figure 8 As shown in Figure 2, after obtaining four sets of text codes and summary codes (y0, y0'), (y1, y1'), (y2, y2'), and (y3, y3') based on four "text-summary" sample pairs, we can generate Figure 8At this time, except for the m0, m1, m2, and m3 obtained on the diagonal of the matrix, which are the first similarity between the text code and the corresponding summary code, the n in the matrix 01 、n 02 、n 03 、n 11 、n 12 、n 13 、n 21 、n 22 、n 23 、n 31 、n 32 、n 33 It is the second similarity obtained by the summary code other than the text code and the corresponding summary code. In some embodiments, the corresponding multiple first similarities and second similarities calculated according to multiple groups of text codes and summary codes can be displayed on the display interface in the form of a matrix arrangement. In this way, the similarities between each text code and summary code can be intuitively displayed, which is conducive to intuitively seeing the training effect and finding problems during the model training process, so that problems can be found in time to improve development efficiency, and it is conducive to timely finding problems in the training process to improve the model training effect.

[0137] In some implementations, formula (2) may also be optimized to obtain formula (3), so that the second loss function L2 is calculated according to the following formula (3):

[0138] (3):

[0139] Here, sim(y,y') is used to represent the first similarity between the text code y and the corresponding summary code y'. It is used to represent the sum of similarities between the text code y and the corresponding summary code y' and the summary code y' other than the corresponding summary code, including the first similarity between the text code y and the corresponding summary code y', and multiple second similarities obtained between the text code y and the summary code y' other than the corresponding summary code. N is the number of summary codes y' other than the summary code corresponding to the text code y, that is, the number of second similarities obtained.

[0140] S250. Train the summary extraction model by using the first loss function and the second loss function.

[0141] Fig. 9 The flowchart schematically shows the steps of training the summary extraction model by using the first loss function and the second loss function in a certain embodiment of the present application. Fig. 9As shown, based on the above embodiment, the step S250 of training the summary extraction model by using the first loss function and the second loss function together may further include the following steps S910 and S920.

[0142] S910. Perform weighted summation on the first loss function and the second loss function to obtain a comprehensive loss function;

[0143] S920. Train the summary extraction model through a comprehensive loss function.

[0144] Specifically, the first loss function and the second loss function are weighted and summed to obtain a comprehensive loss function, which can be shown in the following formula (4):

[0145] (4):L 总 =L1+L2

[0146] That is, the weight of the first loss function L1 and the weight of the second loss function L2 may both be 1. In some embodiments, other weights may be used to perform weighted summation of the first loss function L1 and the second loss function L2 to obtain a comprehensive loss function L 总 Alternatively, other mathematical operations can be performed on the first loss function L1 and the second loss function L2 to obtain a comprehensive loss function L 总 .

[0147] This application also provides a summary generation method. Fig.10 The flowchart schematically shows the steps of the training method of the summary extraction model in some embodiments of the present application. Fig.10 As shown, the training method of the summary extraction model may mainly include the following steps S1010 to S1070.

[0148] S1010. Initialize a summary extraction model, the summary extraction model includes an encoder and a decoder, the summary extraction model is used to generate a text code by semantically encoding the processed text or text sample through the encoder, and decode the text code through the decoder to obtain a text summary;

[0149] S1020. Training the summary extraction model according to the training data set to obtain the first loss function and the training parameters of the encoder, the training data set including a plurality of text samples and summary samples corresponding to each of the text samples;

[0150] S1030. Initializing the parameters of the auxiliary encoder according to the training parameters of the encoder, and periodically updating the training parameters of the auxiliary encoder according to the training parameters of the encoder, the auxiliary encoder is used to assist in the training of the summary extraction model;

[0151] S1040. Input each summary sample into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample;

[0152] S1050. Calculate a first similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample, calculate a second similarity between the text encoding of the text sample and the summary encodings of the summary samples corresponding to a plurality of other text samples in the training data set, and determine a second loss function according to the first similarity and the second similarity;

[0153] S1060. Training the summary extraction model by using the first loss function and the second loss function;

[0154] S1070. Input the text to be processed into the trained summary extraction model to obtain a text summary corresponding to the text to be processed.

[0155] Specifically, the text to be processed may include original news, original articles, web page texts, etc. The summary extraction model may be a machine learning model, a neural network model, etc.

[0156] After completing the training of the summary extraction model, the text to be processed is input into the trained summary extraction model to obtain the text summary corresponding to the text to be processed. This can improve the training effect of the model and improve the accuracy of the generated text summary by shortening the spatial distance between the semantic vectors of the text encoding and the corresponding summary sample and increasing the spatial distance between the semantic vectors of the text encoding and other summary samples.

[0157] The details of the specific embodiments of the summary generation method provided in each embodiment of the present application have been described in detail in the embodiments of the corresponding summary extraction model training method, and will not be further described here.

[0158] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0159] The following introduces an apparatus embodiment of the present application, which can be used to execute the training method of the summary extraction model in the above-mentioned embodiment of the present application. Fig.11 The structure block diagram of the training device of the summary extraction model provided in the embodiment of the present application is schematically shown. The summary extraction model includes an encoder and a decoder. The summary extraction model is used to generate a text code by semantically encoding the text or text sample to be processed by the encoder, and decode the text code by the decoder to obtain a text summary. Fig.11 As shown, the training device 1100 of the summary extraction model includes:

[0160] A first loss function determination module 1110 is configured to train the summary extraction model according to a training data set to obtain training parameters of a first loss function and an encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples;

[0161] A training parameter updating module 1120 is configured to initialize parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically update the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model;

[0162] The auxiliary encoder encoding module 1130 is configured to input each summary sample into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample;

[0163] A second loss function determination module 1140 is configured to calculate a first similarity between a text encoding of a text sample and a summary encoding of a summary sample corresponding to the text sample, calculate a second similarity between the text encoding of the text sample and summary encodings of summary samples respectively corresponding to a plurality of other text samples in the training data set, and determine a second loss function according to the first similarity and the second similarity;

[0164] The model training module 1150 is configured to train the summary extraction model by using the first loss function and the second loss function.

[0165] In some embodiments of the present application, based on the above embodiments, the auxiliary encoder encoding module includes:

[0166] The auxiliary encoding unit is configured to input the summary sample of the current training round into the auxiliary encoder to obtain the sample code corresponding to the summary sample of the current training round.

[0167] In some embodiments of the present application, based on the above embodiments, the second loss function determination module includes:

[0168] A first similarity calculation unit is configured to calculate a first similarity between a text encoding of a text sample of a current training round and a summary encoding of a summary sample of the current training round;

[0169] The summary code storage unit is configured to store the summary code of the summary sample of the current training round into the storage queue.

[0170] In some embodiments of the present application, based on the above embodiments, the second loss function determination module further includes:

[0171] A summary code fetching unit is configured to select from the storage queue a plurality of summary codes stored in the storage queue in a training round before a current training round;

[0172] The second similarity calculation unit is configured to calculate second similarities between the text encoding of the text sample in the current training round and a plurality of summary encodings stored in the storage queue in the training round before the current training round.

[0173] In some embodiments of the present application, based on the above embodiments, the training parameter updating module includes:

[0174] A training parameter reading unit, configured to periodically read the training parameters of the encoder;

[0175] A parameter update value acquisition unit is configured to perform weighted summation of the training parameters of the encoder and the training parameters of the auxiliary encoder to obtain a parameter update value;

[0176] The parameter updating unit is configured to update the training parameters of the auxiliary encoder according to the parameter update value.

[0177] In some embodiments of the present application, based on the above embodiments, the model training module includes:

[0178] A comprehensive loss function acquisition unit is configured to perform a weighted summation of the first loss function and the second loss function to obtain a comprehensive loss function;

[0179] The model training unit is configured to train the summary extraction model through a comprehensive loss function.

[0180] In some embodiments of the present application, based on the above embodiments, the second loss function determination module further includes:

[0181] A sum value obtaining unit is configured to perform a sum operation on the first similarity and the plurality of second similarities to obtain a sum value;

[0182] a ratio obtaining unit, configured to obtain a ratio of the first similarity to the sum value;

[0183] The second loss function determining unit is configured to determine a second loss function according to the ratio, wherein the second loss function has a negative correlation with the ratio.

[0184] The specific details of the training device for the summary extraction model provided in each embodiment of the present application have been described in detail in the corresponding method embodiment and will not be repeated here.

[0185] The following introduces an apparatus embodiment of the present application, which can be used to execute the summary generation method in the above embodiment of the present application. Fig.12The structure block diagram of the summary generation device provided in the embodiment of the present application is schematically shown. Fig.12 As shown, the summary generating device 1200 includes:

[0186] The model initialization module 1210 is configured to initialize a summary extraction model, the summary extraction model includes an encoder and a decoder, the summary extraction model is used to generate a text code by semantically encoding the text or text sample to be processed through the encoder, and decode the text code through the decoder to obtain a text summary;

[0187] A first loss function determination module 1220 is configured to train the summary extraction model according to a training data set to obtain training parameters of a first loss function and an encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples;

[0188] A training parameter updating module 1230 is configured to initialize parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically update the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model;

[0189] The auxiliary encoder encoding module 1240 is configured to input each summary sample into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample;

[0190] A second loss function determination module 1250 is configured to calculate a first similarity between a text encoding of a text sample and a summary encoding of a summary sample corresponding to the text sample, calculate a second similarity between the text encoding of the text sample and summary encodings of summary samples respectively corresponding to a plurality of other text samples in the training data set, and determine a second loss function according to the first similarity and the second similarity;

[0191] A model training module 1260 is configured to train the summary extraction model by using a first loss function and a second loss function;

[0192] The text summary generation module 1270 is configured to input the text to be processed into the trained summary extraction model to obtain a text summary corresponding to the text to be processed.

[0193] The specific details of the summary generation device provided in each embodiment of the present application have been described in detail in the corresponding related method embodiments and will not be repeated here.

[0194] Fig.13 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0195] It should be noted that Fig.13The computer system 1300 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0196] like Fig.13 As shown, the computer system 1300 includes a central processing unit 1301 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1302 (ROM) or the program loaded from the storage part 1308 to the random access memory 1303 (RAM). Various programs and data required for system operation are also stored in the random access memory 1303. The central processing unit 1301, the read-only memory 1302 and the random access memory 1303 are connected to each other through a bus 1304. The input / output interface 1305 (Input / Output interface, i.e., I / O interface) is also connected to the bus 1304.

[0197] The following components are connected to the input / output interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the input / output interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0198] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1309, and / or installed from the removable medium 1311. When the computer program is executed by the central processor 1301, various functions defined in the system of the present application are executed.

[0199] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0200] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0201] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0202] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0203] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0204] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A training method for a summary extraction model, characterized in that: The summary extraction model includes an encoder and a decoder. The summary extraction model is used to generate a text code by semantically encoding the text or text sample to be processed through the encoder, and decode the text code through the decoder to obtain a text summary. The method includes: The summary extraction model is trained according to a training data set to obtain a first loss function and training parameters of the encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples; Initializing parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically updating the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model; Inputting each of the summary samples into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample; Calculating a first similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample, calculating a second similarity between the text encoding of the text sample and the summary encodings of the summary samples respectively corresponding to a plurality of other text samples in the training data set, and determining a second loss function according to the first similarity and the second similarity; The summary extraction model is trained by using the first loss function and the second loss function together.

2. The training method according to claim 1, characterized in that: Inputting each of the summary samples into the auxiliary encoder for semantic encoding to obtain a summary encoding of the summary sample includes: Inputting the summary sample of the current training round into the auxiliary encoder to obtain a sample code corresponding to the summary sample of the current training round; The calculating a first similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample comprises: Calculate a first similarity between a text encoding of a text sample of a current training round and a summary encoding of a summary sample of a current training round; The summary code of the summary sample of the current training round is stored in the storage queue.

3. The training method according to claim 2, characterized in that: The calculating the second similarity between the text encoding of the text sample and the summary encodings of the summary samples respectively corresponding to a plurality of other text samples in the training data set comprises: Selecting from the storage queue a plurality of summary codes stored in the storage queue in a training round before a current training round; The second similarities between the text encoding of the text sample of the current training round and a plurality of summary encodings stored in the storage queue in the training round before the current training round are calculated.

4. The training method according to claim 1, characterized in that: The periodically updating the training parameters of the auxiliary encoder according to the training parameters of the encoder includes: Periodically reading the training parameters of the encoder; Performing a weighted summation on the training parameters of the encoder and the training parameters of the auxiliary encoder to obtain a parameter update value; The training parameters of the auxiliary encoder are updated according to the parameter update value.

5. The training method according to claim 1, characterized in that: The training of the summary extraction model by using the first loss function and the second loss function together includes: Performing a weighted summation on the first loss function and the second loss function to obtain a comprehensive loss function; The summary extraction model is trained by the comprehensive loss function.

6. The training method according to claim 1, characterized in that: The determining a second loss function according to the first similarity and the second similarity includes: Performing a sum operation on the first similarity and a plurality of the second similarities to obtain a sum value; Obtaining a ratio of the first similarity to the sum value; The second loss function is determined according to the ratio, wherein the second loss function has a negative correlation with the ratio.

7. A method for generating a summary, characterized in that: The method comprises: Initializing a summary extraction model, the summary extraction model comprising an encoder and a decoder, the summary extraction model being used to generate a text code by semantically encoding the text or text sample to be processed through the encoder, and to decode the text code through the decoder to obtain a text summary; The summary extraction model is trained according to a training data set to obtain a first loss function and training parameters of the encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples; Initializing parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically updating the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model; Inputting each of the summary samples into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample; Calculating a first similarity between the text encoding of the text sample and the summary encoding of the summary sample corresponding to the text sample, calculating a second similarity between the text encoding of the text sample and the summary encodings of the summary samples respectively corresponding to a plurality of other text samples in the training data set, and determining a second loss function according to the first similarity and the second similarity; Training the summary extraction model by using the first loss function and the second loss function together; The text to be processed is input into the trained summary extraction model to obtain a text summary corresponding to the text to be processed.

8. A training device for a summary extraction model, characterized in that: The summary extraction model includes an encoder and a decoder. The summary extraction model is used to generate a text code by semantically encoding the text or text sample to be processed through the encoder, and decode the text code through the decoder to obtain a text summary. The device includes: A first loss function determination module is configured to train the summary extraction model according to a training data set to obtain a first loss function and training parameters of the encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples; a training parameter updating module, configured to initialize parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically update the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model; an auxiliary encoder encoding module, configured to input each of the summary samples into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample; A second loss function determination module is configured to calculate a first similarity between the text encoding of the text sample and a summary encoding of a summary sample corresponding to the text sample, calculate a second similarity between the text encoding of the text sample and summary encodings of summary samples respectively corresponding to a plurality of other text samples in the training data set, and determine a second loss function according to the first similarity and the second similarity; The model training module is configured to train the summary extraction model by using the first loss function and the second loss function.

9. The training device according to claim 8, characterized in that The auxiliary encoder encoding module comprises: an auxiliary encoding unit, configured to input the summary sample of the current training round into the auxiliary encoder to obtain a sample code corresponding to the summary sample of the current training round; The second loss function determination module includes: A first similarity calculation unit is configured to calculate a first similarity between a text encoding of a text sample of a current training round and a summary encoding of a summary sample of the current training round; The summary code storage unit is configured to store the summary code of the summary sample of the current training round into the storage queue.

10. The training device according to claim 9, characterized in that The second loss function determination module also includes: a summary code fetching unit configured to select from the storage queue a plurality of summary codes stored in the storage queue in a training round before a current training round; The second similarity calculation unit is configured to calculate second similarities between the text encoding of the text sample of the current training round and a plurality of summary encodings stored in the storage queue in the training round before the current training round.

11. The training device according to claim 8, characterized in that The training parameter updating module comprises: A training parameter reading unit, configured to periodically read the training parameters of the encoder; A parameter update value acquisition unit, configured to perform weighted summation of the training parameters of the encoder and the training parameters of the auxiliary encoder to obtain a parameter update value; A parameter updating unit is configured to update the training parameters of the auxiliary encoder according to the parameter update value.

12. The training device according to claim 8, characterized in that The model training module includes: a comprehensive loss function acquisition unit, configured to perform a weighted summation of the first loss function and the second loss function to obtain a comprehensive loss function; The model training unit is configured to train the summary extraction model through the comprehensive loss function.

13. The training device according to claim 8, characterized in that The second loss function determination module also includes: a sum value obtaining unit, configured to perform a sum operation on the first similarity and a plurality of the second similarities to obtain a sum value; a ratio obtaining unit, configured to obtain a ratio of the first similarity to the sum value; A second loss function determining unit is configured to determine the second loss function according to the ratio, wherein the second loss function has a negative correlation with the ratio.

14. A summary generation device, characterized in that: The device comprises: A model initialization module is configured to initialize a summary extraction model, wherein the summary extraction model includes an encoder and a decoder, wherein the summary extraction model is used to generate a text code by semantically encoding the text or text sample to be processed through the encoder, and to decode the text code through the decoder to obtain a text summary; A first loss function determination module is configured to train the summary extraction model according to a training data set to obtain a first loss function and training parameters of the encoder, wherein the training data set includes a plurality of text samples and summary samples corresponding to each of the text samples; a training parameter updating module, configured to initialize parameters of an auxiliary encoder according to the training parameters of the encoder, and periodically update the training parameters of the auxiliary encoder according to the training parameters of the encoder, wherein the auxiliary encoder is used to assist in the training of the summary extraction model; an auxiliary encoder encoding module, configured to input each of the summary samples into the auxiliary encoder for semantic encoding to obtain a summary code of the summary sample; A second loss function determination module is configured to calculate a first similarity between the text encoding of the text sample and a summary encoding of a summary sample corresponding to the text sample, calculate a second similarity between the text encoding of the text sample and summary encodings of summary samples respectively corresponding to a plurality of other text samples in the training data set, and determine a second loss function according to the first similarity and the second similarity; A model training module, configured to train the summary extraction model by using the first loss function and the second loss function; The text summary generation module is configured to input the text to be processed into the trained summary extraction model to obtain the text summary corresponding to the text to be processed.

15. A computer readable medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

16. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform any one of the methods described in 1 to 7 by executing the executable instructions.

Images