Abstract generation method, abstract generation model training method and related device

Abstract

The invention discloses an abstract generation method, an abstract generation model training method and a related device. The method comprises the steps of obtaining a first to-be-processed text; the first to-be-processed text and a topic feature vector corresponding to a first topic are fused to obtain a first fused feature vector, and the topic feature vector corresponding to the first topic is used for indicating an abstract generation model to generate an abstract corresponding to the first topic in the first to-be-processed text; and processing the first fusion feature vector by using the abstract generation model to obtain an abstract text of the first topic in the first text to be processed. Through the method, the abstract generation accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a summary generation method, a training method for a summary generation model, and related devices. Background Art

[0002] In the field of natural language processing, summarization technology refers to the use of computers to automatically extract comprehensive and accurate short text from the text to be summarized, accurately reflecting the central topic of the text to be summarized. However, the text to be summarized is usually long, and long texts often contain rich semantic information, and this semantic information is often multi-layered and complex. In this context, how to effectively improve the accuracy of summary generation has become a pressing issue to be addressed in this application. Summary of the Invention

[0003] The main technical problem solved by this application is to provide a summary generation method, a training method for a summary generation model and related devices, which can improve the accuracy of summary generation.

[0004] To solve the above technical problems, a technical solution adopted in this application is to provide a summary generation method, which includes: obtaining a first text to be processed; fusing the first text to be processed and the topic feature vector corresponding to the first topic to obtain a first fused feature vector, wherein the topic feature vector corresponding to the first topic is used to instruct the summary generation model to generate a summary corresponding to the first topic in the first text to be processed; using the summary generation model to process the first fused feature vector to obtain a summary text of the first text to be processed and the first topic.

[0005] To solve the above technical problems, another technical solution adopted in the present application is: to provide a training method for a summary generation model, the training method comprising: using topic feature vectors corresponding to several first training texts and at least one first training topic to fine-tune a large language model to obtain a fine-tuned large language model, the first training texts being pre-annotated with actual summaries corresponding to the first training topics, wherein the topic feature vectors corresponding to the first training texts and the first training topics are fused and respectively input into the large language model; using topic feature vectors corresponding to several second training texts and at least one second training topic to optimize the fine-tuned large language model to obtain the summary generation model, wherein the second training texts are annotated with actual summaries corresponding to the second training topics, the actual summaries annotated by at least one second training text are generated by the fine-tuned large language model and corrected, and the topic feature vectors corresponding to the second training texts and the second training topics are fused and respectively input into the large language model.

[0006] In order to solve the above technical problems, another technical solution adopted in this application is: to provide a summary generation device, which includes: a text acquisition module for acquiring a first text to be processed; a feature vector acquisition module for fusing the first text to be processed and the topic feature vector corresponding to the first topic to obtain a first fused feature vector, wherein the topic feature vector corresponding to the first topic is used to instruct the summary generation model to generate a summary corresponding to the first topic in the first text to be processed; the summary acquisition module is used to use the summary generation model to process the first fused feature vector to obtain a summary text of the first text to be processed and the first topic.

[0007] To solve the above technical problems, another technical solution adopted in the present application is: to provide a training device for a summary generation model, the training device including: a fine-tuning module for fine-tuning a large language model using topic feature vectors corresponding to several first training texts and at least one first training topic to obtain a fine-tuned large language model, wherein the first training texts are pre-annotated with actual summaries corresponding to the first training topics, wherein the topic feature vectors corresponding to the first training texts and the first training topics are fused and respectively input into the large language model; an optimization module for optimizing the fine-tuned large language model using topic feature vectors corresponding to several second training texts and at least one second training topic to obtain the summary generation model, wherein the second training texts are annotated with actual summaries corresponding to the second training topics, and the actual summaries annotated by at least one second training text are generated by the fine-tuned large language model and corrected, and the topic feature vectors corresponding to the second training texts and the second training topics are fused and respectively input into the large language model.

[0008] To solve the above technical problems, another technical solution adopted by the present application is: to provide an electronic device, which includes a memory and a processor, the memory storing program instructions, and the processor being configured to execute the program instructions to implement the above summary generation method.

[0009] To solve the above technical problems, another technical solution adopted by the present application is: providing a computer-readable storage medium, which is used to store program instructions, and the program instructions can be executed to implement the above summary generation method.

[0010] The above scheme obtains the first text to be processed, fuses the topic feature vectors corresponding to the first text to be processed and the first topic, and obtains the first fused feature vector, wherein the topic feature vector corresponding to the first topic is used to instruct the summary generation model to generate a summary corresponding to the first topic in the first text to be processed. The first fused feature vector is processed by the summary generation model to obtain the summary text of the first topic in the first text to be processed. The present application inputs the fused feature vector of the topic feature vectors corresponding to the first text to be processed and the first topic into the summary generation model for processing. Since the topic feature vector corresponding to the first topic has semantic information indicating the extraction of the summary of the first topic, it can provide directional guidance for the summary generation model, so that the summary generation model can effectively capture the semantic information related to the first topic in the first processed text, thereby accurately extracting the summary text of the first topic, that is, improving the accuracy of summary generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flowchart of an embodiment of a summary generation method provided by this application;

[0012] Figure 2 This is a schematic diagram of constructing a topic feature vector in the summary generation method provided by this application;

[0013] Figure 3 A schematic diagram of the architecture of the summary generation method provided in this application;

[0014] Figure 4 This is a flowchart of a specific embodiment of the training steps of the summary generation model provided in this application;

[0015] Figure 5 This is a flowchart of an embodiment of a training method for a summary generation model provided in this application;

[0016] Figure 6 This is a schematic diagram of the framework of an embodiment of the abstract generation device of the present application;

[0017] Figure 7 This is a schematic diagram of a framework of an embodiment of a training device for a summary generation model of the present application;

[0018] Figure 8 This is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0019] Figure 9 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and effects of this application clearer and more specific, this application is further described in detail below with reference to the accompanying drawings and examples.

[0021] It should be noted that the term "several" in this article means at least one, and the terms "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. The term "and / or" is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0022] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the abstract generation method provided by this application. It should be noted that if there are substantially the same results, this embodiment does not use Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0023] Step S11: Obtain the first text to be processed.

[0024] The first text to be processed mentioned in this article is a long text, which can be directly uploaded by the user or obtained from a related database according to the user's specification. There is no restriction here.

[0025] Step S12: Fusing the first text to be processed and the topic feature vector corresponding to the first topic to obtain a first fused feature vector.

[0026] The topic feature vector corresponding to the first topic is used to instruct the summary generation model to generate a summary text corresponding to the first topic in the first text to be processed.

[0027] In one embodiment, the first topic refers to a topic that the user is interested in. The first topic may refer to at least one topic that the user is interested in and has been pre-established. Of course, the first topic may also refer to at least one new topic that the user is interested in but has not been pre-established. This setting may be based on actual needs and is not limited here.

[0028] In another embodiment, the topic feature vector corresponding to the first topic is obtained by encoding a prompt text of the first topic, where the prompt text is used to instruct the summary generation model to generate a summary text corresponding to the first topic.

[0029] In a specific embodiment, the prompt text includes topics of interest to the user and other prompt information, wherein the other prompt information includes at least one of the user's purpose, reference historical data, and summary generation requirements, and the topics of interest to the user serve as topics corresponding to the topic feature vectors.

[0030] For example, a prompt text may be specifically: "Prompt (prompt text): Please extract key information related to the user's concerns from the following text and generate a concise summary. The topic of user concern is: [topic of user concern], and the user's purpose is: [user's purpose]. Please refer to the typical information in the following historical data for summary: [reference historical data]. When generating the summary, please ensure: 1. Highlight the content related to the user's goals. 2. Extract information that is helpful for user decision-making or goal achievement. 3. Omit irrelevant or redundant information. 4. The summary is concise and clear.", wherein the above prompt text includes clear instructions and contextual limitations. Specifically, the prompt text includes the topic of user concern, the user's purpose, reference historical data, and summary generation requirements (such as the four requirements in the prompt text that must be followed when generating the summary).

[0031] It should be noted that one prompt text may correspond to a topic that a user is interested in. Of course, one prompt text may also correspond to multiple topics that a user is interested in. This can be adjusted according to actual design requirements and is not limited here.

[0032] In another embodiment, the topic feature vector corresponding to the first topic can be obtained by an encoder in the summary generation model before fusing the first to-be-processed text with the topic feature vector corresponding to the first topic. Specifically, prompt text corresponding to at least one topic is obtained. The prompt text corresponding to each topic is encoded using the encoder in the summary generation model to obtain the topic feature vector corresponding to each topic.

[0033] For example, a prompt text corresponding to at least one topic is obtained and input into an encoder in the summary generation model for encoding. Specifically, the encoder encodes the contextual semantics of each prompt text to generate a topic feature vector corresponding to each topic. The topic feature vector can be a multidimensional feature vector that captures the semantic information and potential associations of the corresponding prompt text.

[0034] Combine Figure 2 To illustrate, Figure 2This is a schematic diagram of the construction of topic feature vectors in the summary generation method provided by this application. The summary generation model includes an encoder (LLM-Encoder) and a decoder (LLM-Decoder). Before the prompt text is input into the encoder in the summary generation model, the prompt text can be preprocessed to obtain at least one vector sequence (E1, ..., En) corresponding to the prompt text, where: <s>It can be used to mark the starting position of the input prompt text, instructing the summary generation model to "start processing from here", that is, to start encoding from E1, input all the above vector sequences into the encoder, and further obtain the topic feature vector corresponding to the prompt text.

[0035] In another embodiment, a pre-trained attention network can be used to fuse the first to-be-processed text with the topic feature vector. Specifically, the attention network is used to perform attention processing on the topic feature vector to obtain an attention feature vector. The attention feature vector is then fused with the first to-be-processed text to obtain a first fused feature vector.

[0036] In a specific embodiment, the attention network can be used to perform at least one of feature extraction, nonlinear transformation and weighted processing on the topic feature vector to obtain the attention feature vector.

[0037] In another specific implementation, a weighted strategy may be used to fuse the attention feature vector with the first text to be processed to obtain a first fused feature vector.

[0038] The present application introduces an attention network to perform attention processing on the topic feature vector and performs weighted processing in combination with the first text to be processed. Through weight optimization, the summary generation model can adapt to specific summary generation requirements.

[0039] Step S13: Processing the first fused feature vector using the summary generation model to obtain a summary text corresponding to the first topic in the first text to be processed.

[0040] In one embodiment, considering the need for personalized recommendation summaries, several user preference texts and multiple global texts can also be obtained, and each user preference text and global text can be used as a second text to be processed. For each second text to be processed, the topic feature vectors corresponding to the second text to be processed and at least one second topic are fused to obtain a second fused feature vector corresponding to each second topic. The encoder of the summary generation model is used to encode each second fused feature vector to obtain a summary feature vector of each second text to be processed corresponding to each second topic. Based on the similarity between the summary feature vectors of each global text and each user preference text corresponding to the same second topic, at least one recommended text is selected from the multiple global texts to be recommended to the user.

[0041] In one specific embodiment, the specific description of fusing the topic feature vector corresponding to the second to-be-processed text and at least one second topic can be referred to the relevant description in step S12 and will not be repeated here. By combining the weighted processing of the user preference text and the attention network, it is possible to dynamically adapt to changes in user interests. This can effectively address the limitations of static feature extraction based on text content methods, and also effectively solve the problem of decreased accuracy of recommendation results in collaborative filtering under dynamic changes in user interests.

[0042] In another specific embodiment, the second topic may be selected from pre-constructed topics, that is, at least one second topic is selected from the pre-constructed topics, wherein the pre-constructed topics refer to topics constructed before the summary generation model is trained. For example, the pre-constructed topics include topics A, B, and C. The selected second topics may encompass all pre-constructed topics, that is, each second topic is topic A, B, and C. For another example, the pre-constructed topics include topics A, B, and C. The selected second topic may be a topic related to the topic of interest to the user among all pre-constructed topics, wherein the topics related to the topic of interest to the user include topics A and B. That is, each second topic is topic A and B. There is no limitation here.

[0043] In another specific embodiment, user preferred text is text on which a user performs at least one of the following actions: like or favorite. For example, text that a user likes and favorites is considered user preferred text. For another example, text that a user likes is considered user preferred text. For another example, text that a user favorites is considered user preferred text. This is not a limitation.

[0044] It should be noted that the user preference text can be the complete text that the user likes or collects, or it can be the summary text obtained by processing the complete text that the user likes or collects through the summary generation model as the corresponding user preference text. Similarly, the global text can be the complete text in the total database, or it can be the summary text obtained by processing the complete text in the total database through the summary generation model as the corresponding global text. It can be set according to actual needs and is not limited here. Among them, the user preference text can be stored in the user summary vector library, and the global text can be stored in the global summary vector library.

[0045] In another specific embodiment, before selecting at least one recommended text to be recommended to the user from multiple global texts based on the similarity between the summary feature vectors of each global text and each user preference text corresponding to the same second topic, pooling processing can also be performed on the summary feature vectors of each second to-be-processed text corresponding to each second topic.

[0046] For example, average pooling processing may be performed separately on the summary feature vectors of each second topic corresponding to each second text to be processed to extract key feature information from the summary feature vectors.

[0047] For another example, the summary feature vectors of each second topic corresponding to each second text to be processed may be subjected to average pooling processing to extract key feature information from the summary feature vectors.

[0048] In another specific embodiment, before selecting at least one recommended text from the plurality of global texts to be recommended to the user based on the similarity between the summary feature vectors of each global text and each user preferred text corresponding to the same second topic, principal component analysis may be used to perform dimensionality reduction processing on the summary feature vectors of each second to-be-processed text corresponding to each second topic. For example, PCA may be used to perform dimensionality reduction processing on the summary feature vectors of each second to-be-processed text corresponding to each second topic to reduce redundant information in the summary feature vectors, thereby forming a more compact summary feature vector.

[0049] In another specific embodiment, before selecting at least one recommended text from multiple global texts to be recommended to the user based on the similarity between the summary feature vectors of each global text and each user-preferred text corresponding to the same second topic, a pooling process may be performed on the summary feature vectors of each second-to-be-processed text corresponding to each second topic. Dimensionality reduction processing is performed on the summary feature vectors of each second-to-be-processed text corresponding to each second topic using principal component analysis. For detailed descriptions, please refer to the description of pooling and dimensionality reduction processing in step S13 and will not be repeated here.

[0050] In another specific embodiment, the summary feature vector corresponding to each global text may be stored in a global summary vector library, and the summary feature vector corresponding to each user preference text may be stored in a user summary vector library.

[0051] In another specific embodiment, based on the similarity between the summary feature vectors of each global text and each user preferred text corresponding to the same second topic, at least one recommended text is selected from multiple global texts for recommendation to the user. Specifically, for each second topic, the similarity between each global text and the summary feature vector of each user preferred text corresponding to the second topic is obtained. The similarity between each global text and each user preferred text under the second topic is sorted, and a preset number of different full-text texts from the sorted list are selected as recommended texts for the second topic. Alternatively, for each global text, a central tendency statistic of the similarity between the global text and each user preferred text under the second topic is obtained, and the global text with the highest central tendency statistic among the preset number of global texts is selected as the recommended text for the second topic. The central tendency statistic may be, for example, the average or median of the similarity between the global text and each user preferred text under the second topic. The similarity between each global text and each user preferred text under the second topic may be obtained using a cosine similarity algorithm.

[0052] The similarity between each summary feature vector in the global summary vector library and each summary feature vector in the user summary vector library can be calculated and sorted by similarity. The global texts corresponding to the first preset number of summary feature vectors in the similarity sorting are used as recommended texts.

[0053] For example, for each second topic, the global text includes global text 1, global text 2, and global text 3, and the user preferred text includes user preferred text 1 and user preferred text 2. The similarity between the summary feature vector corresponding to global text 1 and the summary feature vector corresponding to user preferred text 1 is 90, and the similarity between the summary feature vector corresponding to global text 1 and the summary feature vector corresponding to user preferred text 2 is 80. The similarity between the summary feature vector corresponding to global text 2 and the summary feature vector corresponding to user preferred text 1 is 90, and the similarity between the summary feature vector corresponding to global text 2 and the summary feature vector corresponding to user preferred text 2 is 70. The similarity between the summary feature vector corresponding to global text 3 and the summary feature vector corresponding to user preferred text 1 is 80, and the similarity between the summary feature vector corresponding to global text 3 and the summary feature vector corresponding to user preferred text 2 is 60. The average similarity between global text 1 and each user preferred text is 85, the average similarity between global text 2 and each user preferred text is 80, and the average similarity between global text 2 and each user preferred text is 70. The text with the highest average value is selected as the recommended text corresponding to the second topic, that is, the global text 1 is selected as the recommended text corresponding to the second topic.

[0054] For another example, the similarity between the summary feature vector corresponding to global text 1 and the summary feature vector corresponding to user preferred text 1 is 90, and the similarity between the summary feature vector corresponding to global text 1 and the summary feature vector corresponding to user preferred text 2 is 80. The similarity between the summary feature vector corresponding to global text 2 and the summary feature vector corresponding to user preferred text 1 is 75, and the similarity between the summary feature vector corresponding to global text 2 and the summary feature vector corresponding to user preferred text 2 is 70. The similarity between the summary feature vector corresponding to global text 3 and the summary feature vector corresponding to user preferred text 1 is 70, and the similarity between the summary feature vector corresponding to global text 3 and the summary feature vector corresponding to user preferred text 2 is 60. Among them, the similarity ranking is 90, 80, 75, 70, 70, 60, and the two texts with the highest scores in the similarity ranking are selected as the corresponding recommended texts. In this case, the two texts with the highest scores in the similarity ranking are both global text 1, so global text 2 corresponding to the next ranking (similarity score of 75) is selected as the second recommended text. That is, global text 1 and global text 2 are selected as the corresponding recommended texts.

[0055] It should be noted that the preset number of global texts to be selected can be adjusted according to actual business needs. For example, the user can set the preset number according to actual needs, that is, determine the number of recommended texts to be recommended according to actual needs.

[0056] This application generates a user summary vector library from the text that users like or collect, and combines it with the global summary vector library to perform many-to-many similarity calculations to achieve personalized recommendations. This recommendation method based on summary feature vectors makes up for the recommendation accuracy issues caused by the sparsity of user behavior and the lack of semantic understanding of deep learning methods in collaborative filtering methods. Compared with collaborative filtering and text content-based methods, this application can not only recommend highly relevant content, but also cover users' potential interests, avoiding recommendation homogeneity.

[0057] Combine Figure 3 To illustrate, Figure 3 This is a schematic diagram of the architecture of the summary generation method provided in this application. Taking the processing of a first text to be processed as an example, an attention network is used to perform attention processing on the topic feature vector. The attention feature vector obtained after the attention processing is weightedly fused with the first text to be processed to obtain a first fused feature vector. The first fused feature vector is input into a summary generation model. The summary generation model is used to process the first fused feature vector to obtain a summary text corresponding to the first topic in the first text to be processed. Alternatively, the encoder in the summary generation model can be used to encode several user-preferred texts and multiple global texts, that is, to encode the second text to be processed to obtain summary feature vectors corresponding to each second text to be processed. Each summary feature vector is pooled and subjected to principal component analysis. The summary feature vectors after the above processing are then constructed into a user summary vector library corresponding to several user-preferred texts and a global summary vector library corresponding to multiple global texts. Similarity calculations are performed between each summary feature vector in the global summary vector library and each summary feature vector in the user summary vector library, and scores are assigned based on the calculated similarities. Based on the similarity scores, the global text with the first preset number of digits is obtained as the corresponding recommended text.

[0058] See also Figure 4 , Figure 4 This is a flowchart of a specific embodiment of the training step of the summary generation model provided in this application. The summary generation model is the summary generation model mentioned in the summary generation method in steps S11 to S13. It should be noted that if there are substantially the same results, this embodiment does not use Figure 4 The process sequence shown is limited. Figure 4 As shown, this specific embodiment includes:

[0059] Step S41: fine-tuning the large language model using topic feature vectors corresponding to a plurality of first training texts and at least one first training topic to obtain a fine-tuned large language model, wherein the first training texts are pre-annotated with actual summaries corresponding to the first training topics.

[0060] In one embodiment, during the fine-tuning process, a low-rank adaptation method is used to adjust parameters.

[0061] In a specific embodiment, during the fine-tuning process, parameter adjustment is performed by inserting a low-rank matrix.

[0062] In another embodiment, before the large language model is fine-tuned, a group of topics can be pre-constructed. The first training topic can include all pre-constructed topics. Specifically, the large language model can be fine-tuned using topic feature vectors corresponding to several first training texts and at least one pre-constructed topic. For example, a pre-constructed group of topics includes 10 topics, and the large language model can be fine-tuned using topic feature vectors corresponding to 3 topics, or the large language model can be fine-tuned using topic feature vectors corresponding to 10 topics. The specific number of first training topics used can be set according to actual needs and is not limited here. The topic feature vectors corresponding to the pre-constructed topics can be stored uniformly, that is, to form a set of initial semantic representations.

[0063] In another embodiment, the specific description of the method for obtaining the topic feature vector can refer to the relevant description of the method for obtaining the topic feature vector in step S12, which will not be repeated here.

[0064] In another embodiment, topic feature vectors corresponding to the plurality of first training texts and at least one first training topic can be used to generate a predicted summary using a large language model. Based on the differences between the predicted summary and the actual summary, the large language model is fine-tuned. The specific implementation of generating a predicted summary using the topic feature vectors corresponding to the plurality of first training texts and at least one first training topic can be found in the description of steps S11 to S13 and is not further elaborated here.

[0065] Step S42: Optimize the fine-tuned large language model using topic feature vectors corresponding to a plurality of second training texts and at least one second training topic to obtain a summary generation model. The second training texts are annotated with actual summaries corresponding to each second training topic. The actual summaries annotated with at least one second training text are generated by the fine-tuned large language model and corrected.

[0066] In one embodiment, the second training topic may include all pre-constructed topics. Specifically, the topic feature vectors corresponding to several second training texts and at least one pre-constructed topic may be used to fine-tune the large language model. For example, a pre-constructed group of topics includes 10 topics, and the topic feature vectors corresponding to 3 topics may be used to fine-tune the large language model, or the topic feature vectors corresponding to 10 topics may be used to fine-tune the large language model. The specific number of second training topics used can be set according to actual needs and is not limited here. Among them, the topic feature vectors corresponding to the pre-constructed topics can be stored uniformly, that is, to form a set of initial semantic representations.

[0067] In another embodiment, before optimizing the fine-tuned large language model using topic feature vectors corresponding to several second training texts and at least one second training topic to obtain the summary generation model, at least one second training text can also be used as the text to be annotated. The topic feature vectors corresponding to the text to be annotated and each second training topic are fused respectively to obtain a fourth fused feature vector corresponding to each second training topic. The fourth fused feature vector corresponding to each second training topic is processed using the summary generation model to obtain a candidate summary corresponding to each second training topic. The correction result of the candidate summary corresponding to each second training topic by the annotator is obtained as the second actual summary corresponding to each second training topic annotated by the text to be annotated. The candidate summary output by the summary generation model is provided to the annotator for reference. The annotator performs a secondary review and annotation based on the candidate summary to obtain the corresponding correction result, and uses the correction result as the second actual summary corresponding to each second training topic annotated by the text to be annotated.

[0068] This cyclical optimization mechanism significantly improves the accuracy of summaries and recommendations generated by the summary generation model, using a fine-tuned summary generation model to assist in annotating the second training text and improving the annotation quality through manual review. Compared to graph recommendation methods, which suffer from reduced recommendation effectiveness due to graph sparsity, this application continuously optimizes the summary generation model through high-quality annotated text, effectively solving the cold start problem for new users or new text.

[0069] In another embodiment, the large language model is fine-tuned using topic feature vectors corresponding to several first training texts and at least one first training topic, or the fine-tuned large language model is optimized using topic feature vectors corresponding to several second training texts and at least one second training topic. Specifically, the first training text can be used as the target training text and the first training topic as the target topic, or the second training text can be used as the target training text and the second training topic as the target topic. The topic feature vectors corresponding to the target training text and each target topic are fused respectively to obtain a target fused feature vector corresponding to each target topic. The target fused feature vector corresponding to each target topic is processed using the large language model to obtain a predicted summary corresponding to each target topic. The difference between the actual summary and the predicted summary corresponding to each target topic is used to adjust the parameters in the large language model.

[0070] In a specific embodiment, the parameters may be adjusted by using a low-rank adaptation method, for example, by inserting a low-rank matrix.

[0071] It should be noted that this application constructs prompt text corresponding to multiple user-focused topics before training the summary generation model. By leveraging the semantic understanding capabilities of the summary generation model, this application can effectively capture text, especially multi-level semantic information in long texts, and provide directional guidance for the subsequent generation of summary texts. That is, compared to methods based on text content analysis (such as TF-IDF), this application can more accurately extract text, especially themes and deep semantics in long texts, by leveraging the semantic understanding capabilities of the summary generation model. This effectively addresses the problem of traditional methods' insufficient ability to capture the semantics of complex texts.

[0072] See also Figure 5 , Figure 5 This is a flow chart of an embodiment of the training method of the summary generation model provided in this application. The summary generation model can implement the summary generation method in steps S11 to S13. It should be noted that if there are substantially the same results, this embodiment does not use Figure 5 The process sequence shown is limited. Figure 5 As shown, this specific embodiment includes:

[0073] Step S51: fine-tune the large language model using topic feature vectors corresponding to a plurality of first training texts and at least one first training topic to obtain a fine-tuned large language model, wherein the first training texts are pre-annotated with actual summaries corresponding to the first training topics.

[0074] The topic feature vectors corresponding to each first training text and each first training topic are fused and then input into the large language model respectively.

[0075] For the specific description of step S51, please refer to the relevant description of step S41, and no limitation is given here.

[0076] Step S52: Optimize the fine-tuned large language model using the topic feature vectors corresponding to the plurality of second training texts and at least one second training topic to obtain a summary generation model.

[0077] Among them, the second training text annotates the actual summary corresponding to each second training topic, and the actual summary annotated by at least one second training text is generated by the fine-tuned large language model and obtained through correction. The topic feature vectors corresponding to each second training text and each second training topic are fused and input into the large language model respectively.

[0078] For the specific description of step S52, please refer to the relevant description of step S51, and no limitation is given here.

[0079] See also Figure 6 , Figure 6 It is a schematic diagram of the framework of an embodiment of the summary generation device of the present application. The summary generation device 600 includes a text acquisition module 610, a feature vector acquisition module 620, and a summary acquisition module 630. The text acquisition module 610 is used to acquire a first text to be processed. The feature vector acquisition module 620 is used to fuse the first text to be processed and the topic feature vector corresponding to the first topic to obtain a first fused feature vector, wherein the topic feature vector corresponding to the first topic is used to instruct the summary generation model to generate a summary text corresponding to the first topic in the first text to be processed. The summary acquisition module 630 is used to process the first fused feature vector using the summary generation model to obtain a summary text corresponding to the first topic in the first text to be processed.

[0080] In some embodiments, the topic feature vector corresponding to the first topic is obtained by encoding the prompt text of the first topic, and the prompt text is used to instruct the summary generation model to generate a summary text corresponding to the first topic.

[0081] In some embodiments, the prompt text includes topics of interest to the user and other prompt information, wherein the other prompt information includes at least one of the user's purpose, reference historical data, and summary generation requirements, and the topics of interest to the user are the topics corresponding to the topic feature vectors.

[0082] In some embodiments, before fusing the first to-be-processed text with the topic feature vector corresponding to the first topic to obtain the first fused feature vector, the feature vector acquisition module 620 further includes: obtaining prompt text corresponding to at least one topic. The prompt text corresponding to each topic is encoded using an encoder in the summary generation model to obtain a topic feature vector corresponding to each topic.

[0083] In some embodiments, the feature vector acquisition module 620 specifically performs fusing the first text to be processed and the topic feature vector corresponding to the first topic to obtain the first fused feature vector, including: performing attention processing on the topic feature vector using an attention network to obtain an attention feature vector; and fusing the attention feature vector with the first text to be processed to obtain the first fused feature vector.

[0084] In some embodiments, the summary generation device 600 also includes a summary feature vector acquisition module and a recommended text acquisition module. The text acquisition module 610 can also be used to obtain a number of user preference texts and multiple global texts, and use each user preference text and global text as a second text to be processed. The feature vector acquisition module 620 can also be used to fuse the topic feature vectors corresponding to the second text to be processed and at least one second topic for each second text to be processed, to obtain a second fused feature vector corresponding to each second topic. The summary feature vector acquisition module is used to encode each second fused feature vector using the encoder of the summary generation model to obtain a summary feature vector for each second topic corresponding to each second text to be processed. The recommended text acquisition module is used to select at least one recommended text to be recommended to the user from multiple global texts based on the similarity between the summary feature vectors of each global text and each user preference text corresponding to the same second topic.

[0085] In some embodiments, the user preferred text is a text on which the user performs at least one of the following operations: like, favorite.

[0086] In some embodiments, the summary generation device 600 further includes a recommended text acquisition module. Before the recommended text acquisition module specifically selects at least one recommended text from multiple global texts to recommend to the user based on the similarity between the summary feature vectors of each global text and each user preferred text corresponding to the same second topic, the device further includes at least one of the following steps: pooling the summary feature vectors of each second to-be-processed text corresponding to each second topic. And performing dimensionality reduction processing on the summary feature vectors of each second to-be-processed text corresponding to each second topic using principal component analysis.

[0087] In some embodiments, the summary generation device 600 further includes a recommended text acquisition module. The module selects at least one recommended text from a plurality of global texts to recommend to the user based on the similarity between the summary feature vectors of each global text and each user preferred text corresponding to the same second topic. The module includes: for each second topic, obtaining the similarity between each global text and the summary feature vector of each user preferred text corresponding to the second topic; sorting the similarities between each global text and each user preferred text under the second topic, and selecting a preset number of different global texts from the sorted list as recommended texts for the second topic; or, for each global text, obtaining a central tendency statistical value of the similarity between the global text and each user preferred text under the second topic, and selecting the global text with the central tendency statistical value that ranks among the preset number as the recommended text for the second topic.

[0088] In some embodiments, the summary generation model training step includes: fine-tuning a large language model using topic feature vectors corresponding to a plurality of first training texts and at least one first training topic to obtain a fine-tuned large language model, wherein the first training texts are pre-annotated with actual summaries corresponding to the first training topics. Optimizing the fine-tuned large language model using topic feature vectors corresponding to a plurality of second training texts and at least one second training topic to obtain a summary generation model, wherein the second training texts are annotated with actual summaries corresponding to the second training topics, wherein the at least one actual summary annotated by the second training text is generated and corrected by the fine-tuned large language model.

[0089] In some embodiments, during the fine-tuning process, a low-rank adaptation method is used to adjust parameters.

[0090] In some embodiments, before optimizing the fine-tuned large language model using topic feature vectors corresponding to several second training texts and at least one second training topic to obtain a summary generation model, the method further includes: using at least one second training text as the text to be annotated. Fusing the topic feature vectors corresponding to the text to be annotated and each second training topic respectively to obtain a fourth fused feature vector corresponding to each second training topic. Processing the fourth fused feature vector corresponding to each second training topic using the summary generation model to obtain a candidate summary corresponding to each second training topic. Obtaining the correction results of the candidate summary corresponding to each second training topic by the annotator as the second actual summary corresponding to each second training topic annotated by the text to be annotated.

[0091] In some embodiments, the large language model is fine-tuned using topic feature vectors corresponding to several first training texts and at least one first training topic, or the fine-tuned large language model is optimized using topic feature vectors corresponding to several second training texts and at least one second training topic, including: using the first training text as the target training text and the first training topic as the target topic, or using the second training text as the target training text and the second training topic as the target topic. The target training text and the topic feature vectors corresponding to each target topic are fused respectively to obtain a target fused feature vector corresponding to each target topic. The target fused feature vector corresponding to each target topic is processed using the large language model to obtain a predicted summary corresponding to each target topic. The difference between the actual summary and the predicted summary corresponding to each target topic is used to adjust the parameters in the large language model.

[0092] See also Figure 7 , Figure 7 It is a schematic diagram of the framework of an embodiment of the training device for the summary generation model of the present application. The training device 700 for the summary generation model includes a fine-tuning module 710 and an optimization module 720. The fine-tuning module 710 is used to use the topic feature vectors corresponding to several first training texts and at least one first training topic to fine-tune the large language model to obtain a fine-tuned large language model, wherein the first training texts are pre-annotated with the actual summaries corresponding to the first training topics, wherein the topic feature vectors corresponding to the first training texts and the first training topics are fused and respectively input into the large language model. The optimization module 720 is used to use the topic feature vectors corresponding to several second training texts and at least one second training topic to optimize the fine-tuned large language model to obtain a summary generation model, wherein the second training texts are annotated with the actual summaries corresponding to the second training topics, and the actual summaries annotated by the at least one second training text are generated by the fine-tuned large language model and corrected, and the topic feature vectors corresponding to the second training texts and the second training topics are fused and respectively input into the large language model.

[0093] See also Figure 8 , Figure 8 This is a schematic diagram of the framework of an embodiment of an electronic device of the present application. Electronic device 80 includes a memory 81 and a processor 82 coupled to each other. Processor 82 is configured to execute program instructions stored in memory 81 to implement the steps of any of the aforementioned digest generation method embodiments. In a specific implementation scenario, electronic device 80 may include, but is not limited to, a microcomputer and a server. Furthermore, electronic device 80 may also include mobile devices such as laptops and tablet computers, without limitation herein.

[0094] Specifically, the processor 82 is used to control itself and the memory 81 to implement the steps in any of the above-mentioned summary generation method embodiments. The processor 82 can also be called a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip with signal processing capabilities. The processor 82 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. In addition, the processor 82 can be implemented by an integrated circuit chip.

[0095] See also Figure 9 , Figure 9 The computer-readable storage medium 90 stores program instructions 91 that can be executed by a processor, and the program instructions 91 are used to implement the steps of any of the above-mentioned summary generation method embodiments.

[0096] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiment. Its specific implementation can refer to the description of the above method embodiment. For the sake of brevity, it will not be repeated here.

[0097] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0098] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0099] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0100] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.< / s>

Claims

1. A method for generating a summary, characterized in that: The method comprises: Get the first text to be processed; fusing the first text to be processed and the topic feature vector corresponding to the first topic to obtain a first fused feature vector, wherein the topic feature vector corresponding to the first topic is used to instruct a summary generation model to generate a summary text corresponding to the first topic in the first text to be processed; The first fused feature vector is processed using the summary generation model to obtain a summary text corresponding to the first topic in the first text to be processed.

2. The method according to claim 1, characterized in that The topic feature vector corresponding to the first topic is obtained by encoding a prompt text of the first topic, and the prompt text is used to instruct the summary generation model to generate a summary text corresponding to the first topic.

3. The method according to claim 2, characterized in that The prompt text includes topics of interest to the user and other prompt information, wherein the other prompt information includes at least one of the user's purpose, reference history data, and summary generation requirements, and the topic of interest to the user serves as the topic corresponding to the topic feature vector; And / or, before fusing the first to-be-processed text and the topic feature vector corresponding to the first topic to obtain a first fused feature vector, the method further includes: Get the prompt text corresponding to at least one topic; The prompt text corresponding to each of the topics is encoded using the encoder in the summary generation model to obtain a topic feature vector corresponding to each of the topics.

4. The method according to claim 1, wherein The fusing the first to-be-processed text and the topic feature vector corresponding to the first topic to obtain a first fused feature vector includes: Performing attention processing on the topic feature vector using an attention network to obtain an attention feature vector; The attention feature vector is fused with the first to-be-processed text to obtain the first fused feature vector.

5. The method according to claim 1, wherein The method further comprises: Acquire a plurality of user preference texts and a plurality of global texts, and use each of the user preference texts and the global text as a second text to be processed; For each second text to be processed, fusing the topic feature vectors corresponding to the second text to be processed and at least one second topic to obtain a second fused feature vector corresponding to each second topic; Encoding each of the second fused feature vectors using an encoder of the summary generation model to obtain a summary feature vector corresponding to each of the second to-be-processed texts and each of the second topics; At least one recommended text is selected from the plurality of global texts to be recommended to the user based on the similarity between the summary feature vectors of the global texts and the user preferred texts corresponding to the same second topic.

6. The method according to claim 5, characterized in that The user preference text is a text on which the user performs at least one of the following operations: like, favorite; And / or, before selecting at least one recommended text to be recommended to the user from the plurality of global texts based on the similarity between the summary feature vectors of each global text and each user preferred text corresponding to the same second topic, the method further includes at least one of the following steps: performing pooling processing on the summary feature vectors of each second to-be-processed text corresponding to each second topic; Performing dimensionality reduction processing on the abstract feature vectors of each second text to be processed corresponding to each second topic using principal component analysis; And / or, selecting at least one recommended text to the user from the plurality of global texts based on the similarity between the summary feature vectors of each global text and each user preferred text corresponding to the same second topic includes: For each of the second topics, obtaining a similarity between each of the global texts and a summary feature vector of each of the user preferred texts corresponding to the second topic; Sort the similarities between each global text and each user preference text under the second topic, and select a preset number of different global texts from the sorting as recommended texts for the second topic; or, for each global text, obtain a central tendency statistical value of the similarities between the global text and each user preference text under the second topic, and select the global text with the central tendency statistical value in the preset number as the recommended text for the second topic.

7. The method according to claim 1, characterized in that The training step of the summary generation model includes: Fine-tuning the large language model using topic feature vectors corresponding to a plurality of first training texts and at least one first training topic to obtain a fine-tuned large language model, wherein the first training texts are pre-annotated with actual summaries corresponding to the first training topics; The fine-tuned large language model is optimized using topic feature vectors corresponding to several second training texts and at least one second training topic to obtain the summary generation model. The second training texts are annotated with actual summaries corresponding to each of the second training topics. The actual summaries annotated with at least one of the second training texts are generated by the fine-tuned large language model and corrected.

8. The method according to claim 7, characterized in that In the fine-tuning process, a low-rank adaptation method is used to adjust parameters; And / or, before optimizing the fine-tuned large language model using the topic feature vectors corresponding to the plurality of second training texts and the at least one second training topic to obtain the summary generation model, the method further includes: using at least one of the second training texts as a text to be annotated; fusing the text to be annotated and the topic feature vectors corresponding to each of the second training topics respectively to obtain a fourth fused feature vector corresponding to each of the second training topics; Processing the fourth fused feature vector corresponding to each second training topic using the summary generation model to obtain a candidate summary corresponding to each second training topic; Obtaining correction results of the candidate summaries corresponding to the second training topics by the annotator as second actual summaries corresponding to the second training topics annotated by the text to be annotated; And / or, fine-tuning the large language model using topic feature vectors corresponding to the plurality of first training texts and at least one first training topic, or optimizing the fine-tuned large language model using topic feature vectors corresponding to the plurality of second training texts and at least one second training topic, including: Using the first training text as the target training text and the first training topic as the target topic, or using the second training text as the target training text and the second training topic as the target topic; fusing the target training text and the topic feature vectors corresponding to each target topic respectively to obtain a target fused feature vector corresponding to each target topic; Processing the target fusion feature vector corresponding to each target topic using the large language model to obtain a prediction summary corresponding to each target topic; Parameters in the large language model are adjusted using the difference between the actual summary and the predicted summary corresponding to each target topic.

9. A training method for a summary generation model, characterized in that: include: Fine-tuning a large language model using topic feature vectors corresponding to a plurality of first training texts and at least one first training topic to obtain a fine-tuned large language model, wherein the first training texts are pre-annotated with actual summaries corresponding to the first training topics, and the topic feature vectors corresponding to the first training texts and the first training topics are fused and respectively input into the large language model; The fine-tuned large language model is optimized using topic feature vectors corresponding to several second training texts and at least one second training topic to obtain the summary generation model, wherein the second training texts are annotated with actual summaries corresponding to the second training topics, and the actual summaries annotated with at least one second training text are generated and corrected by the fine-tuned large language model, and the topic feature vectors corresponding to the second training texts and the second training topics are fused and input into the large language model respectively.

10. A summary generation device, characterized in that: The device comprises: A text acquisition module, used to acquire a first text to be processed; a feature vector acquisition module, configured to fuse the topic feature vectors corresponding to the first text to be processed and the first topic to obtain a first fused feature vector, wherein the topic feature vector corresponding to the first topic is used to instruct a summary generation model to generate a summary corresponding to the first topic in the first text to be processed; The summary acquisition module is used to process the first fused feature vector using the summary generation model to obtain a summary text of the first topic in the first text to be processed.

11. A training device for a summary generation model, characterized in that: The device comprises: a fine-tuning module, configured to fine-tune the large language model using topic feature vectors corresponding to a plurality of first training texts and at least one first training topic, to obtain a fine-tuned large language model, wherein the first training texts are pre-annotated with actual summaries corresponding to the first training topics, wherein the topic feature vectors corresponding to the first training texts and the first training topics are fused and respectively input into the large language model; An optimization module is used to optimize the fine-tuned large language model using topic feature vectors corresponding to several second training texts and at least one second training topic to obtain the summary generation model, wherein the second training texts are annotated with actual summaries corresponding to the second training topics, the actual summaries annotated with at least one second training text are generated by the fine-tuned large language model and corrected, and the topic feature vectors corresponding to the second training texts and the second training topics are fused and input into the large language model respectively.

12. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the summary generation method according to any one of claims 1 to 8 or the training method of the summary generation model according to claim 9.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the summary generation method according to any one of claims 1 to 8 or the training method of the summary generation model according to claim 9.

Images