Text abstract generation method based on language large model and related device

Abstract

The invention discloses a text abstract generation method based on a language large model and a related device, and relates to the technical field of natural language processing. The method comprises the steps that a training data set is constructed based on data of a target domain, the training data set comprises multiple sets of training data, and each set of training data comprises an original text and a corresponding abstract; constructing a target large model based on the language large model, wherein the target large model is used for generating a corresponding abstract based on the input text; performing fine tuning on the target large model by using the training data set to obtain a fine-tuned large model; and performing abstract generation processing on a target domain text based on the fine-tuned large model to obtain a target abstract. According to the embodiment of the invention, the quality of the generated abstract can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text summary generation method based on a language large model and related devices. Background Art

[0002] Text summarization is a natural language processing task whose main goal is to automatically extract key information from the original text and generate a concise and accurate summary. Text summarization can be divided into extractive summarization and generative summarization. Extractive summarization selects key sentences or keywords from the original text, but may lose some key information in the original text; generative summarization generates new sentences to express the main content of the original text, which can better capture the main content and structure of the original text.

[0003] The development of early text summarization can be divided into rule-based text summarization technology and machine learning-based text summarization technology. Rule-based text summarization technology mainly relies on manually designed rules and strategies, such as keyword extraction, phrase extraction, sentence compression, etc. Although simple and general, it lacks versatility and flexibility. The main methods of machine learning-based text summarization technology include template-based methods, sequence labeling-based methods, and deep learning-based methods. Among them, the template-based method generates a summary by mapping the original text to a predefined summary template through training a template model. The sequence labeling-based method converts the text summarization problem into a sequence labeling problem and generates a summary by training a sequence labeling model. The deep learning-based method generates a summary by training deep learning models, such as recurrent neural networks and convolutional neural networks.

[0004] Large Language Models (LLMs) refer to neural network models with a large number of parameters, which are usually used to process large data sets and complex tasks. Large language models can capture more features and patterns in the data, thereby improving the performance of the task. Text summarization technologies based on large models mainly include methods based on pre-trained large models and methods based on fine-tuning large models. When existing technologies use large language models to generate summaries, it is difficult to generate summaries that meet user expectations based on user preferences, resulting in poor quality of the generated summaries. Summary of the invention

[0005] The embodiment of the present invention provides a text summary generation method based on a large language model and a related device to solve the problem of poor quality of the generated summary in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, an embodiment of the present invention provides a method for generating a text summary based on a large language model, comprising: Get the target domain text; Based on the fine-tuned large model, the target domain text is summarised to obtain the target summary. The fine-tuned large model is constructed by the following steps: Building a training data set based on the data in the target domain, wherein the training data set includes multiple groups of training data, each group of training data includes original text and corresponding summaries; Building a target big model based on the language big model, wherein the target big model is used to generate a corresponding summary based on the input text; The target large model is fine-tuned using the training data set to obtain a fine-tuned large model.

[0007] Optionally, the process of performing summary generation processing on the target domain text based on the fine-tuned large model to obtain a target summary includes: Build a security control module based on the pre-built sensitive word library; Calling the security control module to pre-process the target domain text to obtain input text; Inputting the input text into the fine-tuned large model to generate a summary, thereby obtaining an output text; The security control module is called to post-process the output text to obtain the target summary.

[0008] Optionally, the calling of the security control module to pre-process the target domain text to obtain the input text includes: Matching the target domain text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is recognized that the target domain text contains a target sensitive word in the sensitive word library, the target sensitive word in the target domain text is replaced with a replacement word corresponding to the target sensitive word to obtain the input text, where the target sensitive word is any sensitive word in the sensitive word library.

[0009] Optionally, the calling of the security control module to post-process the output text to obtain the target summary includes: Matching the output text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is identified that the output text contains a target sensitive word in the sensitive word library, the target sensitive word in the output text is replaced with a replacement word corresponding to the target sensitive word to obtain the target summary, where the target sensitive word is any sensitive word in the sensitive word library.

[0010] Optionally, the building of a security control module based on a pre-built sensitive word library includes: Inserting the sensitive words in the pre-built sensitive word library into the Trie tree as a search data structure of the AC automaton; Construct a fail pointer so that when the current character does not match, it jumps to the character with the longest common prefix and suffix to continue matching.

[0011] Optionally, after performing summary generation processing on the target domain text based on the fine-tuned large model to obtain the target summary, the method further includes: Using a manual evaluation method to evaluate the target summary to obtain a first score, and using an automatic evaluation method to evaluate the target summary to obtain a second score; When the first score and the second score meet preset requirements, the target abstract is determined as a qualified abstract.

[0012] In a second aspect, an embodiment of the present invention provides a text summary generation device based on a language large model, comprising: The acquisition module is used to obtain the target domain text; The summary generation module is used to generate a summary of the target domain text based on the fine-tuned large model to obtain the target summary; The fine-tuned large model is constructed by the following steps: Building a training data set based on the data in the target domain, wherein the training data set includes multiple groups of training data, each group of training data includes original text and corresponding summaries; Building a target big model based on the language big model, wherein the target big model is used to generate a corresponding summary based on the input text; The target large model is fine-tuned using the training data set to obtain a fine-tuned large model.

[0013] Optionally, the process of performing summary generation processing on the target domain text based on the fine-tuned large model to obtain a target summary includes: Build a security control module based on the pre-built sensitive word library; Calling the security control module to pre-process the target domain text to obtain input text; Inputting the input text into the fine-tuned large model to generate a summary, thereby obtaining an output text; The security control module is called to post-process the output text to obtain the target summary.

[0014] Optionally, the calling of the security control module to pre-process the target domain text to obtain the input text includes: Matching the target domain text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is recognized that the target domain text contains a target sensitive word in the sensitive word library, the target sensitive word in the target domain text is replaced with a replacement word corresponding to the target sensitive word to obtain the input text, where the target sensitive word is any sensitive word in the sensitive word library.

[0015] Optionally, the calling of the security control module to post-process the output text to obtain the target summary includes: Matching the output text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is identified that the output text contains a target sensitive word in the sensitive word library, the target sensitive word in the output text is replaced with a replacement word corresponding to the target sensitive word to obtain the target summary, where the target sensitive word is any sensitive word in the sensitive word library.

[0016] Optionally, the building of a security control module based on a pre-built sensitive word library includes: Inserting the sensitive words in the pre-built sensitive word library into the Trie tree as a search data structure of the AC automaton; Construct a fail pointer so that when the current character does not match, it jumps to the character with the longest common prefix and suffix to continue matching.

[0017] Optionally, it also includes: The determination module is used to evaluate the target abstract by a manual evaluation method to obtain a first score, and to evaluate the target abstract by an automatic evaluation method to obtain a second score; if the first score and the second score meet preset requirements, the target abstract is determined as a qualified abstract.

[0018] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the method for generating a text summary based on a large language model as described in the first aspect.

[0019] In a fourth aspect, an embodiment of the present invention provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the method for generating a text summary based on a large language model as described in the first aspect.

[0020] In a fifth aspect, an embodiment of the present invention provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the text summary generation method based on a large language model as described in the first aspect.

[0021] Compared with the prior art, the present invention has the following beneficial effects: In an embodiment of the present invention, a training data set is constructed based on data in a target domain, the training data set includes multiple groups of training data, each group of training data includes original text and corresponding summaries; a target large model is constructed based on a language large model, and the target large model is used to generate a corresponding summary based on the input text; the target large model is fine-tuned using the training data set to obtain a fine-tuned large model; and a summary generation process is performed on the target domain text based on the fine-tuned large model to obtain a target summary. The beneficial effect of the present invention is that the target large model is constructed using an open source large model as a model base, and then the target large model is fine-tuned using professional data in the target domain, which can not only ensure a higher generalization ability of the model, but also allow the model to have certain professional knowledge, so that it can generate a summary that is more in line with user expectations based on user needs and preferences, and can understand the implicit information in the text, and can ensure more accurate summary quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0023] Figure 1 is a flowchart of a text summary generation method based on a language large model provided by an embodiment of the present invention; Figure 2 is a schematic diagram of a security control module provided by an embodiment of the present invention; Figure 3 This is one of the flow charts of the customer service text summary generation method based on the power language big model provided by the embodiment of the present invention; Figure 4 This is the second flow chart of the customer service text summary generation method based on the power language big model provided by the embodiment of the present invention; Figure 5 A flowchart of another text summary generation method based on a language large model is provided in an embodiment of the present invention; Figure 6 It is a structural schematic diagram of a text summary generation device based on a language large model provided by an embodiment of the present invention; Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In the embodiments of the present application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In the embodiments of the present application, the term "multiple" refers to two or more, and other quantifiers are similar. The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] See also Figure 1 , Figure 1 The flowchart of the text summary generation method based on the language big model provided by the embodiment of the present invention is as follows. The method provided by the embodiment of the present invention can be applied to process text content in different fields to obtain corresponding text summaries. For example, the embodiment of the present invention can be applied to scenarios such as work order production, news reporting, knowledge base and search engine.

[0028] like Figure 1As shown, the embodiment of the present application provides a method for generating a text summary based on a large language model, and the method specifically includes the following steps: Step 101 : constructing a training data set based on data in a target domain, wherein the training data set includes multiple groups of training data, and each group of training data includes original text and a corresponding summary.

[0029] Step 102: construct a target large model based on the language large model, wherein the target large model is used to generate a corresponding summary based on the input text.

[0030] Step 103: Use the training data set to fine-tune the target large model to obtain a fine-tuned large model.

[0031] Step 104, obtaining the target domain text; performing summary generation processing on the target domain text based on the fine-tuned large model to obtain a target summary.

[0032] For the convenience of description, in the subsequent embodiments, the application of the embodiment of the present invention in the electric power field will be taken as an example for explanation.

[0033] In step 101, data in the target field is collected, and the collected data is adaptively processed so that the collected data is converted into a format that can be recognized by the model, thereby obtaining a training data set.

[0034] In step 102, a target large model is constructed based on the language large model. The language large model is an open source model. In this embodiment, the open source large model is used as the model base to construct the target large model. Specifically, based on the open source base model, the original objective function is changed from the general language model to the objective function of the text summarization task, and the model training code is written according to the training goal of the text summary. Through the above settings, the open source large model is used as the model base, which can ensure a higher generalization ability of the model.

[0035] In step 103, the target large model is fine-tuned using the training data set to obtain a fine-tuned large model. Through the above settings, a summary that better meets the user's expectations can be generated according to the user's needs and preferences, and the implicit information in the text can be understood, thereby ensuring a more accurate summary quality.

[0036] Optionally, in some embodiments, step 104 includes: Build a security control module based on the pre-built sensitive word library; Calling the security control module to pre-process the target domain text to obtain input text; Inputting the input text into the fine-tuned large model to generate a summary, thereby obtaining an output text; The security control module is called to post-process the output text to obtain the target summary.

[0037] In this embodiment, a security control module is constructed using a pre-built sensitive word library. The input and output of the fine-tuned large model are controlled by the security control module, which can ensure the accuracy and timeliness of model filtering and the security of model output.

[0038] In the specific implementation, the target domain text can be input into the fine-tuned large model by writing the security control module code and the fine-tuned large model call interface to generate the target summary. Considering the illusion and security issues of the large model, a security control module is written based on the pre-built sensitive word library as the security control of the model.

[0039] Optionally, in some embodiments, the building of a security control module based on a pre-built sensitive word library includes: Inserting the sensitive words in the pre-built sensitive word library into the Trie tree as a search data structure of the AC automaton; Construct a fail pointer so that when the current character does not match, it jumps to the character with the longest common prefix and suffix to continue matching.

[0040] Optionally, in some embodiments, the purpose of the security control module is to filter out sensitive words in the input / output.

[0041] First, insert all the contents of the pre-built sensitive word library into the Trie tree as the search data structure of the AC automaton; second, construct a fail pointer so that when the current character mismatches, it jumps to the character with the longest common prefix and suffix to continue matching. Just like the KMP algorithm (a string matching algorithm), when the AC automaton matches, if the current character fails to match, it uses the fail pointer to jump. Find the correct point for the fail pointer of each child node through breadth-first traversal; finally, scan the main string for matching.

[0042] It should be understood that the sensitive word library is pre-built and includes multiple sensitive words and replacement words corresponding to the sensitive words. According to actual needs, the sensitive words can be sensitive words or other words, and the sensitive word library can also be increased, decreased or adjusted according to actual needs.

[0043] In the specific implementation, in order to facilitate the call of the fine-tuned large model, fastapi is used to write the model interface, and it is ensured that the input and output of the model will pass through the security control module to ensure security.

[0044] Optionally, in some embodiments, the calling of the security control module to pre-process the target domain text to obtain the input text includes: Matching the target domain text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is recognized that the target domain text contains a target sensitive word in the sensitive word library, the target sensitive word in the target domain text is replaced with a replacement word corresponding to the target sensitive word to obtain the input text, where the target sensitive word is any sensitive word in the sensitive word library.

[0045] Optionally, in some embodiments, the calling the security control module to post-process the output text to obtain the target summary includes: Matching the output text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is identified that the output text contains a target sensitive word in the sensitive word library, the target sensitive word in the output text is replaced with a replacement word corresponding to the target sensitive word to obtain the target summary, where the target sensitive word is any sensitive word in the sensitive word library.

[0046] Optionally, in some embodiments, after step 104, the method further includes: Using a manual evaluation method to evaluate the target summary to obtain a first score, and using an automatic evaluation method to evaluate the target summary to obtain a second score; When the first score and the second score meet preset requirements, the target abstract is determined as a qualified abstract.

[0047] In this embodiment, the generated target summary is evaluated by combining manual evaluation and automatic evaluation. The manual evaluation method is that business experts score the generated target summary according to evaluation criteria (such as accuracy, conciseness, completeness, etc.); the automatic evaluation method is to use automatic evaluation indicators (such as ROUGE, BERTScore, etc.) to evaluate the similarity and coverage between the target summary generated by the model and the input target domain text. Among them, ROUGE (Recall-Oriented Understudy for Gisting Evaluation) focuses on recall rate, and BERTScore is a natural language processing evaluation method, which is usually used to evaluate the quality of generated text.

[0048] The quality of the target abstract is evaluated by combining the first score and the second score. Specifically, in some embodiments, when the first score is greater than or equal to a first threshold, and / or the second score is greater than or equal to a second threshold, the target abstract is determined to be a qualified abstract.

[0049] The training of traditional generative text summarization first encodes the text with an encoder, and then uses a decoder to recursively predict each text in the summary. The fitting target is a manually constructed summary standard answer. For a document, many summaries can be generated, and the quality of these summaries varies. However, the maximum likelihood estimation (MLE) requires that the fitting target must be a unique standard answer. Therefore, there are problems such as inconsistency between training and downstream actual tasks, and the inability to evaluate the quality of summaries. Moreover, the traditional generative text summary model is very poor in judging the relative order of two summaries of different quality, and cannot achieve satisfactory results.

[0050] Through the above method, the quality of the summary can be evaluated. By combining manual evaluation with automatic evaluation, the accuracy of the evaluation results is improved, the quality of the text summary is further improved, and the problem of the inability to evaluate the quality of the summary and the poor evaluation effect is effectively solved.

[0051] See also Figure 2-Figure 4 As a specific embodiment, the method provided in this embodiment specifically includes the following steps: S1: Data preparation. Specifically, collect power customer service data (such as 12 types of data such as installation consultation and single-household repair), and the collected data includes original text and corresponding summaries. Then annotate the collected data into the format required for the text summary task. In this embodiment, the power customer service data includes online customer service data and intelligent customer service data.

[0052] Exemplarily, a data conversion script is used to convert text summary data into a data format that can be recognized by the model: {"instruction":"Please summarize the following text: text", "input":"", "output":"summary",}.

[0053] S2: Design the objective function and write the model training code. Based on the open source base model, change the original objective function from the general language model to the objective function of the text summarization task, write the model training code according to the training goal of the text summary, and obtain the target large model.

[0054] In this embodiment, a probability model is used to predict the probability of the next word in the text sequence. Assuming we have a vocabulary set S containing n words, the probability model can be expressed as: ; in, represents the tth word, Represents the preceding word sequence.

[0055] The training goal is to maximize the log probability of the probability model, assuming that the original text is , the abstract is , then the objective function can be expressed as: .

[0056] S3: Fine-tune the model. Use the power customer service fine-tuning data (i.e., text summary fine-tuning data) to fine-tune the target large model using the low-rank adaptive fine-tuning method to adapt to the text summary task. The model uses the AdamW optimizer during training.

[0057] S4: Write the security control module code and the fine-tuned large model calling interface, input the original text into the fine-tuned large model, and generate a summary. Considering the illusion and security issues of the large model, write a security control module based on the sensitive word library as the security control of the model. In order to make it easier to call the large model, use fastapi to write the model interface, and ensure that the input and output of the model will pass through the security control module to ensure security.

[0058] As a specific example, Figure 2 As shown in the figure, a sensitive word set is obtained based on a pre-built sensitive word library, and sensitive words are inserted into a tree by traversing the set as the search data structure of the AC automaton. The target domain text (or session content) is input from the client, and the session content is traversed to determine whether the target domain text contains sensitive words. If sensitive words are found, the sensitive words are replaced to obtain the input text. The input text is input into a fine-tuned large model (or a text summary large model) for processing to obtain the output text (or model result). By traversing the model output result, it is determined whether the output text contains sensitive words. If sensitive words are found, the sensitive words are replaced to obtain the target summary (or session summary).

[0059] S5: Text summary evaluation. The summaries generated by the large model are evaluated by combining manual evaluation and automatic evaluation. The summaries with a manual evaluation score of more than 0.9 and a model score of more than 0.85 are considered qualified summaries.

[0060] In this embodiment, an open source large model is used as the model base, and it is fine-tuned using power professional data, which can not only ensure the model's higher generalization ability, but also allow the model to have certain power professional knowledge. Using a large model fine-tuned by power data to summarize text can not only generate a summary that better meets user expectations based on user needs and preferences, but also understand the implicit information in the text, which can ensure more accurate summary quality. At the same time, through the above method, the consistency of summary training and downstream actual tasks can also be guaranteed.

[0061] This model helps improve the application of the big model in the power field by fine-tuning the open source big model with power professional data and summarizing the power text. It should be understood that this model is not limited to the power field, and can be constructed to fine-tune data in other fields and expand to other fields for text summarization. For details, please refer to the above description and will not be repeated here.

[0062] The specific application of the embodiments of the invention is introduced below with two specific embodiments.

[0063] Embodiment 1 Extracting text summaries from customer service chat data based on embodiments of the present invention The customer service chat data (i.e. the target domain text) is as follows: “[Agent]: Hello, this is the customer service center of the power company. I am customer service representative XXX. How can I help you? [User]: Hello, there was a sudden power outage in my house and I don’t know why.

[0064]

Agent

[0065] [Agent]: OK, I have recorded your address. Do you use other electrical equipment or make any electrical changes in your home? [User]: No, I have always used electricity normally and have not made any changes.

[0066] Agent: OK, I understand. I will check the cause of the power outage for you immediately and arrange for maintenance personnel to come and handle it as soon as possible. Please keep your phone open so that we can contact you.

[0067] [User]: OK, thank you.

[0068] [Agent]: You're welcome. This is what we should do. Do you have any other questions or need help?

User

[0069] [Agent]: OK, I wish you a happy life! I will withdraw from this service.

[0070]

User

[0071] It is reported that the power company plans to invest a lot of money and resources in the next few years for the construction of smart grid infrastructure, technology research and development, and talent training. The construction of smart grid will include smart meters, smart substations, smart dispatching centers, etc., and through real-time monitoring and data analysis, it will achieve precise control and optimized dispatching of the power grid.

[0072] At the same time, the power company will also strengthen cooperation with the renewable energy industry to promote the development of green energy. Smart grids will have greater flexibility and adaptability, and will be able to better access renewable energy such as wind power and solar energy, and achieve diversified energy supply and efficient utilization. This will help reduce dependence on traditional energy, reduce carbon emissions, and promote environmental protection and sustainable development.

[0073] The power company's smart grid construction plan has received extensive attention and support from the government and all sectors of society. Industry insiders believe that smart grid construction is an important direction for the transformation and upgrading of the power industry and an important way to achieve green energy development. Through the construction of smart grids, the intelligence level of the power system can be improved, the quality and efficiency of power supply can be improved, and the optimization and upgrading of the energy structure can be promoted.

[0074] The power company said it will adhere to customer-centricity, market demand-oriented, and continuously promote the construction of smart grids and green energy development. At the same time, it will strengthen cooperation and exchanges with domestic and foreign counterparts to jointly promote innovation and development in the power industry. The text summary (and target summary) of the fine-tuned large model output is as follows: "A well-known domestic power company announced that it will accelerate the construction of smart grids, aiming to improve the reliability and sustainability of power supply and promote the development of green energy. The plan includes infrastructure upgrades, technology research and development, and talent training. It will strengthen cooperation with the renewable energy industry and promote the optimization of the energy structure. This move has been supported by the government and society and is considered an important way for the transformation and upgrading of the power industry and green development." See also Figure 5 The embodiment of the present invention also provides a text summary generation method based on a language large model, comprising: S100, obtaining target domain text; S200, performing summary generation processing on the target domain text based on the fine-tuned large model to obtain a target summary; The fine-tuned large model is constructed by the following steps: Building a training data set based on the data in the target domain, wherein the training data set includes multiple groups of training data, each group of training data includes original text and corresponding summaries; Building a target big model based on the language big model, wherein the target big model is used to generate a corresponding summary based on the input text; The target large model is fine-tuned using the training data set to obtain a fine-tuned large model.

[0075] Optionally, in some embodiments, the process of generating a summary of the target domain text based on the fine-tuned large model to obtain a target summary includes: Build a security control module based on the pre-built sensitive word library; Calling the security control module to pre-process the target domain text to obtain input text; Inputting the input text into the fine-tuned large model to generate a summary, thereby obtaining an output text; The security control module is called to post-process the output text to obtain the target summary.

[0076] Optionally, in some embodiments, the calling of the security control module to pre-process the target domain text to obtain the input text includes: Matching the target domain text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is recognized that the target domain text contains a target sensitive word in the sensitive word library, the target sensitive word in the target domain text is replaced with a replacement word corresponding to the target sensitive word to obtain the input text, where the target sensitive word is any sensitive word in the sensitive word library.

[0077] Optionally, in some embodiments, the calling the security control module to post-process the output text to obtain the target summary includes: Matching the output text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is identified that the output text contains a target sensitive word in the sensitive word library, the target sensitive word in the output text is replaced with a replacement word corresponding to the target sensitive word to obtain the target summary, where the target sensitive word is any sensitive word in the sensitive word library.

[0078] Optionally, in some embodiments, the building of a security control module based on a pre-built sensitive word library includes: Inserting the sensitive words in the pre-built sensitive word library into the Trie tree as a search data structure of the AC automaton; Construct a fail pointer so that when the current character does not match, it jumps to the character with the longest common prefix and suffix to continue matching.

[0079] Optionally, in some embodiments, after the target domain text is summarised based on the fine-tuned large model to obtain the target summary, the method further includes: Using a manual evaluation method to evaluate the target summary to obtain a first score, and using an automatic evaluation method to evaluate the target summary to obtain a second score; When the first score and the second score meet preset requirements, the target abstract is determined as a qualified abstract.

[0080] See also Figure 6 The embodiment of the present invention further provides a text summary generation device 500 based on a language large model, comprising: A first construction module 501 is used to construct a training data set based on data in a target field, wherein the training data set includes multiple sets of training data, and each set of training data includes original text and a corresponding summary; A second construction module 502 is used to construct a target large model based on the language large model, wherein the target large model is used to generate a corresponding summary based on the input text; A fine-tuning module 503 is used to fine-tune the target large model using the training data set to obtain a fine-tuned large model; The summary generation module 504 is used to perform summary generation processing on the target domain text based on the fine-tuned large model to obtain a target summary.

[0081] Optionally, the summary generation module 504 includes: A construction unit, used to build a security control module based on a pre-built sensitive word library; A first calling unit, configured to call the security control module to pre-process the target domain text to obtain an input text; A generating unit, used for inputting the input text into the fine-tuned large model to generate a summary, thereby obtaining an output text; The second calling unit is used to call the security management and control module to post-process the output text to obtain the target summary.

[0082] Optionally, the first calling unit is specifically used to: Matching the target domain text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is recognized that the target domain text contains a target sensitive word in the sensitive word library, the target sensitive word in the target domain text is replaced with a replacement word corresponding to the target sensitive word to obtain the input text, where the target sensitive word is any sensitive word in the sensitive word library.

[0083] Optionally, the second calling unit is specifically used to: Matching the output text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is identified that the output text contains a target sensitive word in the sensitive word library, the target sensitive word in the output text is replaced with a replacement word corresponding to the target sensitive word to obtain the target summary, where the target sensitive word is any sensitive word in the sensitive word library.

[0084] Optionally, the construction unit is specifically used for: Inserting the sensitive words in the pre-built sensitive word library into the Trie tree as a search data structure of the AC automaton; Construct a fail pointer so that when the current character does not match, it jumps to the character with the longest common prefix and suffix to continue matching.

[0085] Optionally, the text summary generating device 500 based on the language large model further includes: An evaluation module, configured to evaluate the target summary using a manual evaluation method to obtain a first score, and to evaluate the target summary using an automatic evaluation method to obtain a second score; A determination module is used to determine the target abstract as a qualified abstract when the first score and the second score meet preset requirements.

[0086] Optionally, it also includes: The acquisition module is used to acquire the target domain text; the acquired target domain text is used to be input into the summary generation module 504; the summary generation module 504 is used to perform summary generation processing on the target domain text based on the fine-tuned large model to obtain a target summary.

[0087] The text summary generation device 500 based on the language large model provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be repeated in this embodiment.

[0088] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0089] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0090] like Figure 7 As shown, an embodiment of the present application also provides an electronic device, including: a memory 602, a processor 601, and a program stored in the memory 602 and executable on the processor 601; the processor 601 is used to read the program in the memory 602 to implement the steps in the text summary generation method based on the large language model as described above.

[0091] The embodiment of the present application also provides a readable storage medium, on which a program is stored. When the program is executed by a processor, the various processes of the above-mentioned text summary generation method embodiment based on a large language model are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here. Among them, the readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as compact disk (CD), digital video disc (DVD), Blu-ray Disc (BD), high-definition versatile disc (HVD), etc.), and semiconductor storage (such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read only memory (EEPROM), non-volatile memory (NAND FLASH), solid-state drive (Solid State Disk or Solid State Drive, SSD)), etc.

[0092] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned text summary generation method embodiment based on the language large model are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0093] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0095] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A text summary generation method based on a large language model, characterized in that: include: Get the target domain text; Based on the fine-tuned large model, the target domain text is summarised to obtain the target summary. The fine-tuned large model is constructed by the following steps: Building a training data set based on the data in the target domain, wherein the training data set includes multiple groups of training data, each group of training data includes original text and corresponding summaries; Building a target big model based on the language big model, wherein the target big model is used to generate a corresponding summary based on the input text; The target large model is fine-tuned using the training data set to obtain a fine-tuned large model.

2. The method according to claim 1, characterized in that The target domain text is summarised based on the fine-tuned large model to obtain a target summary, including: Build a security control module based on the pre-built sensitive word library; Calling the security control module to pre-process the target domain text to obtain input text; Inputting the input text into the fine-tuned large model to generate a summary, thereby obtaining an output text; The security control module is called to post-process the output text to obtain the target summary.

3. The method according to claim 2, characterized in that The calling of the security control module to pre-process the target domain text to obtain the input text includes: Matching the target domain text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is recognized that the target domain text contains a target sensitive word, when it is recognized that the target domain text contains a target sensitive word in the sensitive word library, the target sensitive word in the target domain text is replaced with a replacement word corresponding to the target sensitive word to obtain the input text, where the target sensitive word is any sensitive word in the sensitive word library.

4. The method according to claim 2, characterized in that: The calling of the security control module to post-process the output text to obtain the target summary includes: Matching the output text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is recognized that the output text contains a target sensitive word, when it is recognized that the output text contains a target sensitive word in the sensitive word library, the target sensitive word in the output text is replaced with a replacement word corresponding to the target sensitive word to obtain the target summary, where the target sensitive word is any sensitive word in the sensitive word library.

5. The method according to claim 2, characterized in that: The security control module is constructed based on the pre-built sensitive word library, including: Inserting the sensitive words in the pre-built sensitive word library into the Trie tree as a search data structure of the AC automaton; Construct a fail pointer so that when the current character does not match, it jumps to the character with the longest common prefix and suffix to continue matching.

6. The method according to claim 1, characterized in that After the target domain text is summarised based on the fine-tuned large model and the target summary is obtained, the method further comprises: Using a manual evaluation method to evaluate the target summary to obtain a first score, and using an automatic evaluation method to evaluate the target summary to obtain a second score; When the first score and the second score meet preset requirements, the target abstract is determined as a qualified abstract.

7. A text summary generation device based on a language large model, characterized in that: include: The acquisition module is used to obtain the target domain text; The summary generation module is used to generate a summary of the target domain text based on the fine-tuned large model to obtain the target summary; The fine-tuned large model is constructed by the following steps: Building a training data set based on the data in the target domain, wherein the training data set includes multiple groups of training data, each group of training data includes original text and corresponding summaries; Building a target big model based on the language big model, wherein the target big model is used to generate a corresponding summary based on the input text; The target large model is fine-tuned using the training data set to obtain a fine-tuned large model.

8. The device according to claim 7, characterized in that The target domain text is summarised based on the fine-tuned large model to obtain a target summary, including: Build a security control module based on the pre-built sensitive word library; Calling the security control module to pre-process the target domain text to obtain input text; Inputting the input text into the fine-tuned large model to generate a summary, thereby obtaining an output text; The security control module is called to post-process the output text to obtain the target summary.

9. The device according to claim 8, characterized in that The calling of the security control module to pre-process the target domain text to obtain the input text includes: Matching the target domain text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is recognized that the target domain text contains a target sensitive word, when it is recognized that the target domain text contains a target sensitive word in the sensitive word library, the target sensitive word in the target domain text is replaced with a replacement word corresponding to the target sensitive word to obtain the input text, where the target sensitive word is any sensitive word in the sensitive word library.

10. The device according to claim 8, characterized in that The calling of the security control module to post-process the output text to obtain the target summary includes: Matching the output text with the sensitive word library, wherein the sensitive word library includes a plurality of sensitive words and replacement words corresponding to the sensitive words; When it is recognized that the output text contains a target sensitive word, when it is recognized that the output text contains a target sensitive word in the sensitive word library, the target sensitive word in the output text is replaced with a replacement word corresponding to the target sensitive word to obtain the target summary, where the target sensitive word is any sensitive word in the sensitive word library.

11. The device according to claim 8, characterized in that The security control module is constructed based on the pre-built sensitive word library, including: Inserting the sensitive words in the pre-built sensitive word library into the Trie tree as a search data structure of the AC automaton; Construct a fail pointer so that when the current character does not match, it jumps to the character with the longest common prefix and suffix to continue matching.

12. The device according to claim 7, characterized in that Also includes: The determination module is used to evaluate the target abstract by a manual evaluation method to obtain a first score, and to evaluate the target abstract by an automatic evaluation method to obtain a second score; if the first score and the second score meet preset requirements, the target abstract is determined as a qualified abstract.

13. An electronic device, characterized in that: It comprises a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement a text summary generation method based on a large language model as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by the processor, it implements the text summary generation method based on the language large model as described in any one of claims 1 to 6.

15. A computer program product, characterized in that The computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, it implements a text summary generation method based on a language large model as described in any one of claims 1-6.

Images