Text generation model training method, text generation method and related device

Abstract

The invention provides a training method of a text generation model, a text generation method and a related device, and relates to the technical field of text generation. The text generation model training method comprises the steps that a pre-established heterogeneous information network is acquired, and the heterogeneous information network is established based on multiple articles; pre-training an initial text generation model by utilizing the heterogeneous information network; and training the pre-trained initial text generation model by utilizing a preset reinforcement learning method based on a preset prompt word to obtain a text generation model. Under the condition that the text generation model obtained through the training mode is used for generating the text, the characteristics of the article structure and the article content obtained through pre-training learning of the text generation model can be reflected in the output text. Compared with an existing text generation model, the matching degree between the actual content of the text generated by the text generation model obtained through the training mode of the scheme and the theme, the title and the like of the text is higher.

Description

Technical Field

[0001] The present application relates to the technical field of text generation, and in particular to a text generation model training method, a text generation method and related devices. Background Art

[0002] In recent years, text generation models have taken on more and more text generation tasks. Generating text through text generation models has improved the efficiency of text generation. However, the actual content of the text generated by the existing text generation models often has a low degree of match with the subject, title, etc. of the text, resulting in the text generated by the existing text generation models being difficult to meet user expectations. Summary of the invention

[0003] The present application provides a training method for a text generation model, a text generation method and related devices to improve the matching degree between the actual content of the generated text and the theme, title, etc. of the text.

[0004] In a first aspect, the present application provides a text generation model training method, comprising: obtaining a pre-established heterogeneous information network, wherein the heterogeneous information network is constructed based on multiple articles; using the heterogeneous information network to pre-train an initial text generation model; based on preset prompt words, using a preset reinforcement learning method to train the pre-trained initial text generation model to obtain a text generation model.

[0005] In the embodiment of the present application, since the heterogeneous information network is constructed based on multiple articles, the heterogeneous information network can reflect the characteristics of the article structure and article content in the multiple articles. Furthermore, when the initial text generation model is pre-trained using the heterogeneous information network, the initial pre-trained model can learn the characteristics of the article structure and article content reflected in the heterogeneous information network. When the text generation model obtained by the pre-training method is used to generate text, the characteristics of the article structure and article content learned by the pre-training of the text generation model can be reflected in the output text. Therefore, compared with the existing text generation model, the actual content of the text generated by the text generation model obtained by the training method of this scheme has a higher degree of match with the theme, title, etc. of the text.

[0006] In combination with the technical solution provided in the first aspect above, in some possible implementations, the heterogeneous information network includes: feature word nodes and classification theme nodes, article title nodes, article body nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and at least one type of node in the article nodes.

[0007] In the embodiment of the present application, since the basic constituent units of the text are all words, therefore, in the case of at least one type of node including feature word nodes and classification theme nodes, article title nodes, article body nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and article nodes in the heterogeneous information network, the association relationship between the feature word nodes and the constituent units of other dimensions of the text can be reflected. Thus, the heterogeneous information network can reflect the characteristics of the article content, that is, reflect the association relationship between the feature word and the theme, title, etc. of the text.

[0008] In combination with the technical solution provided in the first aspect above, in some possible implementations, the heterogeneous information network includes at least one of the following relationships: the relationship between classification topic nodes and feature word nodes, the relationship between article title nodes and feature word nodes, the relationship between article body nodes and feature word nodes, the relationship between article keyword nodes and feature word nodes, the relationship between article tag nodes and feature word nodes, the relationship between article chapter table of contents nodes and feature word nodes, the relationship between article sentence structure nodes and feature word nodes, and the relationship between article nodes and feature word nodes.

[0009] In combination with the technical solution provided in the first aspect above, in some possible implementations, the heterogeneous information network includes feature word nodes and classification subject nodes, article title nodes, article body nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and article nodes; the heterogeneous information network at least includes the relationship between the classification subject nodes and feature word nodes, the relationship between the article title nodes and feature word nodes, the relationship between the article keyword nodes and feature word nodes, the relationship between the article tag nodes and feature word nodes, the relationship between the article chapter directory nodes and feature word nodes, the relationship between the article sentence structure nodes and feature word nodes, and the relationship between the article nodes and feature word nodes.

[0010] In the embodiment of the present application, since the basic constituent units of the text are all words, therefore, in the case of the heterogeneous information network including feature word nodes and classification theme nodes, article title nodes, article text nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and article nodes, by reflecting the association between the feature word nodes and the constituent units of other dimensions of the text, the heterogeneous information network can reflect the relationship between the basic content of the article and the dimensions of the article other than words. Furthermore, when the initial text generation model is pre-trained using the heterogeneous information network, the initial text generation model can learn the relationship between the basic content of each article and the dimensions of the article other than words, so as to achieve the effect of improving the matching degree between the actual content of the generated text and the theme, title, etc. of the text.

[0011] In combination with the technical solution provided in the first aspect above, in some possible implementations, the relationship between the article chapter directory node and the feature word node is the number of times the specified feature word appears in all articles in which the specified article chapter directory appears; the relationship between the article sentence structure node and the feature word node is the number of times the specified feature word appears in all articles in which the specified sentence appears; the relationship between the classification topic node and the feature word node is the number of times the specified feature word appears in all articles with the specified topic as the topic; the relationship between the article keyword node and the feature word node is the number of times the specified feature word appears in all articles in which the specified keyword appears; the relationship between the article title node and the feature word node is the number of times the specified feature word appears in all articles in which the specified article title participle exists in the title; the relationship between the article node and the feature word node is the number of times the specified feature word appears in the specified article.

[0012] In a text, the more times a feature word appears, the deeper the association between the feature word and the text. Therefore, in the embodiment of the present application, the relationship between the feature word node and other nodes is characterized by the number of times the feature word appears in articles that meet different conditions, so that the degree of association between the feature word and other different nodes can be concisely and accurately reflected. Thereby simplifying the expression of the relationship between the above-mentioned nodes, and also accurately reflecting the relationship between different nodes.

[0013] In combination with the technical solution provided in the first aspect above, in some possible implementations, the initial text generation model is pre-trained using the heterogeneous information network, including: converting any two nodes in the heterogeneous information network and the relationship between the two nodes into vector representations; and pre-training the initial text generation model based on all vector representations obtained by the conversion. In an embodiment of the present application, by converting any two nodes and the relationship between the two nodes into vector representations, the types and relationships of the two nodes can be reflected in one vector representation, so that the initial text generation model can learn the relationship between the two types of nodes through pre-training.

[0014] In a second aspect, the present application provides a text generation method, comprising: obtaining a text generation model trained by the method described in the first aspect above and / or in combination with any possible implementation method of the first aspect above; inputting a prompt word into the text generation model to obtain generated text content.

[0015] In a third aspect, the present application provides a training device for a text generation model, comprising: a first acquisition module and a training module, the first acquisition module being used to acquire a pre-established heterogeneous information network, wherein the heterogeneous information network is constructed based on multiple articles; the training module being used to pre-train an initial text generation model using the heterogeneous information network; based on preset prompt words, the pre-trained initial text generation model is trained using a preset reinforcement learning method to obtain a text generation model.

[0016] In a fourth aspect, the present application provides a text generation device, comprising: a second acquisition module and a processing module, the second acquisition module is used to obtain a text generation model trained by the method described in the first aspect above and / or in combination with any possible implementation method of the first aspect above; the processing module is used to input prompt words into the text generation model to obtain generated text content.

[0017] In a fifth aspect, the present application provides an electronic device, comprising: a memory and a processor, the memory and the processor being connected; the memory being used to store programs; the processor being used to call the programs stored in the memory to execute the method described in the first aspect above and / or in combination with any possible implementation of the first aspect above, or to execute the method described in the second aspect above.

[0018] In a sixth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer, it executes the method described in the first aspect above and / or in combination with any possible implementation of the first aspect above, or executes the method described in the second aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of a flow chart of a first text generation model training method shown in an embodiment of the present application; Figure 2 A schematic diagram of a flow chart of a second text generation model training method shown in an embodiment of the present application; Figure 3 A structural block diagram of a document template library shown in an embodiment of the present application; Figure 4 A flowchart of a text generation method shown in an embodiment of the present application; Figure 5 A structural block diagram of a training device for a text generation model shown in an embodiment of the present application; Figure 6 A structural block diagram of a text generation device shown in an embodiment of the present application; Figure 7 A structural block diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0022] It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it is not necessary to further define and explain it in the subsequent drawings. At the same time, in the description of this application, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements that are not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0023] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0024] See also Figure 1 , Figure 1 A training method for a text generation model is shown in the embodiment of the present application. Figure 1 Describe the steps it involves.

[0025] S100: Acquire a pre-established heterogeneous information network.

[0026] Among them, the heterogeneous information network is constructed based on multiple articles.

[0027] The heterogeneous information network can be pre-acquired and stored in a storage medium or database, and can be directly called when needed. Alternatively, it can be acquired from a third-party device in real time when needed.

[0028] In one implementation, a heterogeneous information network may include: feature word nodes, classification theme nodes, article title nodes, article body nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and at least two types of nodes among article nodes.

[0029] Optionally, the heterogeneous information network may include at least one type of node: feature word nodes and classification theme nodes, article title nodes, article body nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, article nodes, etc.

[0030] Since the basic components of text are words, in the case of at least one type of node including feature word nodes and classification theme nodes, article title nodes, article text nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and article nodes in the heterogeneous information network, the association between feature word nodes and components of other dimensions of the text can be reflected. Thus, the heterogeneous information network can reflect the characteristics of the article content, that is, the association between feature words and the theme, title, etc. of the text.

[0031] Optionally, in the case where a heterogeneous information network may include at least one type of node among: feature word nodes and classification subject nodes, article title nodes, article body nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, article nodes, etc., the heterogeneous information network may also include at least one relationship among the relationship between classification subject nodes and feature word nodes, the relationship between article title nodes and feature word nodes, the relationship between article body nodes and feature word nodes, the relationship between article keyword nodes and feature word nodes, the relationship between article tag nodes and feature word nodes, the relationship between article chapter directory nodes and feature word nodes, the relationship between article sentence structure nodes and feature word nodes, and the relationship between article nodes and feature word nodes.

[0032] The relationship between the article chapter directory node and the feature word node is the number of times the specified feature word appears in all articles where the specified article chapter directory appears. Alternatively, the relationship between the article chapter directory node and the feature word node is the average number of times the specified feature word appears in all articles where the specified article chapter directory appears.

[0033] For example, if the designated feature word is word A, the article directory is directory B, and the articles in which the article directory B appears are article A, article B, article C, and article D. And the number of times word A appears in article A is 10; the number of times word A appears in article B is 13; the number of times word A appears in article C is 9; and the number of times word A appears in article D is 4. Then the number of times word A appears is 10+13+9+4=36. Therefore, the relationship between word A and directory B can be expressed as 36. Alternatively, the relationship between word A and directory B can be expressed as 9 (36÷4, that is, the average number of times word A appears in all articles in which directory B appears). The examples here are only for ease of understanding and are not intended to limit the present application.

[0034] The relationship between the article sentence structure node and the feature word node is the number of times the specified feature word appears in all articles where the specified sentence appears. Alternatively, the relationship between the article sentence structure node and the feature word node is: the average number of times the specified feature word appears in all articles where the specified sentence appears.

[0035] For example, if the designated feature word is word A, the sentence structure node of the article is sentence C, and the articles in which sentence C appears are article A, article B, article C, and article D. And the number of times word A appears in article A is 10; the number of times word A appears in article B is 13; the number of times word A appears in article C is 9; and the number of times word A appears in article D is 4. Then the number of times word A appears is 10+13+9+4=36. Therefore, the relationship between word A and sentence C can be expressed as 36. Alternatively, the relationship between word A and sentence C can be expressed as 9 (36÷4, that is, the average number of times word A appears in all articles in which sentence C appears). The examples here are only for ease of understanding and are not intended to limit the present application.

[0036] The relationship between the article body node and the feature word node is the number of times the specified feature word appears in the article body of the specified article.

[0037] For example, if the designated feature word is word A, and the number of word A appearing in the article body of article A is 8, then the relationship between word A and the article body of article A can be expressed as 8. The examples here are only for ease of understanding and are not intended to limit the present application.

[0038] The relationship between the classification topic node and the feature word node is the number of times the specified feature word appears in all articles with the specified topic. Alternatively, the relationship between the classification topic node and the feature word node is the average number of times the specified feature word appears in all articles with the specified topic.

[0039] For example, if the designated feature word is word A, the classification topic node is topic D, and the articles in which topic D appears are article A, article B, article C, and article D. And the number of times word A appears in article A is 10; the number of times word A appears in article B is 13; the number of times word A appears in article C is 9; and the number of times word A appears in article D is 4. Then the number of times word A appears is 10+13+9+4=36. Therefore, the relationship between word A and topic D can be expressed as 36. Alternatively, the relationship between word A and topic D can be expressed as 9 (36÷4, that is, the average number of times word A appears in all articles in which topic D appears). The examples here are only for ease of understanding and are not intended to limit the present application.

[0040] The relationship between the article keyword node and the feature word node is the number of times the specified feature word appears in all articles where the specified keyword appears. Alternatively, the relationship between the article keyword node and the feature word node is the average number of times the specified feature word appears in all articles where the specified keyword appears.

[0041] For example, if the designated feature word is word A, the article keyword node is keyword E, and the articles in which keyword E appears are article A, article B, article C, and article D. And the number of times word A appears in article A is 10; the number of times word A appears in article B is 13; the number of times word A appears in article C is 9; and the number of times word A appears in article D is 4. Then the number of times word A appears is 10+13+9+4=36. Therefore, the relationship between word A and keyword E can be expressed as 36. Alternatively, the relationship between word A and keyword E can be expressed as 9 (36÷4, that is, the average number of times word A appears in all articles in which keyword E appears). The examples here are only for ease of understanding and are not intended to limit the present application.

[0042] The relationship between the article title node and the feature word node is the number of times the specified feature word appears in all articles with the specified article title participle in the title. Alternatively, the relationship between the article title node and the feature word node is the average number of times the specified feature word appears in all articles with the specified article title participle in the title.

[0043] The specified article title segmentation is the words obtained by segmenting the specified article title. For example, the segmentation of the title "Network Security Protection" can be "Network Security" and "Protection". The examples here are only for ease of understanding and should not be used as a limitation to this application.

[0044] For example, if the designated feature word is word A, the article title node is title F, the participles of title F are participle F1 and participle F2, and the articles in which participle F1 appears in the title are article A, article B, article C, and article D. And the number of word A appearances in article A is 10; the number of word A appearances in article B is 13; the number of word A appearances in article C is 9; and the number of word A appearances in article D is 4. Then the number of times word A appears is 10+13+9+4=36. The articles in which participle F2 appears in the title are article A, article B, and article E. And the number of word A appearances in article E is 5. Then the number of times word A appears is 10+13+9+4+5=41. Therefore, the relationship between word A and title F can be expressed as 41. Alternatively, the relationship between word A and title F can be expressed as 8.2 (41÷5, that is, the average number of times word A appears in all articles in which the title F participle appears). The examples here are only for ease of understanding and are not intended to limit the present application.

[0045] The relationship between an article node and a feature word node is the number of times a specified feature word appears in a specified article.

[0046] For example, if the designated feature word is word A, and the number of occurrences of word A in article A is 10, then the relationship between word A and article A can be expressed as 10. The examples here are only for ease of understanding and are not intended to limit the present application.

[0047] The relationship between the article tag node and the feature word node is: whether the specified feature word is the feature word of the target article tag.

[0048] For example, "video generation" is a feature word of the label "AIGC", and the parent node label of "AIGC" is "artificial intelligence". Then the feature word "video generation" is also a feature word of the parent node label "artificial intelligence" of "AIGC". Such a connection may construct two edges, namely "video generation-AIGC" and "video generation-artificial intelligence". The examples here are only for ease of understanding and should not be used as a limitation to this application.

[0049] In one implementation mode, if the heterogeneous information network includes feature word nodes and classification theme nodes, article title nodes, article body nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and article nodes.

[0050] Correspondingly, the heterogeneous information network at least includes the relationship between classification subject nodes and feature word nodes, the relationship between article title nodes and feature word nodes, the relationship between article keyword nodes and feature word nodes, the relationship between article tag nodes and feature word nodes, the relationship between article chapter table of contents nodes and feature word nodes, the relationship between article sentence structure nodes and feature word nodes, and the relationship between article nodes and feature word nodes.

[0051] The method of expressing the relationship between the feature word node and other nodes has been clearly described in the previous text, and will not be repeated here for the sake of brief description.

[0052] Optionally, the heterogeneous information network may also include the relationship between the classification topic node and the article title node (referring to whether the specified article title exists in the classification topic); the relationship between the classification topic node and the article body node (referring to whether the article body belongs to the specified classification topic); the relationship between the classification topic node and the article keyword node (specify the frequency of article keyword appearing in the specified classification topic); the relationship between the classification topic node and the article tag node (specify whether the article tag belongs to the specified classification topic); the relationship between the classification topic node and the article chapter directory node (specify whether the article chapter directory belongs to the specified classification topic); the relationship between the classification topic node and the article sentence structure node (specify the frequency of article sentence structure appearing under the specified classification topic); the relationship between the classification topic node and the article node (specify whether the article belongs to the specified classification topic); and the relationship between the article title node and the article body node (specify whether the article body belongs to the specified article title); the relationship between the article title node and the article keyword node (specify whether the article keyword appears in the specified article title); the relationship between the article title node and the article tag node (specify whether the article tag marks the article corresponding to the specified article title); the relationship between the article title node and the article chapter directory node (specify whether the article chapter directory and the specified article title correspond to the same article); the relationship between the article title node and the article sentence structure node (specify Whether the article sentence structure appears in the article corresponding to the specified article title); the relationship between the article title node and the article node (whether the specified article title appears in the specified article); the relationship between the article body node and the article keyword node (the frequency of the specified article keyword appearing in the specified article body); the relationship between the article body node and the article chapter directory node (whether the specified article body and the specified article chapter directory correspond to the same batch of articles); the relationship between the article body node and the article sentence structure node (whether the specified article sentence structure appears in the specified article body); the relationship between the article body node and the article node (whether the specified article body belongs to the specified article); the relationship between the article keyword node and the article tag node (whether the specified article keyword is marked by the specified article tag); the relationship between the article keyword node and the article chapter directory node (whether the specified article keyword appears in the specified article chapter directory); the relationship between the article keyword node and the article sentence structure node (whether the specified article keyword appears in the specified article sentence structure); the relationship between the article keyword node and the article node (the frequency of the specified article keyword appearing in the specified article); the relationship between the article tag node and the article chapter directory node (whether the specified article tag marks the article corresponding to the specified article chapter directory);The relationship between the article tag node and the article sentence structure node (whether the specified file tag marks the article with the specified article sentence structure); the relationship between the article tag node and the article node (whether the specified article is marked by the specified article tag); the relationship between the article chapter directory node and the article sentence structure node (whether the specified article sentence structure appears in the article content corresponding to the specified article chapter directory); the relationship between the article chapter directory node and the article node (whether the specified article chapter directory appears in the specified article); the relationship between the article sentence structure node and the article node (whether the specified article sentence structure appears in the specified article). One or more relationships. ;

[0053] In one implementation, the specific method of establishing a heterogeneous information network can be: first obtain multiple articles, and then use NLP (Natural Language Processing) technology to extract features from each article to obtain features such as themes, chapter structure, keywords, sentence structure, etc. The extracted features are used as nodes of the heterogeneous information network, and then the association relationship between two nodes is determined to obtain a well-established heterogeneous information network.

[0054] The method of determining the association relationship between two nodes has been clearly described in the previous text, and for the sake of brief description, it will not be repeated here.

[0055] Optionally, the multiple articles obtained may be articles on the same subject, for example, they may all be articles in the field of network security. Alternatively, the subjects of the multiple articles may also be different.

[0056] Among them, an article template library may be pre-established, and then a heterogeneous information network is constructed directly based on the articles in the article template library.

[0057] In order to facilitate understanding of the specific method of feature extraction for each article, the article "Request for Application for the Working Funds of the Second XXXX Conference and the Promotion Conference of 'Eastern Computer Science and Western Computing and Digital Intelligence City A'" is used as an example for explanation. The extracted article theme is Application for Funds (corresponding to the classification theme node); the title is "Request for Application for the Working Funds of the Second XXXX Conference and the Promotion Conference of 'Eastern Computer Science and Western Computing and Digital Intelligence City A'" (corresponding to the article title node); the article body node corresponds to all the text content of the article; the extracted keywords include: Eastern Computer Science and Western Computing, Digital Intelligence City A, Digital Engine Conference, Working Funds, Promotion Conference (corresponding to the article keyword node); the label is Funds Request (corresponding to the article label node); the chapter directory includes: Application Background and Purpose, Activity Overview, Fund Budget, Activity Arrangement, Participating Guests and Organizations (each directory corresponds to an article chapter directory node); each sentence in the article corresponds to an article sentence structure node; the extracted feature words include Fund Budget and Working Funds (each feature word corresponds to a feature word node).

[0058] The examples given here are only for ease of understanding and should not be construed as limiting the present application.

[0059] S200: Pre-train the initial text generation model using heterogeneous information networks.

[0060] In one implementation, the specific method of pre-training the initial text generation model using the heterogeneous information network may be: first, any two nodes in the heterogeneous information network and the relationship between the two nodes are converted into vector representations, and then the initial text generation model is pre-trained based on all the vector representations obtained by the conversion.

[0061] Among them, the method of using vectors to pre-train the text generation model is well known to those skilled in the art and will not be repeated here for the sake of brief description.

[0062] By converting any two nodes and the relationship between the two nodes into vector representations, the types and relationships of the two nodes can be reflected in one vector representation, so that the initial text generation model can learn the relationship between the two types of nodes through pre-training.

[0063] Optionally, the initial text generation model can be, for example, a large language model, a recurrent neural network (RNNs), a long short-term memory network, etc., and its specific model type is not limited here.

[0064] Alternatively, the specific method of pre-training the initial text generation model using the heterogeneous information network may also be: converting the heterogeneous information network into a vector representation, and then pre-training the initial text generation model using the heterogeneous information network converted into the vector representation.

[0065] S300: Based on the preset prompt words, the pre-trained initial text generation model is trained using a preset reinforcement learning method to obtain a text generation model.

[0066] Among them, the reinforcement learning method can be, for example, a proximal policy optimization algorithm (PPO), a policy gradient algorithm, a trust region policy optimization algorithm, etc.

[0067] The preset prompt word is a pre-set sentence that instructs the initial text generation model to perform a task. For example, it can be "generate an article on network security protection", "write a department funding application", etc. There can be only one preset prompt word or multiple preset prompt words.

[0068] The specific content of the prompt word can be set according to actual needs, and its specific content is not limited here.

[0069] In order to facilitate understanding of the specific method of using the preset reinforcement learning method to train the pre-trained initial text generation model, the following is an example of the preset reinforcement learning method being the proximal strategy optimization algorithm.

[0070] First, the reward model, critic model and reference model can be trained to obtain the trained reward model, critic model and reference model. Then the preset prompt word is input into the pre-trained initial text generation model to obtain the first training text output by the initial text generation model. The first training text is evaluated using the trained reward model, critic model and reference model to obtain the evaluation results output by the reward model, critic model and reference model respectively. The parameters of the initial text generation model are adjusted based on the evaluation results. Then the preset prompt word is input again (the preset prompt word can include only one or more), and the second training text output by the initial text generation model is evaluated using the trained reward model, critic model and reference model, and the parameters of the initial text generation model are adjusted again. Repeat the above steps until the preset cutoff condition is met, stop training, and obtain the trained text generation model.

[0071] Optionally, the preset cutoff condition may be that the number of iterations reaches a preset iteration threshold; or, it may be that the evaluation output by the reward model, the critic model, and the reference model meets preset requirements.

[0072] Among them, the specific methods of training the reward model, the critic model and the reference model are well known to those skilled in the art and will not be repeated here for the sake of brief description.

[0073] To facilitate understanding of the above training process, an example is given below. First, a custom environment class named ShowerEnv can be defined. ShowerEnv inherits from the Env class of gym. In this environment, a discrete action space Discrete (3) and an observation space Box (with a lower limit of 0 and an upper limit of 100) are set. Then, preference data is collected to select the better response between a preset prompt and two responses generated by the supervised fine-tuning model. The collected data will be used to train the reward model and guide the optimization of the policy model (the policy model is also the PPO algorithm). The reward model is trained using the collected human preference data. Then, the text generation model is trained in the ShowerEnv environment using the PPO algorithm, and the parameters of the model are adjusted by simulating human feedback through the reward model. During the training of the text generation model, some key indicators can be monitored, such as the average round length and the average round reward. The performance of the text generation model is evaluated by monitoring the indicators to ensure that the text generated by the text generation model meets the expected preferences (that is, the preferences simulated by the reward model). After the text generation model is trained, the text generation model can also be evaluated and saved for subsequent use. The text generation model can be run in a test environment and can be fine-tuned again as needed to further optimize the performance of the text generation model.

[0074] In order to facilitate the understanding of the above text generation model training method, the following will combine Figure 2 Herein, Figure 2 The text generation model training method described is only one implementation of the text generation model training method provided in this application.

[0075] like Figure 2 As shown, firstly, the articles in the preset document template library are analyzed using NLP technology to extract the key features of the documents. Among them, the implementation method of the document template library can be as follows Figure 3 In addition to storing articles, the document template library can also store information such as tags, outline information, paragraphs, key sentences, etc. obtained by splitting articles.

[0076] The key features extracted include themes, chapter structure, keywords, sentence structure, etc. Then, a heterogeneous information network is constructed based on the extracted features. After that, any two nodes in the heterogeneous information network and the relationship between the two nodes are converted into vector representations, and the initial text generation model is pre-trained based on the vector representations.

[0077] Then the reward model is trained. After the reward model training is completed, the proximal strategy optimization algorithm is used to train the initial text generation model (the initial text generation model after pre-training) to obtain a trained text generation model.

[0078] Furthermore, after obtaining the trained text generation model, it is possible to further receive feedback from users on the use of the text generation model and make further adjustments to the text generation model to obtain a text generation model with better effects.

[0079] in, Figure 2 The specific implementation methods of each step shown have been clearly described in the previous text and will not be repeated here for the sake of brevity.

[0080] Based on the same technical concept, this application also provides a text generation method. Figure 4 As shown, the following will be combined Figure 4 Describe the steps it involves.

[0081] S500: Acquire a text generation model.

[0082] The obtained text generation model is a text generation model obtained by training the aforementioned text generation model. The specific process of the training has been clearly described in the previous text, and will not be repeated here for the sake of brief description.

[0083] S600: Input prompt words into the text generation model to obtain generated text content.

[0084] The prompt word is a sentence that instructs the text generation model what task to perform. For example, it can be "generate an article on network security protection", "write a department funding application", etc. The specific content of the prompt word can be set according to actual needs, and its specific content is not limited here.

[0085] Based on the same technical concept, the present application also provides a training device for a text generation model, such as Figure 5 As shown, the text generation model training device 100 includes: a first acquisition module 110 and a training module 120.

[0086] The first acquisition module 110 is used to acquire a pre-established heterogeneous information network, wherein the heterogeneous information network is constructed based on multiple articles.

[0087] The training module 120 is used to pre-train the initial text generation model using the heterogeneous information network; based on the preset prompt words, the pre-trained initial text generation model is trained using a preset reinforcement learning method to obtain a text generation model.

[0088] In one implementation, the heterogeneous information network includes: feature word nodes and classification theme nodes, article title nodes, article text nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and at least one type of node in article nodes.

[0089] In one implementation mode, the heterogeneous information network includes at least one of the following relationships: the relationship between classification subject nodes and feature word nodes, the relationship between article title nodes and feature word nodes, the relationship between article body nodes and feature word nodes, the relationship between article keyword nodes and feature word nodes, the relationship between article tag nodes and feature word nodes, the relationship between article chapter directory nodes and feature word nodes, the relationship between article sentence structure nodes and feature word nodes, and the relationship between article nodes and feature word nodes.

[0090] In one implementation mode, the heterogeneous information network includes feature word nodes and classification subject nodes, article title nodes, article body nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and article nodes; correspondingly, the heterogeneous information network at least includes the relationship between the classification subject nodes and feature word nodes, the relationship between the article title nodes and feature word nodes, the relationship between the article keyword nodes and feature word nodes, the relationship between the article tag nodes and feature word nodes, the relationship between the article chapter directory nodes and feature word nodes, the relationship between the article sentence structure nodes and feature word nodes, and the relationship between the article nodes and feature word nodes.

[0091] In one implementation mode, the relationship between the article chapter directory node and the feature word node is the number of times the specified feature word appears in all articles where the specified article chapter directory appears; the relationship between the article sentence structure node and the feature word node is the number of times the specified feature word appears in all articles where the specified sentence appears; the relationship between the classification topic node and the feature word node is the number of times the specified feature word appears in all articles with the specified topic as the subject; the relationship between the article keyword node and the feature word node is the number of times the specified feature word appears in all articles where the specified keyword appears; the relationship between the article title node and the feature word node is the number of times the specified feature word appears in all articles with the specified article title participle in the title; the relationship between the article node and the feature word node is the number of times the specified feature word appears in the specified article.

[0092] The training module 120 is specifically used to convert any two nodes in the heterogeneous information network and the relationship between the two nodes into vector representations; and pre-train the initial text generation model based on all vector representations obtained by the conversion.

[0093] The text generation model training device 100 provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned text generation model training method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned text generation model training method embodiment.

[0094] Based on the same technical concept, the present application also provides a text generation device, such as Figure 6 As shown, the text generating device 200 includes a second acquisition module 210 and a processing module 220 .

[0095] The second acquisition module 210 is used to acquire the text generation model trained by the above method.

[0096] The processing module 220 is used to input prompt words into the text generation model to obtain generated text content.

[0097] The text generation device 200 provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned text generation method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned text generation method embodiment.

[0098] See also Figure 7 , which is an electronic device 300 provided in an embodiment of the present application. The electronic device 300 includes: a processor 310 and a memory 320.

[0099] The memory 320 and the processor 310 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 320 is used to store computer programs, such as Figure 5 and / or Figure 6 The software function modules shown in the figure are the training device 100 of the text generation model and / or the text generation device 200. The training device 100 of the text generation model and / or the text generation device 200 include at least one software function module that can be stored in the memory 320 in the form of software or firmware or fixed in the operating system (OS) of the electronic device 300. The processor 310 is used to execute the executable modules stored in the memory 320, such as the software function modules or computer programs included in the training device 100 of the text generation model and / or the text generation device 200.

[0100] Processor 310 is used to obtain a pre-established heterogeneous information network, wherein the heterogeneous information network is constructed based on multiple articles; use the heterogeneous information network to pre-train an initial text generation model; based on preset prompt words, use a preset reinforcement learning method to train the pre-trained initial text generation model to obtain a text generation model.

[0101] Alternatively, the processor 310 is configured to obtain a text generation model; input a prompt word into the text generation model to obtain generated text content.

[0102] Among them, the memory 320 can be, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0103] The processor 310 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), etc.; it may also be a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. A general-purpose processor may be a microprocessor or the processor 310 may also be any conventional processor, etc.

[0104] The electronic device 300 mentioned above includes but is not limited to a personal computer, a server, etc.

[0105] The embodiment of the present application also provides a computer-readable storage medium (hereinafter referred to as storage medium), on which a computer program is stored. When the computer program is run by a computer such as the above-mentioned electronic device 300, the training method of the text generation model and / or the text generation method shown above are executed. The computer-readable storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and other media that can store program codes.

[0106] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A training method for a text generation model, characterized in that: include: Obtaining a pre-established heterogeneous information network, wherein the heterogeneous information network is constructed based on multiple articles; Pre-training an initial text generation model using the heterogeneous information network; Based on the preset prompt words, the pre-trained initial text generation model is trained using a preset reinforcement learning method to obtain a text generation model.

2. The method according to claim 1, characterized in that The heterogeneous information network includes: At least one type of node among feature word nodes and classification theme nodes, article title nodes, article body nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and article nodes.

3. The method according to claim 2, characterized in that The heterogeneous information network includes at least one of the following relationships: the relationship between classification subject nodes and feature word nodes, the relationship between article title nodes and feature word nodes, the relationship between article body nodes and feature word nodes, the relationship between article keyword nodes and feature word nodes, the relationship between article tag nodes and feature word nodes, the relationship between article chapter directory nodes and feature word nodes, the relationship between article sentence structure nodes and feature word nodes, and the relationship between article nodes and feature word nodes.

4. The method according to claim 1, characterized in that: The heterogeneous information network includes feature word nodes and classification theme nodes, article title nodes, article text nodes, article keyword nodes, article tag nodes, article chapter directory nodes, article sentence structure nodes, and article nodes; The heterogeneous information network at least includes the relationship between classification subject nodes and feature word nodes, the relationship between article title nodes and feature word nodes, the relationship between article keyword nodes and feature word nodes, the relationship between article tag nodes and feature word nodes, the relationship between article chapter directory nodes and feature word nodes, the relationship between article sentence structure nodes and feature word nodes, and the relationship between article nodes and feature word nodes.

5. The method according to claim 4, characterized in that The relationship between the article chapter directory node and the feature word node is the number of times the specified feature word appears in all articles where the specified article chapter directory appears; The relationship between the article sentence structure node and the feature word node is the number of times the specified feature word appears in all articles where the specified sentence appears; The relationship between the classification topic node and the feature word node is the number of times the specified feature word appears in all articles with the specified topic; The relationship between the article keyword node and the feature word node is the number of times the specified feature word appears in all articles where the specified keyword appears; The relationship between the article title node and the feature word node is the number of times the specified feature word appears in all articles that have the specified article title participle in the title; The relationship between an article node and a feature word node is the number of times a specified feature word appears in a specified article.

6. The method according to any one of claims 1 to 5, characterized in that: Pre-training the initial text generation model using the heterogeneous information network includes: Convert any two nodes in the heterogeneous information network and the relationship between the two nodes into vector representation; The initial text generation model is pre-trained based on all vector representations obtained by conversion.

7. A text generation method, characterized in that: include: Obtain a text generation model trained by the method according to any one of claims 1 to 6; Input prompt words into the text generation model to obtain generated text content.

8. A training device for a text generation model, characterized in that: include: A first acquisition module is used to acquire a pre-established heterogeneous information network, wherein the heterogeneous information network is constructed based on multiple articles; The training module is used to pre-train the initial text generation model using the heterogeneous information network; based on the preset prompt words, the pre-trained initial text generation model is trained using a preset reinforcement learning method to obtain a text generation model.

9. A text generation device, characterized in that: include: A second acquisition module, used to acquire a text generation model trained by the method according to any one of claims 1 to 6; The processing module is used to input prompt words into the text generation model to obtain generated text content.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are connected; The memory is used to store programs; The processor is used to call the program stored in the memory to execute the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is executed.

Images