Text generation model training method, text generation method and system

By conducting multi-task parallel training of the text generation model and using a large language model for assistance, the problem of lack of structure and flexibility in existing text generation models is solved, and clear structure and high-quality text generation are achieved.

CN119026005APending Publication Date: 2024-11-26CHINA TELECOM ARTIFICIAL INTELLIGENCE TECHNOLOGY (BEIJING) CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411081399.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing text generation models are unable to generate text with clear structure and hierarchy, making it difficult to meet the structural requirements of specific writing tasks. Furthermore, the generated text has a low correlation with the reference content, lacks the flexibility of paragraph-by-paragraph and independent paragraph generation, and the training data is not rigorous enough.

Method used

By acquiring batches of sample text data and splitting them into element unit sequences, multiple training tasks are constructed, including generating text based on titles, generating outlines based on titles, generating text based on outlines, and generating text segment by segment based on outlines with attached content. A large language model is used to assist in generating summaries and keywords, and multi-task parallel training is performed to improve the model's structure and flexibility.

Benefits of technology

It improves the performance of the text generation model in processing structured text, enhances the clarity and hierarchy of the generated text, increases the relevance and flexibility of the generated text with reference content, and improves the quality and credibility of the generated text.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026005A_ABST
    Figure CN119026005A_ABST
Patent Text Reader

Abstract

The invention discloses a training method of a text generation model and a text generation method and system.The method comprises the steps that batch sample text data are obtained, the sample text data are split, and element unit sequences corresponding to the sample text data are obtained; obtaining training data according to the sample text data and the element unit sequence; according to the training data, training a text generation model through multiple parallel training tasks to obtain a trained text generation model; wherein the multiple training tasks comprise a task of generating a text based on a title, a task of generating an outline based on the title, a task of generating the text based on the outline and a task of generating the text segment by segment based on the outline and mounting content. According to the method, different training data can be constructed to realize systematic and diversified training, the performance of processing the structured text generation task by the text generation model can be improved, and the practicability is better. The method can be widely applied to the technical field of artificial intelligence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method for a text generation model, a text generation method and a system. Background Art

[0002] Intelligent writing refers to the technology of automatically generating text content using artificial intelligence technology. Through natural language processing, knowledge graphs and other technical means, computers have the ability to create texts similar to humans, and can autonomously learn and improve the quality of text generation, greatly improving the efficiency of manual writing. Intelligent writing technology has a wide range of application scenarios, including news reports, corporate reports, product descriptions, social media content and other fields, with huge market potential and broad development prospects. Some specific scenarios, such as official document writing and research reports, usually have high requirements for structured writing and reference text writing capabilities.

[0003] In the related art, when generating text, only a continuous text is often generated, and the generated text content cannot be ensured to have a clear structure and hierarchy. The lack of structure and hierarchy is not conducive to readers' understanding and absorption of information, and it is difficult to meet the structural requirements of specific writing tasks (such as academic papers, reports, etc.). Moreover, the current text generation applications often use a relatively simple text generation model processing logic, which can only achieve specific text generation tasks and has poor practicality.

[0004] In summary, the problems existing in related technologies need to be solved urgently. Summary of the invention

[0005] The purpose of this application is to solve one of the technical problems existing in the related art to at least a certain extent.

[0006] To this end, one purpose of the embodiments of the present application is to provide a training method for a text generation model, a text generation method and a system.

[0007] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present application include:

[0008] On the one hand, an embodiment of the present application provides a method for training a text generation model, the method comprising:

[0009] Acquire a batch of sample text data, split the sample text data, and obtain a sequence of element units corresponding to each of the sample text data; wherein the sequence of element units includes a plurality of element units, each of which includes identification information, title text, title type, and text content;

[0010] According to the sample text data and the element unit sequence, training data is obtained; wherein each piece of the training data includes an identification field, a framework field, a summary field and a data field, the framework field includes the original title, introduction, titles of each level and summary information of each level of the sample text data; the summary field includes the outline information of the sample text data; the data field includes a type field, a title field, a content field, a summary field and an element identification field;

[0011] According to the training data, the text generation model is trained through parallel multiple training tasks to obtain a trained text generation model; wherein the multiple training tasks include a task of generating text based on a title, a task of generating an outline based on a title, a task of generating text based on an outline, and a task of generating text paragraph by paragraph based on an outline and mounted content.

[0012] In addition, the training method of the text generation model according to the above embodiment of the present application may also have the following additional technical features:

[0013] Further, in one embodiment of the present application, the task of generating text based on the title includes the following steps:

[0014] Constructing a first prompt word according to the original title; the first prompt word is used to indicate generating text content according to the original title;

[0015] Inputting the first prompt word into the text generation model, predicting the main text data of the sample text data by the text generation model, and obtaining a first prediction result;

[0016] Determining a first loss value according to the text data of the sample text data and the first prediction result;

[0017] The text generation model is trained according to the first loss value.

[0018] Further, in one embodiment of the present application, the task of generating an outline based on the title includes the following steps:

[0019] Acquire a second prompt word; the second prompt word is used to indicate that text content that meets predetermined hierarchical constraint information is generated according to the original title;

[0020] Inputting the second prompt word into the text generation model, predicting the framework field of the sample text data by the text generation model, and obtaining a second prediction result;

[0021] Determining a second loss value according to the framework field of the sample text data and the second prediction result;

[0022] The text generation model is trained according to the second loss value.

[0023] Further, in one embodiment of the present application, the task of generating text based on the outline includes the following steps:

[0024] Constructing a third prompt word according to the outline information; the third prompt word is used to indicate generating text content according to the outline information;

[0025] Inputting the third prompt word into the text generation model, predicting the text data of the sample text data by the text generation model, and obtaining a third prediction result;

[0026] Determining a third loss value according to the text data of the sample text data and the third prediction result;

[0027] The text generation model is trained according to the third loss value.

[0028] Further, in one embodiment of the present application, the task of generating texts paragraph by paragraph based on the outline and the mounted content includes the following steps:

[0029] Constructing a fourth prompt word; the fourth prompt word is used to indicate that the text content corresponding to the target element unit is generated according to the original title, the mounting text, the previous content of the target element unit, and the title field and the abstract field of the target element unit;

[0030] Inputting the fourth prompt word into the text generation model, predicting the text content corresponding to the target element unit by the text generation model, and obtaining a fourth prediction result;

[0031] Determining a fourth loss value according to the text content corresponding to the target element unit and the fourth prediction result;

[0032] The text generation model is trained according to the fourth loss value.

[0033] Furthermore, in one embodiment of the present application, the method further includes:

[0034] Get a pre-trained large language model;

[0035] The summary field is generated according to the content field by using the large language model.

[0036] On the other hand, an embodiment of the present application provides a text generation method, comprising:

[0037] Obtaining a target prompt word input by a user; the target prompt word is used to instruct the text generation model to perform any one of the tasks of generating text based on a title, generating an outline based on a title, generating text based on an outline, and generating text paragraph by paragraph based on an outline and mounted content;

[0038] The target prompt word is input into the text generation model trained by the aforementioned text generation model training method, and the corresponding target text is generated by the text generation model.

[0039] On the other hand, an embodiment of the present application provides a training system for a text generation model, the system comprising:

[0040] An acquisition unit is used to acquire a batch of sample text data, split the sample text data, and obtain a sequence of element units corresponding to each of the sample text data; wherein the sequence of element units includes a plurality of element units, each of which includes identification information, title text, title type, and text content;

[0041] A processing unit, configured to obtain training data according to the sample text data and the element unit sequence; wherein each piece of the training data includes an identification field, a framework field, a summary field and a data field, wherein the framework field includes the original title, introduction, titles of each level and summary information of each level of the sample text data; the summary field includes outline information of the sample text data; and the data field includes a type field, a title field, a content field, a summary field and an element identification field;

[0042] An updating unit is used to train the text generation model through parallel multiple training tasks according to the training data to obtain a trained text generation model; wherein the multiple training tasks include a task of generating text based on a title, a task of generating an outline based on a title, a task of generating text based on an outline, and a task of generating text paragraph by paragraph based on an outline and mounted content.

[0043] On the other hand, an embodiment of the present application provides an electronic device, including:

[0044] at least one processor;

[0045] at least one memory for storing at least one program;

[0046] When the at least one program is executed by the at least one processor, the at least one processor is enabled to implement the above method.

[0047] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, which stores a program executable by a processor, and the above-mentioned processor-executable program is used to implement the above-mentioned method when executed by the processor.

[0048] The advantages and benefits of the present application will be partially given in the following description, and partially become apparent from the following description, or be understood through the practice of the present application:

[0049] The training method, text generation method and system of the text generation model disclosed in the embodiment of the present application, the method obtains a batch of sample text data, splits the sample text data, and obtains a sequence of element units corresponding to each of the sample text data; wherein the element unit sequence includes a plurality of element units, each of which includes identification information, title text, title type and text content; training data is obtained according to the sample text data and the element unit sequence; wherein each piece of the training data includes an identification field, a framework field, a summary field and a data field, the framework field includes the original title, introduction, titles of each level and summary information of each level of the sample text data; the summary field includes outline information of the sample text data; the data field includes a type field, a title field, a content field, a summary field and an element identification field; according to the training data, the text generation model is trained through parallel multi-training tasks to obtain a trained text generation model; wherein the multi-training tasks include a task of generating text based on a title, a task of generating an outline based on a title, a task of generating text based on an outline, and a task of generating text paragraph by paragraph based on an outline and mounted content. This method can construct different training data to achieve systematic and diversified training, which can improve the performance of the text generation model in handling structured text generation tasks and has better practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present application or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present application. For technical personnel in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A schematic diagram of an implementation environment of a text generation model training method provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of a method for training a text generation model provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a specific implementation flow of a training method for a text generation model provided in an embodiment of the present application;

[0054] Figure 4 A flowchart of a task of generating text based on a title provided in an embodiment of the present application;

[0055] Figure 5 A flowchart of a task of generating an outline based on a title provided in an embodiment of the present application;

[0056] Figure 6 A flowchart of a task of generating an outline based on a title provided in an embodiment of the present application;

[0057] Figure 7 A flowchart of a task of generating an outline based on a title provided in an embodiment of the present application;

[0058] Figure 8 A flowchart of a text generation method provided in an embodiment of the present application;

[0059] Fig. 9 A schematic diagram of the structure of a training system for a text generation model provided in an embodiment of the present application;

[0060] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0062] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0064] 1) Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0065] 2) Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning (deep learning) usually includes artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0066] 3) Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. The natural language involved in this field is the language used by people in daily life, so it is also closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0067] 4) Markdown format, a lightweight markup language that allows people to write documents in a plain text format that is easy to read and write. Markdown was originally designed to allow non-technical people to easily write documents, and can be easily converted into other formats such as HTML, PDF or Word. Markdown syntax is simple and intuitive, easy to learn, and for example, the following is one of its data forms:

[0068] ####title

[0069] - `#` for first level headings

[0070] - `##` for secondary headings

[0071] - `###` for level 3 headings

[0072] 5) Intelligent writing refers to the technology that uses artificial intelligence technology to automatically generate text content. Through natural language processing, knowledge graphs and other technical means, computers have the ability to create texts similar to humans, and can autonomously learn and improve the quality of text generation, greatly improving the efficiency of manual writing. Intelligent writing technology has a wide range of application scenarios, including news reports, corporate reports, product descriptions, social media content and other fields, with huge market potential and broad development prospects. Some specific scenarios, such as official document writing and research reports, usually have high requirements for structured writing and reference text writing capabilities.

[0073] The existing AI writing solutions mainly have the following problems:

[0074] 1) The generated text lacks structure and hierarchy: Current long text generation methods can only generate a continuous text, but cannot ensure that the generated text has a clear structure and hierarchy, such as introduction, main text, conclusion, etc. The lack of structure and hierarchy is not conducive to readers' understanding and absorption of information, and it is difficult to meet the structural requirements of specific writing tasks (such as academic papers, reports, etc.).

[0075] 2) The writing effect of reference-mounted text is poor, and it is difficult to reflect the influence of reference content: When generating long texts based on reference texts, existing methods often cannot effectively integrate reference content into the generated text, resulting in a low correlation between the generated text and the reference text.

[0076] 3) Lack of functions for paragraph-by-paragraph generation and independent paragraph generation, and low generation flexibility: Current long text generation methods can usually only generate the entire text at one time, but cannot generate text paragraph by paragraph or sentence by sentence as required, nor can they generate independent paragraphs, which reduces the flexibility and controllability of the generated text and makes it difficult to meet users' customization and refinement requirements for the text generation process.

[0077] 4) Low rigor of training data: The training data of existing intelligent writing models is often insufficient or biased, resulting in the text generated by the model being discriminatory, inaccurate, or not meeting the requirements of professional fields, which reduces the quality and credibility of the generated text and may also have a negative impact on specific groups or fields.

[0078] In view of this, an embodiment of the present application provides a training method for a text generation model, which method obtains a batch of sample text data, splits the sample text data, and obtains a sequence of element units corresponding to each of the sample text data; wherein the element unit sequence includes a plurality of element units, each of which includes identification information, title text, title type and text content; training data is obtained according to the sample text data and the element unit sequence; wherein each piece of the training data includes an identification field, a framework field, a summary field and a data field, the framework field includes the original title, introduction, titles of each level and summary information of each level of the sample text data; the summary field includes outline information of the sample text data; the data field includes a type field, a title field, a content field, a summary field and an element identification field; according to the training data, a text generation model is trained through parallel multi-training tasks to obtain a trained text generation model; wherein the multi-training tasks include a task of generating text based on a title, a task of generating an outline based on a title, a task of generating text based on an outline, and a task of generating text paragraph by paragraph based on an outline and mounted content. This method can construct different training data to achieve systematic and diversified training, which can improve the performance of the text generation model in handling structured text generation tasks and has better practicality.

[0079] Please refer to Figure 1 , Figure 1 The implementation environment diagram of a text generation model training method provided in the embodiment of the present application is shown. In the implementation environment, the main hardware and software entities involved include a terminal device 110 and a backend server 120. The terminal device 110 and the backend server 120 are connected in communication.

[0080] Specifically, the training method of the text generation model provided in the embodiments of the present application can be executed separately on the terminal device 110 side, or can be executed separately on the background server 120 side, or can be executed based on data interaction between the terminal device 110 and the background server 120.

[0081] Among them, the terminal device 110 of the above embodiment may include a mobile phone, a computer, a smart wearable device, a PDA device, an intelligent voice interaction device, a vehicle terminal, etc., but is not limited thereto.

[0082] The backend server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0083] The terminal device 110 and the backend server 120 may establish a communication connection via a wireless network or a wired network. The wireless network or wired network uses standard communication technology and / or protocols, and the network may be set to the Internet or any other network, such as but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network.

[0084] Of course, it is understandable that Figure 1 The implementation environment in the text generation model training method provided in the embodiment of the present application is only some optional application scenarios, and the actual application is not fixed. Figure 1 The hardware and software environment shown.

[0085] Below, in combination with the introduction of the aforementioned implementation environment, a training method for a text generation model provided in an embodiment of the present application is introduced and explained.

[0086] Please refer to Figure 2 , Figure 2 is a schematic diagram of a text generation model training method provided in an embodiment of the present application, and the text generation model training method includes but is not limited to:

[0087] Step 210: obtaining a batch of sample text data, splitting the sample text data, and obtaining a sequence of element units corresponding to each of the sample text data; wherein the sequence of element units includes a plurality of element units, each of which includes identification information, title text, title type, and text content;

[0088] Step 220: obtaining training data according to the sample text data and the element unit sequence; wherein each piece of the training data includes an identification field, a framework field, a summary field and a data field; the framework field includes the original title, introduction, titles of each level and summary information of each level of the sample text data; the summary field includes outline information of the sample text data; the data field includes a type field, a title field, a content field, a summary field and an element identification field;

[0089] Step 230: According to the training data, the text generation model is trained through parallel multi-training tasks to obtain a trained text generation model; wherein the multi-training tasks include a task of generating text based on the title, a task of generating an outline based on the title, a task of generating text based on the outline, and a task of generating text paragraph by paragraph based on the outline and the mounted content.

[0090] In an embodiment of the present application, a method for training a text generation model is provided, which can train a text generation model suitable for multiple types of scenarios.

[0091] In an embodiment of the present application, when training a text generation model, a batch of sample text data can be obtained, and the sample text data can be split and processed to obtain a sequence of element units corresponding to each sample text data. The sample text data here can be text in various fields, and the present application does not limit the specific subject matter covered. After obtaining the sample text data, it can be split and annotated. Specifically, in an embodiment of the present application, the original sample text data can be converted into Markdown format, and the final sample text data will be converted into a sequence composed of element units. In an embodiment of the present application, each sample text data corresponds to an element unit sequence, and the number of element units contained in the element unit sequence corresponding to different sample text data is not limited by the present application, and it can be any value of two or more.

[0092] In the embodiment of the present application, for each element unit, it can include the following information: identification information, title text, title type and text content. Among them, the identification information can be the ID number of the element unit. When the same sample text data is split into levels, the prefix of the identification information of each split element unit should be the same, indicating that it corresponds to the same sample text data. In the embodiment of the present application, the specific data format of the identification information can be flexibly set as needed. The title text refers to the hierarchical title information of the element unit. If the sample text data already includes the corresponding hierarchical title information, it can be directly used as the title text. If there is no corresponding hierarchical title information, it can be supplemented manually. The title type can be used to indicate the level corresponding to the element unit, such as the type can include "first-level title", "second-level title", etc. The present application does not limit the specific number of title types. The text content is all the text content corresponding to the element unit. If there is no corresponding text content under the title of a certain element unit, it can be marked as empty.

[0093] It should be noted that, in the embodiment of the present application, the title type of each element unit needs to be determined according to the actual order. For example, the title type of the next element unit of an element unit with the type of "first-level title" may be "first-level title" or "second-level title" (determined according to the original content), but not "third-level title".

[0094] In the embodiment of the present application, training data can be obtained based on the sample text data and the element unit sequence. Specifically, in the embodiment of the present application, one sample text data can correspond to one training data, and the training data can include fields determined globally based on the sample text data, and fields determined based on each element unit in the element unit sequence corresponding to the sample text data.

[0095] Specifically, in an embodiment of the present application, the fields determined globally based on the sample text data in the training data cover the overall information of the text, and may include an identification (id) field, a frame (frame) field, and an outline (outline) field. The fields determined based on the element unit can usually be in the form of a list. In an embodiment of the present application, an element unit represents a chapter in the sample text data. By splitting the long text into structured unit components, the modularization and flexible application of the text can be achieved. In an embodiment of the present application, the level of each element unit may include a data field, and the data field may specifically include a type (type) field, a title (title) field, a content (content) field, an abstract (abstract) field, and an element identification (sub_id) field, and the field content may be in a string format.

[0096] The specific implementation process of this scheme is as follows: first, the format of a large amount of split and annotated data obtained in the previous process is converted to obtain training data. Each piece of training data consists of multiple fields, including fields corresponding to multiple element units. These element units correspond to chapters corresponding to each outline level in the previous split and annotated process. In this way, the structured and hierarchical splitting and definition of the entire text is completed, so that each element unit of the text is flexible and usable and has rich structured information.

[0097] In the embodiment of the present application, in order to improve the quality and efficiency of long-form writing, a set of task construction schemes is proposed, and an innovative multi-task learning framework is built. This framework relies on an advanced recall model and integrates the technical advantages of a large model. It customizes a special training data set for each link of long-form writing, such as outline construction, paragraph-by-paragraph generation, etc. This application not only systematically enhances the performance of the model in long-form writing tasks, but also realizes diversified ability training, including but not limited to generating outlines from titles, generating full texts based on outlines, and generating texts paragraph by paragraph based on existing content. Specifically, in the embodiment of the present application, according to the training data, the text generation model can be trained through parallel multi-training tasks to obtain a trained text generation model. Here, the multi-training tasks include tasks of generating text based on titles, tasks of generating outlines based on titles, tasks of generating text based on outlines, and tasks of generating text paragraph by paragraph based on outlines and mounted content.

[0098] The following is a brief description of each task:

[0099] Task 1: Generate text based on title

[0100] In the embodiment of the present application, a task of generating text based on a title is constructed to train a large model so that it can directly generate complete text content based on the prompt word information related to the text title input by the user. Since the user input requirements are relatively few, the content and structure of the generated text have a high degree of freedom, which is suitable for scenarios where the format and details of the generated text are not required.

[0101] Task 2: Generate an outline based on the title

[0102] In an embodiment of the present application, a task of generating an outline based on a title is constructed to train a large model so that it can generate a complete outline based on the text title input by the user and the preset outline level information, including an introduction, the title of each chapter, as well as a summary and conclusion information, and generate an outline that meets the structural requirements according to the outline level specified by the user.

[0103] Task 3: Generate text based on the outline

[0104] In an embodiment of the present application, a task of generating text based on an outline is constructed to train a large model so that it can generate a complete text based on the text outline information input by the user, and can automatically identify the hierarchical information of the outline input by the user to generate text that meets the structural requirements. At the same time, the generated text content can highly reflect the summary information contained in the outline input by the user.

[0105] Task 4: Generate text paragraph by paragraph based on the outline and mounted content

[0106] In the embodiment of the present application, a task of generating text paragraph by paragraph based on the outline and the mounted content is constructed to train the large model to generate independent chapter content based on the text title input by the user, the previous generated content, the reference mounted content obtained by recalling the model and introducing the large model capabilities, and the outline information of the current chapter. The generated content can refer to the ideas and logic of the previously generated text content, draw on the important information of the recalled mounted text fragments, and highly reflect the summary information contained in the current chapter outline.

[0107] Please refer to Figure 3 , Figure 3 A specific implementation flow diagram of a text generation model training method provided in an embodiment of the present application is shown. Figure 3 , the application of the training method of the text generation model in the embodiment of the present application is introduced and explained in detail.

[0108] In an embodiment of the present application, a training method for a text generation model is provided, which specifically includes the following steps:

[0109] S1: Data Collection

[0110] In an embodiment of the present application, high-quality texts may be collected as sample text data, and the subject matter may cover a variety of types. The collected text format is required to conform to the outline format, have a clear hierarchical structure, and be able to be split hierarchically according to titles.

[0111] S2: Multi-level outline structure long text data annotation

[0112] After obtaining a large amount of high-quality text in step S1, the data is converted into Markdown format. An annotation template is created based on the converted data and sent to the annotators. The data is split and annotated according to the annotation specifications of long-text writing data with a multi-level outline structure. The text data is split into a list of multiple element units. Each element unit needs to contain the following attributes: identification information, title text, title type, and text content.

[0113] An example of the result of a complete feature unit is as follows: (JSON format)

[0114] {

[0115] "Identification Information": "0_1",

[0116] "Title text": "4. Satisfy people's yearning for a better life and continuously promote reform and innovation",

[0117] "Title Type": "Secondary Title",

[0118] "Text content": "{text content}"

[0119] }

[0120] S3: Original training data construction

[0121] S31: Field information construction

[0122] After the S2 step, a large amount of high-quality annotated data (.xlsx) is obtained, and then the format is converted (.json) to obtain the original training data. Each piece of training data consists of 4 fields, namely, the identification field, the framework field, the summary field and the data field.

[0123] (1) Identification field: The unique identifier of the sample text data, which is mapped one-to-one with the sample text data.

[0124] (2) Framework field: The sample text data is sorted according to the hierarchical structure and is composed of outline information, generally in Markdown format, which usually includes the original title, introduction, titles of each level, and summary information of each level. This information is obtained by extracting the relevant fields of the element unit. The framework field reflects the overall structure of the text and the main content of each level. An example of a framework field is as follows:

[0125] Original title:

[0126] {Text Title}

[0127] introduction:

[0128] {Introduction Text}

[0129] Outline:

[0130] ##1. Level 1 title

[0131] ###1. Secondary title

[0132] Abstract: {Abstract text}

[0133] ###2. Secondary title

[0134] Abstract: {Abstract text}

[0135] ###3. Secondary title

[0136] Abstract: {Abstract text}

[0137] ##2. First level title

[0138] Abstract: {Abstract text}

[0139] ###1. Secondary title

[0140] Abstract: {Abstract text}

[0141] ###2. Secondary title

[0142] Abstract: {Abstract text}

[0143] ##Third, first level title

[0144] ###1. Secondary title

[0145] Abstract: {Abstract text}

[0146] ###2. Secondary title

[0147] Abstract: {Abstract text}

[0148] (3) Summary field: Based on the framework field, the introduction information, summary information, and outline information after the guide words are removed, and it contains more concise structured information. An example of a summary field is as follows:

[0149] {Text Title}

[0150] ##1. Level 1 title

[0151] ###1. Secondary title

[0152] ###2. Secondary title

[0153] ###3. Secondary title

[0154] ##2. First level title

[0155] ###1. Secondary title

[0156] ###2. Secondary title

[0157] ##Third, first level title

[0158] ###1. Secondary title

[0159] ###2. Secondary title

[0160] (4) Data field: Data fields generally include type field, title field, content field, summary field and element identification field. The field content is usually in string format.

[0161] ① Type field: The hierarchical title type of the original text corresponding to the current element unit, which can distinguish the structural information of the text, is an important part of the hierarchical outline, has strict order, and needs to strictly follow the order and hierarchy during the early annotation, otherwise it may lead to problems of disordered text levels in the subsequent generated text. The type field is usually one of a finite number of phrases that can reflect the hierarchy, such as "Original Title", "First-level Title", "Second-level Title", or "Conclusion", etc.

[0162] ② Title field: The title content of the original text section corresponding to the current element unit. It should be noted that if the title content contains serial numbers, it is necessary to establish a corresponding specification between the type field and the serial number format in the title field in advance. For example, the title content with the type field of "First-level Title" should start with capital Arabic numerals, such as "I. XXX", and the title content with the type field of "Second-level Title" should start with lowercase Arabic numerals, such as "1. XXX". Strictly stipulate the serial number specification to avoid problems of inconsistent text formats in the subsequent generated text.

[0163] ③ Content field: The main content of the original text corresponding to the current element unit, with the coverage range being the content between two adjacent titles, and Markdown format is supported.

[0164] ④ Abstract field: A concise abstract of the content field of the current element unit. In the embodiments of the present application, by introducing the capabilities of the large model, the content of the content field can be refined and extracted, and usually the number of words needs to be limited. It should be noted that the content of the abstract field needs to be highly relevant to the content of the content field and can reflect the key points of the content field as much as possible. At the same time, if the content field is empty, the abstract field should also be empty accordingly.

[0165] ⑤ Element identification field: The unique identifier of the current element field, which is mapped one-to-one with the element unit.

[0166] An example of a complete element unit is as follows:

[0167]

[0168] S32: Assisted generation by large model

[0169] In some embodiments, the method further includes:

[0170] Obtaining a pre-trained large language model;

[0171] Generating the abstract field according to the content field through the large language model.

[0172] In the embodiment of the present application, in the process of constructing the original training data, additional information needs to be added to facilitate the subsequent fine-tuning of the data construction. This additional information may not be obtained through the original text, and usually requires extraction and generation from the original text. If manual annotation is introduced, it often takes a lot of manpower and time. Therefore, in the embodiment of the present application, the task of field auxiliary generation can be completed by introducing the existing large model capabilities. Specifically, in the embodiment of the present application, the relevant large language model can be used to assist in the generation of the following content:

[0173] 1) Summary field: The summary field can call the large model interface based on the content field of the element unit to generate summary information semantically related to the content field. In this step, the summary information is required to be related to the content field and to be able to generate summaries with different word count requirements according to the content length of the original training data.

[0174] 2) Keyword list: The keyword list is based on the content field and title field of the element unit. In the embodiment of the present application, a specified number of keywords are generated through the large model capability and summarized into a list.

[0175] S33: Data preprocessing

[0176] After the basic training data is constructed in step S32, in the embodiment of the present application, a series of preprocessing operations can be performed on the data to improve the text cleanliness of the data.

[0177] Specifically, in the embodiments of the present application, the location information of the data can be removed: in the process of writing a long article, try to avoid expressing specific location information. The reasons are as follows: the versatility of the data can be improved: if the pre-training data of the long article writing model contains too much specific location information, the model may be overfitted, resulting in a decrease in generalization ability in other regions or countries. The adaptability of the model can be improved: by avoiding specific location information, the model can be made more flexible and able to adapt to writing tasks in different regions and cultural backgrounds, thereby improving the applicability of the model.

[0178] In the embodiment of the present application, the content field of the element unit can be manually annotated based on the type field and title field of the original training data, and the data containing specific geographic location information can be filtered.

[0179] In an embodiment of the present application, the training data can be deduplicated. It is understandable that duplicate data will reduce the quality of the data set and may lead to poor model training results. Data content duplication includes three situations: 1) Chapter-level duplication: there are texts with completely repeated content; 2) Inter-chapter paragraph duplication: there are one or more repeated paragraphs between different texts; 3) Intra-chapter paragraph duplication: there are one or more repeated paragraphs in the same text. Through automatic comparison of field content, filter the data containing situations one and three. For situation two, if the number of repetitions is small (less than a certain value), you can choose to retain it.

[0180] In an embodiment of the present application, punctuation correction can be performed on the training data: missing and incorrect punctuation in the content field of the element unit that may be caused by annotation errors can be corrected through rules. This can help improve the quality of text generated by the model.

[0181] In an embodiment of the present application, serial number correction can be performed on the training data: the serial number format error problem in the title field of the element unit that may be caused by labeling errors is corrected through rules, which helps to improve the correctness of the text format generated by the model.

[0182] S4: Parallel construction of multi-task training data

[0183] S41: Task 1: Generating text based on titles

[0184] Specifically, in some embodiments, referring to Figure 4 , the task of generating text based on the title includes the following steps:

[0185] Constructing a first prompt word according to the original title; the first prompt word is used to indicate generating text content according to the original title;

[0186] Inputting the first prompt word into the text generation model, predicting the main text data of the sample text data by the text generation model, and obtaining a first prediction result;

[0187] Determining a first loss value according to the text data of the sample text data and the first prediction result;

[0188] The text generation model is trained according to the first loss value.

[0189] In the embodiment of the present application, the training data of Task 1 can be constructed based on sample text data. <input> and <output>It consists of two parts. Among them, <input> Part is a prompt word containing the original title. In the embodiment of the present application, it is recorded as the first prompt word. The first prompt word can be used to indicate that text content is generated according to the original title, wherein the text title can directly extract the original title of the sample text data. <output>Part of it can be used as label data. In the embodiment of the present application, the label data can be the entire text content of the sample text data, supporting the Markdown format. By inputting the first prompt word into the text generation model, the text data of the sample text data can be predicted based on the text generation model to obtain a prediction result, which is recorded as the first prediction result in the embodiment of the present application. Based on the first prediction result and the text data of the sample text data, a first loss value can be determined, so that the first loss value can be used to update the parameters of the text generation model.

[0190] S42: Task 2: Generate an outline based on the title

[0191] Specifically, in some embodiments, referring to Figure 5 , the task of generating an outline based on the title includes the following steps:

[0192] Acquire a second prompt word; the second prompt word is used to indicate that text content that meets predetermined hierarchical constraint information is generated according to the original title;

[0193] Inputting the second prompt word into the text generation model, predicting the framework field of the sample text data by the text generation model, and obtaining a second prediction result;

[0194] Determining a second loss value according to the framework field of the sample text data and the second prediction result;

[0195] The text generation model is trained according to the second loss value.

[0196] Similarly, in the embodiment of the present application, the training data of Task 2 can be constructed based on sample text data. <input> and <output>It consists of two parts. Among them, <input> Part is a prompt word containing the original title and predetermined hierarchical constraint information. In an embodiment of the present application, it is recorded as a second prompt word. The second prompt word can be used to indicate the generation of text content that complies with the predetermined hierarchical constraint information based on the original title, wherein the text title can directly extract the original title of the sample text data. <output>Part of it can be used as label data. In the embodiment of the present application, the label data can be the frame field of the sample text data. By inputting the second prompt word into the text generation model, the frame field of the sample text data can be predicted based on the text generation model to obtain a prediction result, which is recorded as the second prediction result in the embodiment of the present application. Based on the second prediction result and the frame field of the sample text data, a second loss value can be determined, so that the second loss value can be used to update the parameters of the text generation model.

[0197] In an embodiment of the present application, when constructing training data for Task 2, the number of text levels can be specified in the second prompt word through hierarchical constraint information, such as a secondary outline or a tertiary outline. Accordingly, the output content format needs to correspond to the number of input outline levels.

[0198] For example, the training data for task 2 is as follows:

[0199] {

[0200] "input": "<_user>Generate a text outline based on the following information. Generate outline level: Level 2. Requirements: Standard outline format, covering knowledge. Complete title: {title} Keywords: {keyword list} Start generating outline: ",

[0201] "output": "Title: {text title} Introduction: {introduction text} Outline: ##1. First-level title ###1. Second-level title summary: {summary text} ###2. Second-level title summary: {summary text} ###3. Second-level title summary: {summary text} ##2. First-level title summary: {summary text} ###1. Second-level title summary: {summary text} ###2. Second-level title summary: {summary text} ##3. First-level title ###1. Second-level title summary: {summary text} ###2. Second-level title summary: {summary text}"

[0202] }

[0203] S43: Task 3: Generate text based on the outline

[0204] Specifically, in some embodiments, referring to Figure 6 , the task of generating an outline based on the title includes the following steps:

[0205] Constructing a third prompt word according to the outline information; the third prompt word is used to indicate generating text content according to the outline information;

[0206] Inputting the third prompt word into the text generation model, predicting the text data of the sample text data by the text generation model, and obtaining a third prediction result;

[0207] Determining a third loss value according to the text data of the sample text data and the third prediction result;

[0208] The text generation model is trained according to the third loss value.

[0209] In the embodiment of the present application, the training data of Task 3 can also be constructed based on sample text data. <input> and <output>It consists of two parts. Among them, <input> Part is a prompt word containing outline information. In the embodiment of the present application, it is recorded as a third prompt word. The third prompt word can be used to indicate that text content is generated according to the outline information. <output>Part of it can be used as label data. In the embodiment of the present application, the label data can be the body data of the sample text data, supporting the Markdown format. By inputting the third prompt word into the text generation model, the body data of the sample text data can be predicted based on the text generation model to obtain a prediction result, which is recorded as the third prediction result in the embodiment of the present application. Based on the third prediction result and the body data of the sample text data, a third loss value can be determined, so that the third loss value can be used to update the parameters of the text generation model.

[0210] In particular, in the embodiment of the present application, in the process of constructing the training data of Task 3, data enhancement can be achieved by constructing training data with a transformed paragraph order. By introducing input combinations of different orders, the model can learn the independence and combination between different elements, thereby improving the model's generalization ability to new situations; at the same time, it can reduce position deviations. If the model is only trained on data in a fixed order, it may become dependent on a specific order. Through data enhancement, the model can learn the relationship between elements rather than their positions. Therefore, this embodiment designs the following three construction methods for data enhancement.

[0211] 1. Randomly delete the first-level title: Randomly delete an element unit with the type field of "first-level title" and all its subordinate sub-element units (such as second-level units, third-level units...) in the element unit list of each original training data. Accordingly, the title serial numbers of the remaining element units in the element unit list should also be changed, and the frame field and summary field of the original training data should also be changed accordingly, thereby obtaining a new batch of original training data for the new task three.

[0212] 2. Randomly delete the second-level title: Randomly delete an element unit with the type field of the second-level title and all its subordinate sub-element units (such as the third-level title) from the element unit list of each original training data. At the same time, the title numbers of the remaining sub-element units in the upper-level element unit to which it belongs need to be changed at the same time, and the frame field and summary field of the original training data must also be changed accordingly, thereby obtaining a new batch of original training data for the new task three.

[0213] 3. Randomly swap the order of the secondary titles: Randomly select two secondary titles with the type field of "secondary title" under the same first-level title from the element unit list of each original training data, and swap their order. The serial numbers in the titles also need to be changed accordingly, and the framework field and summary field of the original training data also need to be changed accordingly, thereby obtaining a new batch of original training data for the new task three.

[0214] For example, the training data of a task three is as follows:

[0215] {

[0216] "input": "<_user>Generate text content based on the title, introduction, and outline in the following text outline. Requirements: the text format is standardized, the elements are complete, and the text content covers all the first and second level titles in the outline. Text outline: Title: {text title} Introduction: {introduction text} Outline: ##1. First level title###1. Second level title summary: {summary text}###2. Second level title summary: {summary text}###3. Second level title summary: {summary text}##2. First level title summary: {summary text}###1. Second level title summary: {summary text}###2. Second level title summary: {summary text}##3. First level title###1. Second level title summary: {summary text}###2. Second level title summary: {summary text}Conclusion: {conclusion text} Start generating text: ",

[0217] "output": "{full text content}"

[0218] }

[0219] S44: Task 4: Generate text paragraph by paragraph based on the outline and mounted content

[0220] Specifically, in some embodiments, referring to Figure 7 , the task of generating an outline based on the title includes the following steps:

[0221] Constructing a fourth prompt word; the fourth prompt word is used to indicate that the text content corresponding to the target element unit is generated according to the original title, the mounting text, the previous content of the target element unit, and the title field and the abstract field of the target element unit;

[0222] Inputting the fourth prompt word into the text generation model, predicting the text content corresponding to the target element unit by the text generation model, and obtaining a fourth prediction result;

[0223] Determining a fourth loss value according to the text content corresponding to the target element unit and the fourth prediction result;

[0224] The text generation model is trained according to the fourth loss value.

[0225] In the embodiment of the present application, the construction scheme of Task 4 can support the user to manually add and modify the reference text and manually modify the outline information to perform the secondary generation of a single paragraph. The training data of Task 4 can be constructed based on the original training data. <input> and <output>It consists of two parts. <input> The part is recorded as the fourth prompt word information, which can include the following parts: text title: the original title of the original training data; mounting text: based on the recall model, the content field of the element unit in the original training data set is recalled according to the title and abstract of the current outline. At the same time, the large model capability is introduced to expand the mounting data based on the title and abstract to form a mounting text list. N fragments are randomly selected from the list as mounting texts, which are used as reference information in long text writing; previous content: all the main text content before the current element unit (ie, the target element unit); current outline: the title and abstract content of the current element unit. <output>Part: the content field content of the current element unit. By inputting the fourth prompt word into the text generation model, the text content corresponding to the target element unit can be predicted based on the text generation model to obtain a prediction result, which is recorded as the fourth prediction result in the embodiment of the present application. Based on the fourth prediction result and the text content corresponding to the target element unit, a fourth loss value can be determined, so that the fourth loss value can be used to update the parameters of the text generation model.

[0226] The process of constructing the data of Task 4 includes the following three modules, namely, the mounting module, the addition, deletion and modification module, and the post-filtering module: Mounting module: In the actual production environment, users may choose to mount text as a reference for writing, so it is necessary to add a recall field in the process of constructing data to improve the model's ability to refer to the recall fragment writing. Introducing the recall fragment includes three steps: a. Recall model: Construct a recall model, sort the content field of each element unit, and take the top k recall results as the recall content. b. GPT generation: Introduce the large model capability, and generate a specified number of expanded fragments as GPT recall content based on the summary field content of the element unit. c. Random selection: Summarize the recall content obtained in the above steps into a recall result list, and randomly select a specified number of recall fragments from it as the mounting field of the final data. Addition, deletion and modification module: The purpose is the same as the previous embodiment, and the original training data with the transformed paragraph order is constructed for data enhancement, which is divided into three subtasks: randomly delete the first-level title, randomly delete the second-level title, and randomly swap the order of the second-level title. The training data of Task 4 is constructed based on the processed original training data. Post-filtering module: In order to ensure the quality of the output content, it is necessary to filter the data with too few output words.

[0227] For example, the training data of task 4 is as follows:

[0228] {

[0229] "input": "<_user>Generate the corresponding text paragraph content based on the above generated content, text outline and the reference fragment information corresponding to the current outline. Requirements: the context is coherent, the generated content contains certain reference fragment information, and is consistent with the outline.

[0230] Text title: {Title}

[0231] Reference recall snippet:

[0232] Snippet 1: {Recall content 1}

[0233] Snippet 2: {Recall content 2}

[0234] Previous content: {Previous text content}

[0235] Outline content: ##{title} Abstract: {Abstract content}",

[0236] "output": "{content of this paragraph}"

[0237] }

[0238] Reference Figure 8 In an embodiment of the present application, a text generation method is also provided, comprising:

[0239] Obtaining a target prompt word input by a user; the target prompt word is used to instruct the text generation model to perform any one of the tasks of generating text based on a title, generating an outline based on a title, generating text based on an outline, and generating text paragraph by paragraph based on an outline and mounted content;

[0240] The target prompt word is input into the text generation model trained by the aforementioned text generation model training method, and the corresponding target text is generated by the text generation model.

[0241] It can be understood that in the embodiments of the present application, different training data are constructed for different tasks to train the text generation model, which can provide more systematic and diversified training for the model in the field of long-text writing, and cultivate the model's ability in long-text writing, including the ability to generate outlines based on titles, generate articles based on outlines, and generate articles paragraph by paragraph based on outlines and mounted article content, effectively improving the quality and efficiency of long-text writing.

[0242] Reference Fig. 9 In an embodiment of the present application, a training system for a text generation model is also provided, comprising:

[0243] The acquisition unit 910 is used to acquire a batch of sample text data, split the sample text data, and obtain a sequence of element units corresponding to each of the sample text data; wherein the sequence of element units includes a plurality of element units, each of which includes identification information, title text, title type, and text content;

[0244] The processing unit 920 is used to obtain training data according to the sample text data and the element unit sequence; wherein each piece of the training data includes an identification field, a framework field, a summary field and a data field, the framework field includes the original title, introduction, titles of each level and summary information of each level of the sample text data; the summary field includes the outline information of the sample text data; the data field includes a type field, a title field, a content field, a summary field and an element identification field;

[0245] The updating unit 930 is used to train the text generation model through parallel multiple training tasks according to the training data to obtain a trained text generation model; wherein the multiple training tasks include a task of generating text based on the title, a task of generating an outline based on the title, a task of generating text based on the outline, and a task of generating text paragraph by paragraph based on the outline and the mounted content.

[0246] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0247] Reference Fig.10 , an embodiment of the present application provides an electronic device, including:

[0248] at least one processor 1010;

[0249] At least one memory 1020, used to store at least one program;

[0250] When at least one program is executed by at least one processor 1010, at least one processor 1010 implements the above-mentioned text generation model training method or text generation method.

[0251] Similarly, the contents of the above method embodiments are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0252] An embodiment of the present application also provides a computer-readable storage medium, which stores a program executable by the processor 1010. When the program executable by the processor 1010 is executed by the processor 1010, it is used to execute the above-mentioned text generation model training method or text generation method.

[0253] Similarly, the contents of the above method embodiments are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0254] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the application is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part described as a larger operation is performed independently.

[0255] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present application. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional techniques of the engineer. Therefore, those skilled in the art can implement the present application set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the attached claims and their equivalents.

[0256] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0257] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, apparatus or device and execute the instructions), or in conjunction with such instruction execution system, apparatus or device. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, apparatus or device, or in conjunction with such instruction execution system, apparatus or device.

[0258] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0259] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0260] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0261] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0262] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Technical personnel familiar with the field can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.< / output> < / output> < / output> < / output> < / output> < / output> < / output> < / output>

Claims

1. A training method for a text generation model, characterized in that: The method comprises: Acquire a batch of sample text data, split the sample text data, and obtain a sequence of element units corresponding to each of the sample text data; wherein the sequence of element units includes a plurality of element units, each of which includes identification information, title text, title type, and text content; According to the sample text data and the element unit sequence, training data is obtained; wherein each piece of the training data includes an identification field, a framework field, a summary field and a data field, the framework field includes the original title, introduction, titles of each level and summary information of each level of the sample text data; the summary field includes the outline information of the sample text data; the data field includes a type field, a title field, a content field, a summary field and an element identification field; According to the training data, the text generation model is trained through parallel multi-training tasks to obtain a trained text generation model; wherein the multi-training tasks include a task of generating text based on a title, a task of generating an outline based on a title, a task of generating text based on an outline, and a task of generating text paragraph by paragraph based on an outline and mounted content.

2. The training method of a text generation model according to claim 1, characterized in that: The task of generating text based on the title includes the following steps: Constructing a first prompt word according to the original title; the first prompt word is used to indicate generating text content according to the original title; Inputting the first prompt word into the text generation model, predicting the main text data of the sample text data by the text generation model, and obtaining a first prediction result; Determining a first loss value according to the text data of the sample text data and the first prediction result; The text generation model is trained according to the first loss value.

3. The training method of a text generation model according to claim 1, characterized in that: The task of generating an outline based on the title includes the following steps: Acquire a second prompt word; the second prompt word is used to indicate that text content that meets predetermined hierarchical constraint information is generated according to the original title; Inputting the second prompt word into the text generation model, predicting the framework field of the sample text data by the text generation model, and obtaining a second prediction result; Determining a second loss value according to the framework field of the sample text data and the second prediction result; The text generation model is trained according to the second loss value.

4. The training method of a text generation model according to claim 1, characterized in that: The task of generating text based on the outline includes the following steps: Constructing a third prompt word according to the outline information; the third prompt word is used to indicate generating text content according to the outline information; Inputting the third prompt word into the text generation model, predicting the text data of the sample text data by the text generation model, and obtaining a third prediction result; Determining a third loss value according to the text data of the sample text data and the third prediction result; The text generation model is trained according to the third loss value.

5. The training method of a text generation model according to claim 1, characterized in that: The task of generating text paragraph by paragraph based on the outline and the mounted content includes the following steps: Constructing a fourth prompt word; the fourth prompt word is used to indicate that the text content corresponding to the target element unit is generated according to the original title, the mounting text, the previous content of the target element unit, and the title field and the abstract field of the target element unit; Inputting the fourth prompt word into the text generation model, predicting the text content corresponding to the target element unit by the text generation model, and obtaining a fourth prediction result; Determining a fourth loss value according to the text content corresponding to the target element unit and the fourth prediction result; The text generation model is trained according to the fourth loss value.

6. The training method of a text generation model according to claim 1, characterized in that: The method further comprises: Get a pre-trained large language model; The summary field is generated according to the content field by using the large language model.

7. A text generation method, characterized in that: The method comprises: Obtaining a target prompt word input by a user; the target prompt word is used to instruct the text generation model to perform any one of the tasks of generating text based on a title, generating an outline based on a title, generating text based on an outline, and generating text paragraph by paragraph based on an outline and mounted content; The target prompt word is input into a text generation model trained by the text generation model training method according to any one of claims 1 to 6, and a corresponding target text is generated by the text generation model.

8. A training system for a text generation model, characterized in that: The system comprises: An acquisition unit is used to acquire a batch of sample text data, split the sample text data, and obtain a sequence of element units corresponding to each of the sample text data; wherein the sequence of element units includes a plurality of element units, each of which includes identification information, title text, title type, and text content; A processing unit, configured to obtain training data according to the sample text data and the element unit sequence; wherein each piece of the training data includes an identification field, a framework field, a summary field and a data field, wherein the framework field includes the original title, introduction, titles of each level and summary information of each level of the sample text data; the summary field includes outline information of the sample text data; and the data field includes a type field, a title field, a content field, a summary field and an element identification field; An updating unit is used to train the text generation model through parallel multiple training tasks according to the training data to obtain a trained text generation model; wherein the multiple training tasks include a task of generating text based on a title, a task of generating an outline based on a title, a task of generating text based on an outline, and a task of generating text paragraph by paragraph based on an outline and mounted content.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to implement a method according to any one of claims 1 to 7 when executed by the processor.

Citation Information

Cited By

  • Text data generation method and device, storage medium and program product

    CN121167323A