Method for training text generation model and text generation method
By building training corpus and using a Transformer model with a copy mechanism, the diversity and stability of text generation in the existing text automatic generation method is solved, and efficient and smooth text generation effect is achieved.
Patent Information
- Application Number
- CN202210050521.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-17
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-01-17
AI Technical Summary
In the existing automatic text generation methods, the text generated by the template slot filling method is rigid and lacks diversity, while the deep learning model requires a large number of training samples, has a long training time and is poor in stability, which cannot guarantee the smoothness and correctness of the generated text.
By collecting multiple texts describing objects, generating the first and second tags as training data sets, selecting the same tags and sentences for random sorting, building training corpus, and using the Transformer model with a copying mechanism to train the text generation model to generate new describing object text.
It realizes the sequential selection and combination of new and smooth paragraphs from multiple sentences, solving the diversity and stability of text generation and reducing training data and time requirements.
Smart Images

Figure CN114492384B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer network technology, and in particular to a text automatic generation solution. Background Art
[0002] Automatic text generation is one of the core issues in the field of natural language processing. Being able to automatically generate articles with specified content can greatly reduce the workload of authors. However, how to generate articles with smooth text and correct details is challenging.
[0003] At present, there are two main ways to achieve automatic text generation. One is the template-filling method, which is to pre-set a batch of article templates, and there are content slots to be filled in the templates. The templates are converted into text by filling the slots, thereby achieving automatic text generation. The text generated by the template is relatively rigid and prone to stereotypes, especially when there are few templates and a large number of texts to be generated.
[0004] The other is a generative text generation method. A common example is a deep learning model. By inputting some "information" into the model, the "information" can be encoded into a vector, and then the vector is decoded into text. Therefore, the model needs to learn how to encode and decode "information". The method of automatically generating text through deep learning can generate text with high diversity. However, this method has the following disadvantages: a large number of (hundreds of thousands+) training samples are required to use deep learning technology to learn the rules of writing; at the same time, the training time of the deep learning model is long, and the requirements for hardware such as GPU are high; in addition, this method is not very stable, and the form of the generated text cannot be predicted, so it cannot ensure whether the generated text is smooth and correct.
[0005] Therefore, there is an urgent need for a text generation solution to solve the above problems. Summary of the invention
[0006] The present disclosure provides a method for training a text generation model and a text generation method, in an effort to solve or at least alleviate at least one of the above problems.
[0007] According to one aspect of the present disclosure, a method for training a text generation model is provided, comprising the steps of: collecting a plurality of texts describing an object, and generating a first label and a second label corresponding to each paragraph in the text as a training data set; selecting a first number of paragraphs having the same first label and second label from the training data set; selecting a second number of sentences from the first number of paragraphs, and randomly sorting the selected sentences as a first subset; selecting a third number of sentences from the first number of paragraphs as a second subset; and training an initial text generation model using the first subset and the second subset to obtain a trained text generation model for generating new text describing an object.
[0008] According to another aspect of the present disclosure, a text generation method is provided, comprising the steps of: classifying a plurality of descriptive texts for an object; selecting some sentences from a plurality of paragraphs belonging to the same category; randomly sorting the selected sentences to obtain disordered sentences; and inputting the disordered sentences into a text generation model to generate new text about the object.
[0009] According to another aspect of the present disclosure, there is provided a device for training a text generation model, comprising: a training data processing module, adapted to collect a plurality of descriptive texts for an object, and generate a first label and a second label corresponding to each paragraph in the text as a training data set; a training data generation module, adapted to select a first number of paragraphs having the same first label and second label from the training data set, and select a second number of sentences from the first number of paragraphs, and randomly sort the selected sentences as a first subset; and select a third number of sentences from the first number of paragraphs as a second subset; and a model training module, adapted to train an initial text generation model using the first subset and the second subset to obtain a trained text generation model.
[0010] According to another aspect of the present disclosure, a text generation device is provided, comprising: a first text processing module, adapted to classify acquired descriptive texts for an object; a second text processing module, adapted to select some sentences from a plurality of paragraphs belonging to the same category, and further adapted to randomly sort the selected sentences to obtain disordered sentences; and a model prediction module, adapted to input the disordered sentences into a text generation model to generate new text about the object.
[0011] According to another aspect of the present disclosure, a computing device is provided, comprising: one or more processor memories; one or more programs, wherein the one or more programs are stored in the memories and configured to be executed by one or more processors, and the one or more programs include instructions for executing any of the above methods.
[0012] According to another aspect of the present disclosure, a computer-readable storage medium storing one or more programs is provided. The one or more programs include instructions. When the instructions are executed by a computing device, the computing device performs any of the methods described above.
[0013] In summary, according to the scheme disclosed in the present invention, by splitting and processing a number of paragraphs of the same dimension describing the same object category in units of sentences, a first subset and a second subset are constructed as training expectations. Based on the constructed training corpus, the text generation model is trained so that the trained text generation model can sequentially select n sentences (N>n) from N sentences and combine these n sentences into a new and coherent paragraph. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] To achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The above and other purposes, features and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout the present disclosure, the same reference numerals generally refer to the same parts or elements.
[0015] Figure 1 A schematic diagram of a computing device 100 according to some embodiments of the present disclosure is shown;
[0016] Figure 2 A flowchart of a method 200 for training a text generation model according to some embodiments of the present disclosure is shown;
[0017] Figure 3 A flowchart of a text generation method 300 according to some embodiments of the present disclosure is shown;
[0018] Figure 4 A schematic diagram showing an apparatus 400 for training a text generation model according to some embodiments of the present disclosure; and
[0019] Figure 5 A schematic diagram of a text generating device 500 according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0020] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0021] The text automatic generation scheme based on deep learning provided by the embodiment of the present disclosure is mainly used for the automatic generation of descriptive text of a certain object. First, the historical text describing the object is collected and processed as a data set. After that, for the paragraphs belonging to the same category in the data set, they are split into sentences as units, and a certain number of sentences are selected from them. The selected sentences are input into the trained text generation model to obtain a new text paragraph describing the object. The text paragraph also achieves good fluency in writing.
[0022] Furthermore, this solution classifies each paragraph in the text by analyzing the category of the object described in each paragraph and the dimension of the description.
[0023] The automatic text generation solution and the solution for training a text generation model of the embodiments of the present disclosure may be executed in one or more computing devices. Figure 1 is a block diagram of an example computing device 100 .
[0024] In a basic configuration 102, computing device 100 typically includes a system memory 106 and one or more processors 104. A memory bus 108 may be used for communication between processor 104 and system memory 106.
[0025] Depending on the desired configuration, the processor 104 can be any type of processor, including but not limited to: a microprocessor (μP), a microcontroller (μC), a digital information processor (DSP), or any combination thereof. The processor 104 can include one or more levels of cache such as a first-level cache 110 and a second-level cache 112, a processor core 114, and registers 116. An example processor core 114 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 118 can be used with the processor 104, or in some implementations, the memory controller 118 can be an internal part of the processor 104.
[0026] Depending on the desired configuration, the system memory 106 can be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. The system memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some embodiments, the application 122 can be arranged to operate on the operating system using the program data 124. In some embodiments, the computing device 100 is configured to execute the method 200 and the text generation method 300 for training a text generation model, and the program data 124 includes instructions for executing the above methods.
[0027] The computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to the basic configuration 102 via the bus / interface controller 130. Example output devices 142 include a graphics processing unit 148 and an audio processing unit 150. They can be configured to facilitate communication with various external devices such as a display or speakers via one or more A / V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices) or other peripherals (e.g., printers, scanners, etc.) via one or more I / O ports 158. Example communication devices 146 may include a network controller 160, which may be arranged to facilitate communication with one or more other computing devices 162 via a network communication link via one or more communication ports 164.
[0028] A network communication link can be an example of a communication medium. Communication media can generally be embodied as computer-readable instructions, data structures, program modules in modulated data signals such as carrier waves or other transmission mechanisms, and can include any information delivery medium. A "modulated data signal" can be a signal in which one or more of its data sets or its changes can be performed in a manner that encodes information in the signal. As a non-limiting example, communication media can include wired media such as wired networks or dedicated line networks, and various wireless media such as sound, radio frequency (RF), microwaves, infrared (IR) or other wireless media. The term computer-readable medium used here can include both storage media and communication media. In some embodiments, one or more programs are stored in a computer-readable medium, and the one or more programs include instructions for executing certain methods.
[0029] The computing device 100 may be implemented as part of a small-sized portable (or mobile) electronic device, such as a cellular phone, a personal digital assistant (PDA), a personal media player device, a wireless network browsing device, a personal head-mounted device, an application-specific device, or a hybrid device that may include any of the above functions. The computing device 100 may also be implemented as a personal computer including a desktop computer and a notebook computer configuration. The computing device 100 may also be implemented as a server having the above configuration.
[0030] For ease of explanation, the following takes a car as an example to illustrate the method of training and generating a text generation model and the text generation method according to the embodiment of the present disclosure. It should be understood that based on the content disclosed in the embodiment of the present disclosure, those skilled in the art can apply it to other objects (such as smart homes, clothing, books, etc.) to automatically generate text for other objects, all of which are within the scope of protection of the present disclosure.
[0031] Figure 2 A flow chart of a method 200 for training a text generation model according to some embodiments of the present disclosure is shown. The text generation model can generate new text describing an object based on several sentences describing an object. Figure 2 As shown, the method 200 starts at step S210 .
[0032] In step S210, a plurality of texts describing objects are collected, and a first label and a second label corresponding to each paragraph in the text are generated as a training data set.
[0033] According to one embodiment, historical articles about cars on the car recommendation platform are collected, for example, articles introducing cars from one year to ten years ago are collected as texts describing cars.
[0034] Afterwards, a deep learning classification model is used to understand the content of the article, and the first label and the second label corresponding to each paragraph in the collected article are generated respectively. According to an embodiment of the present disclosure, the first label indicates the category of the object described in the paragraph, and the second label indicates the description dimension of the paragraph. In one embodiment, when the object being described is a car, the first label is the car series, and the dimension of the second label includes at least one of the six dimensions: appearance, power, interior, handling, comfort, and configuration. For example: If a paragraph describes the appearance of the BMW 3 Series, the first label of the paragraph is "BMW 3 Series" and the second label is "Appearance". It should be noted that the present disclosure does not limit the specific deep learning model used to process the input data to output the corresponding first label and second label.
[0035] In addition, if a collected paragraph does not contain the above tags, the paragraph will be discarded.
[0036] Finally, all the paragraphs describing the car and their corresponding first and second labels are used as a training dataset. The purpose of constructing this training dataset is to process and obtain the training corpus for the text generation model.
[0037] Then, in step S220, a first number of paragraphs having the same first label and second label are selected from the training data set.
[0038] In one embodiment, a first number of paragraphs are randomly selected from paragraphs of the same vehicle series and the same dimension. Let the first number be M, which are recorded as the first paragraph, the second paragraph, ... the Mth paragraph, respectively, wherein each paragraph contains a number of sentences, and these M paragraphs have a total of N sentences. It should be understood that there is a logical relationship between the sentences contained in a paragraph, and they constitute an orderly and smooth paragraph (also called text).
[0039] Then, in step S230, a second number of sentences are selected from the first number of paragraphs, and the selected sentences are randomly sorted as a first subset.
[0040] According to an embodiment of the present disclosure, the first subset is generated in the following manner.
[0041] First, the first number of paragraphs are divided into a first part and a second part. In one embodiment, the paragraphs other than the last paragraph of the M paragraphs (i.e., paragraphs 1 to M-1) are taken as the first part, and the last paragraph of the M paragraphs (i.e., the Mth paragraph) is taken as the second part.
[0042] It should be noted that these M paragraphs are randomly selected from several articles. They may come from different articles and have no order. They are only numbered to distinguish them. The present disclosure does not limit the order of the paragraphs. In other words, any paragraph can be the last paragraph, that is, the second part.
[0043] Next, two sentences are selected from each paragraph in the first part, and combined with all the sentences in the second part, as the second number of sentences selected. In one embodiment, two sentences are randomly selected from each paragraph from the 1st to the M-1st paragraphs, for a total of 2*(M-1) sentences; and all the sentences in the Mth paragraph (the number of sentences is denoted as m) are added as the second number of sentences selected, so the second number is 2*(M-1)+m.
[0044] Afterwards, the selected second number of sentences are randomly sorted as the first subset. In one embodiment, the sentences are broken up into a disordered set of sentences as the first subset.
[0045] It should be understood that although it is shown here that 2 sentences are randomly selected from each paragraph to generate the first subset, the present disclosure is not limited thereto. Those skilled in the art may also randomly select 3 sentences, 4 sentences, etc. from each paragraph to construct a corpus for training a text generation model.
[0046] Then, in step S240, a third number of sentences are selected from the first number of paragraphs as a second subset.
[0047] In one embodiment, all the sentences in the last paragraph of the M paragraphs, that is, all the sentences in the second part in the above step, are taken as the second subset.
[0048] In summary, the first subset contains 2*(M-1)+m sentences, and the sentences are unordered; the second subset contains m sentences, and the sentences are ordered. The first and second subsets are the training corpus for the text generation model.
[0049] Then, in step S250, the initial text generation model is trained using the first subset and the second subset to obtain a trained text generation model.
[0050] According to an embodiment of the present disclosure, method 200 further includes the step of constructing an initial text generation model and its network parameters.
[0051] In one embodiment, the text generation model is a Transformer model with a copy mechanism, including an encoder module and a decoder module. In short, the text generation model is different from the general Transformer model in that, when decoding, it not only depends on the generation probability, but also depends on the sum of the generation probability and the attention weight of the encoding. The copy mechanism has been applied to various NLP models, such as the Transformer model, so this disclosure will not go into details.
[0052] After constructing the initial text generation model, the first subset is used as input data and input into the text generation model for training to obtain output data; the second subset is used as target data, and the loss function is calculated with the output data, and the network parameters of the text generation model are adjusted according to the loss function. The above steps are repeated and iterated until the loss function meets the conditions (of course, the conditions may also be that the number of trainings reaches a certain number, or the loss function gradually converges, etc.), then the training ends and a trained text generation model is obtained.
[0053] According to the method 200 of the present disclosure, by splitting and processing a number of paragraphs describing the same dimensions of the same car series (appearance, power, interior, etc.) in units of sentences, a first subset and a second subset are constructed as training expectations. Based on the constructed training corpus, the text generation model is trained so that the trained text generation model can sequentially select n sentences (N>n) from N sentences and combine these n sentences into a new and coherent paragraph. That is: maxP(x k ,x i ,x j ,..,x t |x1,x2,...,x N ), where x k ,x i ,x j ,..,x t There are n sentences in total, existing in x1,x2,...,x N and the order may be x1,x2,...,x N Inconsistent, as long as x is satisfied k ,x i ,x j ,..,x t These n sentences just need to be fluent.
[0054] Figure 3 FIG. 3 is a flow chart of a text generation method 300 according to some embodiments of the present disclosure. Figure 3 As shown, the method 300 starts at step S310 .
[0055] In step S310, a plurality of description texts for objects are classified.
[0056] According to an embodiment, a plurality of description texts for an object are first obtained, for example, historical articles about cars on a car recommendation platform are collected, for example, articles introducing cars in the past six months are collected.
[0057] Afterwards, the first label and the second label corresponding to each paragraph in the acquired text are determined. As mentioned above, the first label indicates the category of the object described by the paragraph, and the second label indicates the description dimension of the paragraph, which will not be repeated here.
[0058] Afterwards, the paragraphs are classified based on the first label and the second label. Optionally, the paragraphs with the same first label and the same second label are classified as the same category. In other words, the paragraphs describing the same dimension of the same car series are classified as one category.
[0059] Then, in step S320, some sentences are selected from multiple paragraphs belonging to the same category.
[0060] According to an embodiment of the present disclosure, some paragraphs are randomly selected from paragraphs belonging to the same category, and 2 sentences are respectively selected from the selected paragraphs.
[0061] In one embodiment, K paragraphs are randomly selected from the paragraphs belonging to the same category. The embodiment of the present disclosure does not limit the value of K. Generally speaking, K is at least greater than 2.
[0062] It should be understood that here, 2 sentences are randomly selected from each paragraph, which corresponds to the method 200, in which 2 sentences are randomly selected from each paragraph when constructing the first subset. In other words, if 3 sentences are selected from each paragraph when constructing the first subset, then here, 3 sentences can also be selected from paragraphs belonging to the same category. And so on.
[0063] Then, in step S330, the selected sentences are randomly sorted to obtain disordered sentences.
[0064] According to an embodiment of the present disclosure, the selected 2K sentences are randomly sorted in units of sentences so that they no longer maintain the order in the original paragraph.
[0065] Then, in step S340, the disordered sentences are input into a text generation model to generate new text about the object.
[0066] The text generation model is generated by executing method 200. According to an embodiment, 2K disordered sentences are input into the text generation model for processing, and the text generation model can select several sentences from the 2K sentences and combine them into orderly and smooth paragraphs.
[0067] The following is an example of automatically generated text.
[0068] Suppose that descriptive texts about the "appearance" of the "Audi A4L" are collected and 5 paragraphs are randomly selected from them, as shown below.
[0069] In terms of appearance, the new Audi A4L basically continues the design style of the new overseas Audi A4, with a hexagonal integrated air intake grille that is flatter, which reduces the visual effect of the whole vehicle. The domestic S-line model will be equipped with a titanium black honeycomb air intake grille with the same visual effect as the RS, and aluminum trim strips will be used on the air intake shape to create an aggressive and dynamic appearance. The new Audi A4L is equipped with LED digital daytime running lights and segmented LED taillights as standard. The top-of-the-line model supports matrix intelligent LED technology, which can illuminate the road more accurately. The LED lights at the rear not only provide dynamic turn signals, but also display dynamic effects, which are highly recognizable. Echoing the front headlight group, it is more eye-catching when lit at night, attracting attention all day long.
[0070] The most obvious difference at the rear is the taillight shape, which continues the design of the overseas version of the A4. When lit, it looks more technological. With the S-Line kit, the lower surround also looks more radical. The exhaust part is also decorated, with a rectangular flat-mouthed decorative cover and rear bumper integrated design, making it look a bit like a small A6.
[0071] In terms of appearance, the car basically continues the design style of the current model. The front part adopts a hexagonal integrated air intake grille and flattens it, and the sharp headlights on both sides enhance the visual extension of the front. The sharp waistline on the side of the body runs through the whole car, and the five-spoke wheels create a better sporty posture for the body. The ducktail design on the top of the trunk, coupled with the double-sided exhaust design below, further enhances the sporty attributes of the car. The taillight group adopts a split design, and the overall shape echoes the headlight group. It is highly recognizable at night when lit.
[0072] In terms of appearance, the A4L has a more prominent sporty style. The entire series is integrated with RS sports style and Sline kits. This appearance, which is considered to be unique to high-end models, can be seen in all models this time. Compared with the BMW 3 Series, which only has sports components in mid-to-high-end models, the A4L is much more conscientious this time. But this set of sports components adds a lot of points, and it doesn't look like a car that costs just over 300,000 yuan. The front face is still the Audi family's nesting doll design. We have seen a lot of hexagonal large-mouth air intake grilles, and it is still so beautiful. Look carefully at the design of the air intake grille, which adopts a black honeycomb shape, and the details are still very beautiful.
[0073] The side of the car body adopts a three-stage waistline design, which makes the new car's muscularity well highlighted. At the same time, the new car is also equipped with a new style of wheels. In terms of body size, the length, width and height of the new Audi A4L are 4851mm (elegant type) / 4858mm (dynamic type), 1847mm, 1439mm (comfort suspension) / 1411mm (sports suspension), and the wheelbase is 2908mm. Compared with the current model, the length has increased by 14mm / 21mm, and the width has increased by 4mm.
[0074] Select 2 sentences from each of the 5 paragraphs above, for a total of 10 sentences. As shown below, enclose the 2 sentences in one paragraph in curly braces "{}".
[0075] {The LED lights at the rear not only provide dynamic turn signals, but also display dynamic effects, making them highly recognizable. They echo the front headlights and are more eye-catching when lit at night, attracting attention all day long.}
[0076] {The most obvious difference at the rear is the taillight shape, which continues the design of the overseas version of the A4. When lit, it has a more technological feel. With the support of the S-Line kit, the lower surround also looks more radical. The exhaust part is also decorated, and the rectangular flat-mouthed decorative cover and the rear bumper are integrated, making it look a bit like a small A6.}
[0077] {The sharp waistline on the side of the car runs through the whole car, and the five-spoke wheels create a good sporty posture for the car. The taillight group adopts a split design, and the overall shape echoes the headlight group. When lit, it is highly recognizable at night.}
[0078] {In terms of appearance, the A4L has a more prominent sporty style. The entire series is integrated with RS sports style and Sline kit. This appearance, which is considered to be unique to high-end models, can be seen in all models this time. Look carefully at the design of the air intake grille, which adopts a black honeycomb shape. The details are still very beautiful.}
[0079] {The side of the car body adopts a three-stage waistline design, which makes the new car's muscularity well highlighted. At the same time, the new car is also equipped with a new style of wheels. Compared with the current model, the length is increased by 14mm / 21mm and the width is increased by 4mm.}
[0080] After that, the above 10 sentences are randomly sorted and input into the text generation model. After processing, a new paragraph is output, which is a new description text about the "appearance" of the "Audi A4L". As shown below.
[0081] In terms of appearance, the A4L has a more prominent sporty style. The entire series is integrated with RS sports style and Sline kits. This appearance, which is considered to be unique to high-end models, can be seen in all models this time. The side of the body adopts a three-stage waistline design, which makes the muscularity of the new car very well highlighted. At the same time, the new car is also equipped with a new style of wheels. The most obvious difference at the rear is the taillight shape, which still continues the design of the overseas version of A4. It has a more technological feel when lit. With the blessing of the S-Line kit, the lower surround also looks more radical.
[0082] According to the present disclosure, a method for quickly and efficiently automatically generating text of a specified car series and a specified dimension is provided. First, based on method 200, a training corpus is constructed by breaking up the historical text data in units of sentences and then reconstructing them; at the same time, a Transformer model with a copy mechanism is built, and the training corpus is input for model training. Afterwards, a generated corpus is constructed by breaking up recent (e.g., the past 6 months) text data in units of sentences; and the generated corpus is input into a trained text generation model, and the broken corpus is reconstructed into a new text after processing.
[0083] In summary, this solution has the characteristics of quickly and massively generating correct and reliable texts of specified car series and specified dimensions. In the automatic writing system, it has portability and scalability.
[0084] Figure 4 A schematic diagram of an apparatus 400 for training a text generation model according to some embodiments of the present disclosure is shown. Figure 4 and Figure 2 The method 200 shown in the figure complements each other, so for the content of the device 400, please refer to the relevant description above, and no further expansion will be made here.
[0085] The apparatus 400 for training and generating a text generation model includes: a training data processing module 410 , a training data generation module 420 and a model training module 430 .
[0086] The training data processing module 410 collects a plurality of description texts for an object, and generates a first label and a second label corresponding to each paragraph in the text as a training data set.
[0087] The training data generation module 420 selects a first number of paragraphs having the same first label and second label from the training data set, selects a second number of sentences from the first number of paragraphs, and randomly sorts the selected sentences as a first subset. At the same time, the training data generation module 420 also selects a third number of sentences from the first number of paragraphs as a second subset.
[0088] The model training module 430 uses the first subset and the second subset to train the initial text generation model to obtain a trained text generation model.
[0089] Figure 5 A schematic diagram of a text generating device 500 according to some embodiments of the present disclosure is shown. Figure 4 and Figure 3 The method 300 shown in the figure complements each other, so for the content of the device 500, please refer to the relevant description above, and no further expansion will be made here.
[0090] like Figure 5 As shown, the text generation device 500 includes: a first text processing module 510, a second text processing module 520 and a model prediction module 530.
[0091] The first text processing module 510 classifies the acquired description text for the object.
[0092] The second text processing module 520 selects some sentences from multiple paragraphs belonging to the same category, and randomly sorts the selected sentences to obtain disordered sentences.
[0093] The model prediction module 530 inputs the disordered sentences into the text generation model to generate new text about the object. The text generation model is trained and generated by the device 400.
[0094] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0095] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various disclosed aspects, in the above description of the exemplary embodiments of the present disclosure, the various features of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: the claimed disclosure requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, the disclosed aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present disclosure.
[0096] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.
[0097] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.
[0098] Also disclosed here:
[0099] A3. A method as described in A1 or 2, wherein the step of selecting a second number of sentences from the first number of paragraphs and randomly sorting the selected sentences as the first subset includes: dividing the first number of paragraphs into a first part and a second part; selecting 2 sentences from each paragraph included in the first part respectively, and combining them with all the sentences in the second part as the second number of sentences selected; and randomly sorting the second number of sentences selected as the first subset.
[0100] A4. A method as described in A3, wherein the step of dividing the first number of paragraphs into a first part and a second part comprises: taking the other paragraphs in the first number of paragraphs except the last paragraph as the first part; and taking the last paragraph in the first number of paragraphs as the second part.
[0101] A5. A method as described in any one of A1-4, wherein the step of selecting a third number of sentences from the first number of paragraphs as the second subset includes: selecting all sentences in the last paragraph of the first number of paragraphs as the second subset.
[0102] A6. A method as described in any one of A1-5, wherein the step of using the first subset and the second subset to train the text generation model to obtain the trained text generation model includes: using the first subset as input data, inputting the text generation model for training, and obtaining output data; using the second subset as target data, and calculating a loss function with the output data, until the loss function satisfies a condition, the training is terminated, and the trained text generation model is obtained.
[0103] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of this disclosure and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.
[0104] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices that are used to implement the functions performed by the elements for the purpose of implementing the disclosure.
[0105] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved, and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.
[0106] Although the present disclosure has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the present disclosure thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and didactic purposes, rather than for the purpose of explaining or limiting the subject matter of the present disclosure. Therefore, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The disclosure made with respect to the present disclosure is illustrative, not restrictive, with respect to the scope of the present disclosure, and the scope of the present disclosure is defined by the appended claims.
Claims
1. A method for training a text generation model, comprising the steps of: Collect multiple texts describing objects, and generate first labels and second labels corresponding to each paragraph in the text as training data sets; Selecting a first number of paragraphs having the same first label and second label from the training data set; Selecting a second number of sentences from the first number of paragraphs, and randomly sorting the selected sentences as a first subset, including: dividing the first number of paragraphs into a first part and a second part; selecting 2 sentences from each paragraph included in the first part, and combining all the sentences in the second part as the selected second number of sentences; randomly sorting the selected second number of sentences as a first subset, wherein the other paragraphs except the last paragraph in the first number of paragraphs are taken as the first part, and the last paragraph in the first number of paragraphs is taken as the second part; Selecting a third number of sentences from the first number of paragraphs as a second subset includes: selecting all sentences in the last paragraph of the first number of paragraphs as the second subset; and The initial text generation model is trained using the first subset and the second subset to obtain a trained text generation model, which is used to generate new text describing an object. The steps of training the text generation model using the first subset and the second subset to obtain the trained text generation model include: using the first subset as input data and inputting it into the text generation model for training to obtain output data; using the second subset as target data and calculating a loss function with the output data until the loss function meets a condition, the training is terminated, and the trained text generation model is obtained.
2. The method of claim 1, wherein: The first label indicates the category of the object described in the paragraph, and the second label indicates the description dimension of the paragraph.
3. A text generation method, comprising the steps of: Classify multiple description texts for objects; Select some sentences from multiple paragraphs belonging to the same category; Randomly sort the selected sentences to obtain disordered sentences; as well as The disordered sentences are input into a text generation model to generate new text about the object, wherein the text generation model is generated by executing the method according to claim 1 or 2.
4. The method of claim 3, wherein: The step of classifying a plurality of description texts for objects comprises: Get multiple description texts for objects; Determine a first label and a second label corresponding to each paragraph in the acquired text, wherein the first label indicates the category of the object described in the paragraph, and the second label indicates the description dimension of the paragraph; The paragraph is classified based on the first label and the second label.
5. The method according to claim 3 or 4, wherein: The step of selecting some sentences from multiple paragraphs belonging to the same category includes: The paragraphs having the same first label and the same second label are regarded as the same category; Select some paragraphs from the paragraphs belonging to the same category, and select 2 sentences from each of the selected paragraphs.
6. A device for training a text generation model, comprising: A training data processing module, adapted to collect a plurality of description texts for an object and generate a first label and a second label corresponding to each paragraph in the text as a training data set; The training data generation module is adapted to select a first number of paragraphs having the same first label and second label from the training data set, and select a second number of sentences from the first number of paragraphs, and randomly sort the selected sentences as a first subset, including: dividing the first number of paragraphs into a first part and a second part; selecting 2 sentences from each paragraph included in the first part, respectively, and combining all the sentences in the second part as the selected second number of sentences; randomly sorting the selected second number of sentences as a first subset, wherein the other paragraphs except the last paragraph in the first number of paragraphs are taken as the first part, and the last paragraph in the first number of paragraphs is taken as the second part; and selecting a third number of sentences from the first number of paragraphs as a second subset, including: taking all the sentences in the last paragraph in the first number of paragraphs as the second subset; The model training module is suitable for using the first subset and the second subset to train the initial text generation model to obtain a trained text generation model for generating new text describing an object, wherein the steps of using the first subset and the second subset to train the text generation model to obtain the trained text generation model include: using the first subset as input data, inputting the text generation model for training, and obtaining output data; using the second subset as target data, and calculating a loss function with the output data, until the loss function meets a condition, the training ends, and the trained text generation model is obtained.
7. A text generation device, comprising: A first text processing module, adapted to classify the acquired description text for the object; The second text processing module is adapted to select some sentences from a plurality of paragraphs belonging to the same category, and is also adapted to randomly sort the selected sentences to obtain disordered sentences; A model prediction module is adapted to input the disordered sentences into a text generation model to generate new text about the object, wherein the text generation model is generated by executing the method as claimed in claim 1 or 2.
8. A computing device comprising: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1-5.
9. A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Chinese text abstract generation system and method
CN106919646A
Text abstract automatic generation method and device based on heuristic rule
CN109977219A
Text generation method and device, storage medium and electronic equipment
CN110287489A
Article generation method and device, computer equipment and storage medium
CN111046645A
Self-supervised conversation text abstracting method and system
CN111639175A
