Training Method for Multi-Product Copywriting Generation Model

Abstract

The present invention discloses a training method for a multi-product copywriting generation model, belonging to the field of natural language processing. The model training method includes: restoring damaged copywriting; generating multi-product copywriting; and polishing the multi-product copywriting. By applying the solution of the present invention, the comparison relationship between products in the generated copywriting can be controlled, and the quality of the generated copywriting can be improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and more particularly, to a method for training a multi-product copywriting generation model. Background Art

[0002] Text Generation is a basic task in the field of natural language processing, whose purpose is to generate corresponding text according to user requirements. It has a wide range of applications in tasks such as text translation and dialogue generation. With the booming development of Internet technology, we have now entered the era of information explosion. How to enable the model to automatically learn copywriting writing from a vast amount of copywriting data and liberate human labor has become particularly important. Especially in the e-commerce field, automatic copywriting generation can automatically write advertising copy according to user preferences, better build a bridge between users and high-quality products, help users purchase their favorite products, and bring users a better shopping experience.

[0003] Although existing text generation training methods can enable the model to generate text of relatively high quality, training the model to generate multi-product copywriting is still very challenging. Because each product involved in a single piece of copywriting has its inherent characteristics, it is very easy to get confused and make mistakes during generation. In addition, multi-product copywriting often involves comparisons between products, and how to control the generation of such comparison content is also an urgent research topic.

[0004] Most existing copywriting generation model training methods are designed for short copywriting that contains only a single product. However, in actual situations, it is often the case that a piece of copywriting contains multiple product introductions and descriptions of comparisons between products. In view of the characteristics of multi-product copywriting with comparisons and the actual needs of the copywriting generation task, there is an urgent need to propose a copywriting generation model training method. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for training a multi-product copywriting generation model, which can make the comparison relationship between products in the generated copywriting controllable and improve the quality of the generated copywriting.

[0006] The specific technical solution for achieving the purpose of the present invention is as follows:

[0007] A method for training a multi-product copywriting generation model, the method comprising:

[0008] Restore damaged copywriting;

[0009] Construct damaged copywriting: respectively construct the first type of damaged copywriting and the second type of damaged copywriting;

[0010] The first training task: Based on the pre-trained model, input the first type of damaged copywriting into the encoder, input the hidden state output by the encoder into the first decoder, and use backpropagation to update the parameters of the encoder and the first decoder, so that the first decoder outputs the original copywriting;

[0011] The second training task: Based on the pre-trained model, input the second type of damaged copywriting into the encoder, input the hidden state output by the encoder into the second decoder, and use backpropagation to update the parameters of the encoder and the second decoder, so that the second decoder outputs the original copywriting;

[0012] Use the first training task and the second training task for joint training, that is, a batch of data contains both the first type of damaged copywriting and the second type of damaged copywriting, and the loss updated at each step is the sum of the losses of the first training task and the second training task;

[0013] Generate multi-product copywriting;

[0014] Construct product description data: Extract product information from the original copywriting and organize it in the following format:

[0015] Copywriting generation: [Product information 1], [Product information 2], …, [Product information m]; Comparison information 1, Comparison information 2, …, Comparison information n; where m is the number of products in the copywriting, and n is the number of product comparison descriptions in the copywriting; The content described in a pair of square brackets is the information of one product;

[0016] The third training task: Input the product description data into the encoder, input the hidden state output by the encoder into the second decoder, and use backpropagation to update the parameters of the encoder and the second decoder, so that the second decoder outputs the original copywriting;

[0017] On the basis of the model jointly trained using the first training task and the second training task, use the third training task for training;

[0018] Polish multi-product copywriting;

[0019] Construct data to be polished: Extract product dimension data from the original copywriting in the training set, input the extracted data into the encoder trained by the first training task, the second training task, and the third training task for forward inference, take the output hidden state, input it into the second decoder trained by the first training task, the second training task, and the third training task for forward inference, and save the output copywriting as the data to be polished;

[0020] The fourth training task: Input the data to be polished in the format of "Copywriting polishing: Copywriting to be polished" into the encoder, input the hidden state output by the encoder into the second decoder, and use backpropagation to update the parameters of the encoder and the second decoder, so that the second decoder outputs the original copywriting;

[0021] Based on the model that has completed the third training task, the model is trained using the fourth training task.

[0022] The construction of the first type of damaged copywriting is as follows: randomly replace words in the original copywriting with the mask symbol [MASK] with a probability of 15%.

[0023] The construction of the second type of damaged copywriting is as follows: randomly replace the product information of a certain product in the original copywriting with the product information of other products with a probability of 25%.

[0024] The product information has the following format: product name: product name i, product embedding a, product dimension 1: value 1, …, product dimension k: value k;

[0025] Among them, k is the number of product dimensions contained in the current product;

[0026] The construction of the product embedding: products and product embeddings are in one-to-one correspondence, and multiple aliases of a product correspond to the same product embedding. Suppose a certain product has multiple aliases and it will be mentioned by different q aliases, and the set of names that will be mentioned is {product name 1, product name 2, …, product name q}. For the copywriting in the training set, replace the positions of any product name in the name set that appears with the mask symbol [MASK], input the replaced copywriting into the encoder trained by the first training task and the second training task for forward inference, take the hidden state output by the encoder and input it into the first decoder trained by the first training task and the second training task for forward inference, and save the output embedding corresponding to the position of the mask symbol in the input; add all the extracted embeddings and take the average as the product embedding a of the current product.

[0027] The comparison information has the following format: product name i > product name j: evaluation aspect: view.

[0028] The beneficial effects of the present invention are as follows: (1) Training multiple tasks on the same model is beneficial to the full interaction of data in the model and improves the quality of the advertising copy generated by the model. (2) Through training, the model has a polishing function, which can iteratively polish the output copywriting and improve the quality of the output copywriting. (3) By adding the comparison control information between products to the input of the training data, the product comparison relationship in the output copywriting can be controlled. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flowchart of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0030] The present invention will be further described in detail below in conjunction with the following specific embodiments and accompanying drawings. The processes, conditions, experimental methods, etc. for implementing the present invention are all common knowledge and general knowledge in the art except for the specifically mentioned content below, and the present invention has no particularly restricted content.

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings. Embodiment

[0032] The encoder and the first decoder can adopt the Transformer Encoder structure, while the second decoder can adopt the Transformer Decoder structure. The overall pre-trained model can be a CPT (Chinese Pre-trained Unbalanced Transformer) model, which conforms to the model structure described in the present invention. The training described in the present invention can be carried out on the basis of CPT.

[0033] The method proposed by the present invention can be applied to the generation of copywriting for any type of commodity.

[0034] In this embodiment, taking beauty copywriting as an example, the explanation is as follows. The content of the sample copywriting is: "The small brown bottle of Brand A has a dropper design, is easy to spread and absorb on the face! It is slightly thick and stuffy to use in summer! The small black bottle of Brand B was recommended by a colleague and has an egg white texture, is easy to spread and absorb on the face. After comparing the two serums, I prefer the small black bottle of Brand B. After all, I have oily skin. The small brown bottle is really too greasy and may easily cause clogged pores and acne. The small black bottle is very refreshing and delicate, and its repair effect is also super good!"

[0035] Refer to Figure 1 , this embodiment specifically includes:

[0036] Restore the damaged copywriting, specifically:

[0037] Construct damaged copywriting, and use the first training task and the second training task for joint training, that is, a batch of data contains both the first type of damaged copywriting and the second type of damaged copywriting, and the loss updated in each step is the sum of the losses of the first training task and the second training task.

[0038] The constructed damaged text: It includes the first type of constructed damaged text and the second type of constructed damaged text. Randomly replace the words in the original text with the mask symbol "[MASK]" with a probability of 15% as the first type of damaged text. Randomly replace the product information of a certain product in the original text with the product information of other products with a probability of 25% as the second type of damaged text. The example text after processing is: "A brand [MASK] brown bottle, it has a dropper design, easy to spread and absorb on the face! [MASK] is slightly thick, a bit [MASK]! B brand small black bottle, recommended by colleagues, has an egg white texture, easy to spread and absorb on the face. After comparing the two [MASK], I prefer the B brand small black bottle. After all, I have oily skin, and the small brown bottle is really too greasy, and it may even cause clogged pores easily. The small black bottle is very refreshing [MASK], and its repair effect is also super good!"

[0039] The first training task: Input the first type of damaged text into the encoder, input the hidden state output by the encoder into the first decoder, and use backpropagation to update the parameters of the encoder and the first decoder so that the first decoder outputs the original text;

[0040] The second training task: Input the second type of damaged text into the encoder, input the hidden state output by the encoder into the second decoder, and use backpropagation to update the parameters of the encoder and the second decoder so that the second decoder outputs the original text;

[0041] Generate multi-product text, specifically:

[0042] Construct product description data and use the third training task to train the model.

[0043] The constructed product description data: Extract product information from the original text and organize it in the following format:

[0044] Text generation: [Product information 1], [Product information 2], …, [Product information m]; Comparison information 1, Comparison information 2, … Comparison information n; For example, the product information extracted from the example text is:

[0045] Text generation: [Product name: A brand small brown bottle (e1), Texture: thick], [Product name: B brand small black bottle (e2), Texture: egg white], …, [Product information m]; B brand small black bottle > A brand small brown bottle: Overall: prefer, B brand small black bottle > A brand small brown bottle: Texture: fresh and delicate.

[0046] The format of the product information is: Product name: Product name i, Product embedding a, Product dimension 1: Value 1, …, Product dimension k: Value k; For example: Product name: A brand small brown bottle (e1), Texture: thick.

[0047] The embedding of the product: The product and its embedding are in one-to-one correspondence. Multiple aliases of a product correspond to the same product embedding. Suppose product A's Little Brown Bottle has multiple aliases and is mentioned by different q aliases. The possible set of names it may be mentioned by is {A's Little Brown Bottle, Little Brown Bottle, Est A's Little Brown Bottle}. For the copywriting in the training set, replace the positions of any product name in the set of names with the mask symbol [MASK]. Input the replaced copywriting into the encoder trained through the first training task and the second training task for forward inference. Take the hidden state output by the encoder and input it into the first decoder trained through the first training task and the second training task for forward inference. Save the output embedding corresponding to the position of the mask symbol in the input; Add up all the extracted embeddings and take the average as the product embedding e1 of the current product.

[0048] The format of the comparison information is: Product name i > Product name j: Evaluation aspect: Opinion; For example: Brand B's Little Black Bottle > Brand A's Little Brown Bottle: Overall: More favored.

[0049] The third training task: Input the product description data into the encoder, input the hidden state output by the encoder into the second decoder, and use backpropagation to update the parameters of the encoder and the second decoder to make the second decoder output the original copywriting;

[0050] Polishing of multi-product copywriting, specifically:

[0051] Construct the data to be polished and train the model using the fourth training task.

[0052] The construction of the data to be polished: Extract the product dimension data from the original copywriting in the training set, input the extracted data into the encoder trained through the first training task, the second training task, and the third training task for forward inference, take the output hidden state, input it into the second decoder trained through the first training task, the second training task, and the third training task for forward inference, and save the output copywriting as the data to be polished.

[0053] The fourth training task: Input the data to be polished in the format "Copywriting polishing: Copywriting to be polished" into the encoder, input the hidden state output by the encoder into the second decoder, and use backpropagation to update the parameters of the encoder and the second decoder to make the second decoder output the original copywriting.

[0054] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A training method for a multi-commodity copywriting generation model, characterized in that: The method includes: Restoring damaged copywriting; Constructing damaged copywriting: respectively constructing the first type of damaged copywriting and the second type of damaged copywriting; The first training task: Based on the pre-trained model, input the first type of damaged copywriting into the encoder, input the hidden state output by the encoder into the first decoder, and use backpropagation to update the parameters of the encoder and the first decoder, so that the first decoder outputs the original copywriting; The second training task: Based on the pre-trained model, input the second type of damaged copywriting into the encoder, input the hidden state output by the encoder into the second decoder, and use backpropagation to update the parameters of the encoder and the second decoder, so that the second decoder outputs the original copywriting; Perform joint training using the first training task and the second training task, that is, a batch of data contains both the first type of damaged copywriting and the second type of damaged copywriting, and the loss updated at each step is the sum of the losses of the first training task and the second training task; Generating multi-product copywriting; Constructing product description data: Extract product information from the original copywriting and organize it in the following format: Copywriting generation: [Product information 1], [Product information 2], …, [Product information m]; Comparison information 1, Comparison information 2, …, Comparison information n; where m is the number of products in the copywriting, and n is the number of product comparison descriptions in the copywriting; the information described in a pair of square brackets is the information of one product; The third training task: Input the product description data into the encoder, input the hidden state output by the encoder into the second decoder, and use backpropagation to update the parameters of the encoder and the second decoder, so that the second decoder outputs the original copywriting; Based on the model after joint training using the first training task and the second training task, use the third training task for training; Polishing multi-product copywriting; Constructing data to be polished: Extract product dimension data from the original copywriting in the training set, input the extracted data into the encoder trained by the first training task, the second training task, and the third training task for forward inference, take the output hidden state, input it into the second decoder trained by the first training task, the second training task, and the third training task for forward inference, and save the output copywriting as the data to be polished; The fourth training task: Input the data to be polished in the format "Copywriting polishing: Copywriting to be polished" into the encoder, input the hidden state output by the encoder into the second decoder, and use backpropagation to update the parameters of the encoder and the second decoder, so that the second decoder outputs the original copywriting; Based on the model that has completed the third training task, use the fourth training task to train the model.

2. The method according to claim 1, wherein The construction of the first type of damaged copywriting is: randomly replace 15% of the words in the original copywriting with the mask symbol [MASK]; The construction of the second type of damaged copywriting is: randomly replace the product information of a certain product in the original copywriting with the product information of other products with a probability of 25%.

3. The method according to claim 1, characterized in that The product information has the following format: Product name: Product name i, Product embedding a, Product dimension 1: Value 1, …, Product dimension k: Value k; Among them, k is the number of product dimensions contained in the current product; Construction of the embedding of the commodity: Each commodity corresponds to a unique commodity embedding, and multiple aliases of a commodity correspond to the same commodity embedding. Suppose a certain commodity has multiple aliases and will be mentioned by different q aliases, and the set of names that will be mentioned is {Commodity Name 1, Commodity Name 2, …, Commodity Name q}. For the copywriting in the training set, replace the positions of any commodity name in the name set with the mask symbol [MASK]. Input the replaced copywriting into the encoder trained through the first training task and the second training task for forward inference. Take the hidden state output by the encoder and input it into the first decoder trained through the first training task and the second training task for forward inference, and save the output embedding corresponding to the position of the mask symbol in the input; Add up all the extracted embeddings and take the average as the commodity embedding a of the current commodity.

4. The method according to claim 1, wherein The format of the comparison information is: Commodity Name i > Commodity Name j: Evaluation aspect: Viewpoint.

Images