Advertising copywriting creation method and its device, equipment, medium and product

Abstract

The present application relates to a method for creating advertising copy and its apparatus, equipment, medium, and product, the method comprising: obtaining a product image of a target product; extracting the image features of the product image through an image encoder, and obtaining an image feature vector that maps the product image to a specific semantic space; providing the image feature vector as an initial input vector to a decoder to gradually autoregressively decode each text vector in a text sequence of the advertising copy, wherein the text vector of the previous step is used to query the key features in the image features of the product image as the input vector corresponding to the text vector of the next step; and restoring the text sequence of the advertising copy according to the text vector sequence predicted by the decoder. The advertising copy obtained in the present application is obtained based on the image features of the product image, can more comprehensively reflect the selling point characteristics in the product image, and has uniform quality without being affected by the creator's style.

Description

Technical Field

[0001] The present application relates to the field of e-commerce information technology, and in particular to an advertising copywriting method and its corresponding device, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] E-commerce platforms are usually equipped with advertising pages for store users to place advertisements corresponding to the products on the shelves in the store into the advertising system, so as to achieve the purpose of online traffic and promote the increase of product transaction volume.

[0003] Advertising copy is an important part of advertising and has a direct impact on the final effect of advertising. Advertising copy has the function of highlighting product information and selling points, deepening consumer impressions, and attracting consumers. However, in real practical scenarios, the creation of advertising copy requires steps such as product understanding, selling point extraction, organization and expression, and completion of writing, which is a very time-consuming and labor-intensive task. Moreover, with so many types of products, it is easy to not understand the selling points of products, resulting in creative errors and affecting advertising effects. Therefore, an automated advertising copywriting algorithm is very urgent and necessary.

[0004] Existing advertising copy generation algorithms mainly create advertising copy based on text information such as product titles. Such algorithms have at least the following problems:

[0005] 1. Since text information such as product titles is limited by display or platform length requirements, it is often only possible to give a general description of the product and is unable to give a detailed and accurate description of the product, resulting in the omission of key product information, which in turn affects the subsequent generation of advertising copy.

[0006] 2. Due to the differences in cultural backgrounds and creative styles of different merchants, it is easy to cause large differences in the style and content of product titles.

[0007] 3. The omission of key product information and the large differences in text data styles can easily lead to problems such as difficulty in algorithm training convergence and poor quality of created advertising copy.

[0008] The existence of the above problems leads to the limited effectiveness of existing advertising copy generation algorithms. Therefore, it is necessary to explore effective solutions. Summary of the invention

[0009] The purpose of this application is to solve the above problems and provide an advertising copywriting method and its corresponding device, computer equipment, computer readable storage medium, computer program product,

[0010] To meet the various purposes of this application, the following technical solutions are adopted:

[0011] In one aspect, a method for creating advertising copy is provided to meet one of the purposes of the present application, comprising:

[0012] Get the product image of the target product;

[0013] The image features of the product image are extracted by the image encoder, and the image feature vector of the product image mapped to a specific semantic space is obtained;

[0014] The image feature vector is provided as an initial input vector to a decoder to gradually autoregressively decode each text vector in the text sequence of the advertisement copy, wherein the text vector of the previous step is used to query the key features in the image features of the product image as the input vector corresponding to the text vector of the next step;

[0015] The text sequence of the advertisement copy is restored based on the text vector sequence predicted by the decoder.

[0016] On the other hand, an advertising copywriting device is provided to meet one of the purposes of the present application, including:

[0017] Image acquisition module, used to acquire product images of target products;

[0018] The image encoding module is used to extract the image features of the product image through the image encoder and obtain the image feature vector mapped to the specific semantic space of the product image;

[0019] A decoding output module is used to provide the image feature vector as an initial input vector to a decoder to gradually autoregressively decode each text vector in the text sequence of the advertisement copy, wherein the text vector of the previous step is used to query the key features in the image features of the product image as the input vector corresponding to the text vector of the next step;

[0020] The text restoration module is used to restore the text sequence of the advertising copy based on the text vector sequence predicted by the decoder.

[0021] On the other hand, a computer device is provided to meet one of the purposes of the present application, including a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the advertising copywriting method described in the present application.

[0022] On the other hand, a computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the advertising copy creation method in the form of computer-readable instructions, and when the computer program is called and run by a computer, it executes the steps included in the method.

[0023] On the other hand, a computer program product provided to meet another purpose of the present application includes a computer program / instructions, which, when executed by a processor, implement the steps of the advertising copy creation method described in any embodiment of the present application.

[0024] Compared with the prior art, this application has many advantages, including but not limited to:

[0025] First, this application can generate advertising copy corresponding to the target product by relying only on the image features provided by the product image of the target product. Since the product image provides more practical product characteristics, the image features obtained by semantic understanding of the product image can effectively and more comprehensively represent the selling point characteristics of the target product, so that effective advertising copy can be obtained based on these image features.

[0026] Secondly, when creating advertising copy, the present application does not need to rely on text information as the initial input of the advertising copy creation model, is not affected by artificial cultural background and creative style, and can create advertising copy with consistent style.

[0027] In addition, the algorithm implemented according to the principles of the present application can obtain more comprehensive selling point characteristics through product pictures and is not affected by differences in text styles. It only needs to use product pictures for training, so sample preparation is more efficient, the algorithm is easier to converge, and the quality of the created advertising copy is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0029] Figure 1 This is a block diagram of an exemplary network architecture principle of the advertising copywriting model implemented in this application.

[0030] Figure 2 A flowchart of a typical embodiment of the advertising copywriting method of the present application.

[0031] Figure 3 Schematic diagram of the process of obtaining the image feature vector of the target product in the embodiment of the present application.

[0032] Figure 4 Schematic diagram of the decoding process of the decoder in the embodiment of the present application.

[0033] Figure 5 Schematic diagram of the process of obtaining key feature vectors in the attention layer query in the embodiment of this application

[0034] Figure 6This is a block diagram of an exemplary network architecture for implementing training of a generative adversarial network in this application.

[0035] Figure 7 This is a schematic diagram of an exemplary network architecture of a discriminator in a generative adversarial network of the present application.

[0036] Figure 8 This is a flow chart of implementing two-stage training for the advertising copywriting model in an embodiment of the present application.

[0037] Fig. 9 This is a principle block diagram of the advertising copy creation device of this application;

[0038] Fig.10 A schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0039] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0040] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0041] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0042] It will be understood by those skilled in the art that the "client", "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication system), which can combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant, personal digital assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System, global positioning system) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable and / or configured to run locally, and / or in a distributed form, at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playback function, or a smart TV, a set-top box and other devices.

[0043] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with capabilities equivalent to those of a personal computer. It is a hardware device that has the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0044] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0045] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.

[0046] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0047] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0048] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0049] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0050] An advertising copywriting method of the present application can be programmed as a computer program product and deployed in a client or server for execution. For example, in the exemplary application scenario of the present application, it can be deployed and implemented in a server of an e-commerce platform, thereby accessing an interface opened after the computer program product is running and performing human-computer interaction with the process of the computer program product through a graphical user interface to execute the method.

[0051] The exemplary application scenario of this application is the application of advertising scenarios in e-commerce platforms. E-commerce platforms are usually equipped with advertising systems, which open corresponding advertising publishing interfaces to merchant users of various online stores, obtain advertising copy and advertising configuration information corresponding to advertisements placed by merchant users of any store to the advertising system through the advertising publishing interface, and submit them to the advertising system's own advertising publishing channels or third-party advertising publishing channels for public display.

[0052] In the present application, the advertising copy is mainly expressed as natural language text, which can be text in any language. Each of the stores can be deployed in an independent site of the e-commerce platform. Each store can list a large number of goods, and the product information corresponding to each product is stored in the product information library of the online store. Users can enter the store and visit the transaction page of any product therein to place orders and make payments, thereby realizing e-commerce transactions. Each store can place advertisements corresponding to any product in its store to the advertising system. Such products can be called advertising products. In the process of publishing advertisements, the advertising copy corresponding to the advertising products is provided, and submitted to the advertising system through the advertising publishing interface for publishing.

[0053] The advertising copy creation model of the present application can automatically generate the advertising copy for the specified target product. In order to generate the advertising copy, the product image corresponding to the target product can be determined as the input of the advertising copy creation model, and then the advertising copy creation model outputs the advertising copy corresponding to the selling point characteristics of the product presented by the product image.

[0054] See also Figure 1 , which shows the network architecture of the advertising copy creation model of the present application, the model includes a picture encoder, a decoder and an attention layer, the picture encoder can extract image features from the product picture input therein, the decoder can decode the text vector sequence of the advertising copy according to the image features of the product picture, and subsequently restore the text sequence corresponding to the advertising copy according to the text vector sequence.

[0055] After the advertising copywriting model is pre-trained to a convergence state, it can learn the ability to generate corresponding advertising copywriting according to a given product image. The training method of the advertising copywriting model can be flexibly implemented.

[0056] In one embodiment, the advertising copy creation model is trained based on the idea of ​​generative adversarial training. First, a discriminator suitable for distinguishing the quality categories of text vectors output by a decoder in the advertising copy creation model is prepared, and the discriminator is trained to a convergence state. Then, the advertising copy creation model is used as a generator and then connected to the discriminator to form a generative adversarial network. Under the condition of freezing the weight of the discriminator, the generative adversarial network is trained using corresponding training samples. After the generator therein is trained to a convergence state, the generator can be output as an advertising copy creation model suitable for online reasoning.

[0057] In another embodiment, the advertising copy creation model is trained using a two-stage training method. In the first stage, independent pre-training is performed on the image encoder and decoder respectively, and the image encoder and decoder are trained to a convergence state respectively. Then, in the second stage, the pre-trained image encoder is connected to the decoder to form an advertising copy creation model, and then the corresponding training samples and supervision labels are used to train the advertising copy creation model to a convergence state.

[0058] The embodiments related to the advertising copywriting creation model training of the present application will be disclosed in depth later and will not be discussed here for the time being.

[0059] In the advertising copywriting creation model, the image encoder can be any basic model suitable for extracting image features, for example, AlexNet, Resnet, DenseNet, EfficientNet based on multiple convolution operations, Transformer encoders based on attention mechanisms, Vision Transformer encoders, and Swin Transformer based on multiple Transformer encoders, and so on, as long as they are suitable for extracting image features from the input images.

[0060] In the advertising copywriting model, the decoder can be any recurrent neural network (RNN) that processes sequence tasks, or LSTM (long short-term memory network) evolved on the basis of RNN. In one embodiment, it is recommended to directly use the decoder of Transformer, or the decoder of BART model, etc. These decoders are all based on RNN and add attention mechanism, which can effectively decode sequence information. Those skilled in the art can flexibly select according to the principles disclosed herein.

[0061] In the advertising copywriting model, the attention layer is used to query the key features in the image features of the product image according to the text vector obtained by decoding after each decoding step of the decoder, and then append the key features to the input of the decoder, so that the decoder can further combine the key features to predict the corresponding text vector for the next step. The attention layer can be a multi-head attention layer or a single-head attention layer.

[0062] It is not difficult to understand that when the advertising copy creation model of the present application is finally trained to a convergence state, it can be put into use for online reasoning, and thus, can be used to realize the automatic generation of advertising copy desired by the present application.

[0063] See also Figure 2 In a typical embodiment, the advertising copywriting method of the present application includes the following steps:

[0064] Step S1100, obtaining a product image of a target product;

[0065] The product images mainly include the default display images of the target product, which are prominently displayed in the detail display page of the target product, or displayed as the default images of the summary display interface of the target product. In addition, other supporting images corresponding to the target product can also be collected. These configuration sets are generally displayed in the detail display area of ​​the detail display page of the target product. The selection of product images can be flexibly determined by those skilled in the art. The product images provide a description of the characteristic features of the target product, namely the selling point characteristics, in the form of images. Therefore, semantic information corresponding to the selling point characteristics of the target product can be extracted through the product images.

[0066] In one embodiment, in combination with the advertising scenario of the e-commerce platform, the product image of the target product can be obtained through the following process:

[0067] Step S1110, responding to the advertising copy prediction request and determining the target product specified in the request;

[0068] The advertisement publishing user can trigger an advertisement copy prediction request on the backend page of his online store, specify the corresponding target product in the request, and the server determines the target product in response to the request.

[0069] Step S1120: query and obtain a product image corresponding to the target product from a product information database;

[0070] According to the target product specified in the advertisement copy prediction request, the product image corresponding to the target product can be queried from the product information library of the online store to which the advertisement publishing user has access rights. In one embodiment, multiple product images of the target product can be obtained at the same time, and then the product image with the most complete information can be determined through a preset product image classification model as the product image to be obtained. The product image finally obtained can be either the default display image set in the product information library for the target product or other supporting images.

[0071] Step S1130: Complete format preprocessing of the product image.

[0072] For the product images of the target products, we can perform pre-processing such as enhancement or conversion on the product images based on the characteristics of the product images in the e-commerce scenario, such as most products are horizontally centered, some are moderately rotated, the product size is large, the image is clearer, etc., and based on the input requirements of the image encoder, such as image size, number of channels, pixel value normalization, etc., so as to make the product images more suitable for processing by the image encoder.

[0073] Through the above process, it is possible to obtain the corresponding product images according to the target product specified by the user, and obtain standardized information resources, ensuring that the corresponding advertising copy can be predicted efficiently and accurately based on these information resources in the future.

[0074] Step S1200: extracting image features of the product image through the image encoder, and obtaining an image feature vector of the product image mapped to a specific semantic space;

[0075] The product image for which the corresponding advertising copy needs to be generated is input into the image encoder of the advertising copy creation model of this application. The image encoder extracts deep semantic information from the product image, and extracts the image features representing the selling characteristics of the target product in the product image. Finally, an image feature vector that conforms to the specifications of a specific semantic space can be obtained. The specific semantic space can be coordinated and standardized with the decoder so that the image feature vector can be converted into the input of the decoder. For example, the specific semantic space may require that the dimension of the obtained image feature vector is 512 dimensions.

[0076] Step S1300: providing the image feature vector as an initial input vector to a decoder for stepwise autoregressive decoding to obtain various text vectors in a text sequence of the advertisement copy, wherein the text vector of the previous step is used to query the key features in the image features of the product image as the input vector corresponding to the text vector of the next step;

[0077] The decoder is suitable for using the input vector of the initial input as prefix information for feature interaction based on the attention layer. In the present application, the image feature vector obtained by extracting the product image from the picture encoder of the advertising copy creation model is used as the prefix information input of the decoder. Then, the decoder outputs the text vector corresponding to each word in the advertising copy one by one in an autoregressive manner. When selecting the text vector to be generated, a beam search algorithm (Beam Search) or a greedy search algorithm (Greedy Search) can be used to improve the search efficiency. This text vector is mapped to the probability of each word in the vocabulary corresponding to the decoder through a fully connected layer, so each text vector is processed using the same fully connected layer with the help of a forward feedback layer. Each text vector is output as a specific dimension, such as 512 dimensions, according to the specification of the specific semantic space. After passing through the fully connected layer, it is converted into a text vector with the same dimension as the vocabulary size, wherein the word element pointing to the maximum probability in the vocabulary is the word element corresponding to the text vector, which is suitable for constituting a word in the advertising copy.

[0078] The decoder can be a decoder corresponding to the GPT-2 (Generative Pre-Training) version. Others, such as XLNet, DistilGPT2, etc., may also be suitable for constructing the decoder required by this application.

[0079] The decoder generates a text vector of the text corresponding to the input based on the input vector. This input can be used as the previous context, that is, the decoder predicts a text vector of the following text corresponding to the previous context. The initial previous context is the prefix information. After predicting a text vector based on the prefix information, based on the autoregressive principle of the decoder, the text vector will be appended to the previous context to form a new previous context, and then the next text vector will be generated based on the new previous context. This process is iteratively executed to continuously predict the text vectors corresponding to all the text in the advertising copy.

[0080] In the present application, in the process of the decoder autoregressively appending the text vector predicted by the previous step to the context of the previous step for predicting the text vector of the next step, an independently set attention layer is used to query the key features in the image features of the product image based on the text vector of the previous step, and then the key features are appended to the context of the previous step to generate a new context corresponding to the prediction of the next text vector. In this way, the key features in the product image are autoregressively queried using the information gradually output by the decoder, and the image features of the product image are gradually deeply mined. The key features effectively represent the feature information in the product image that is valuable for generating advertising copy, thereby more effectively guiding the decoding process of the decoder, in order to produce high-quality advertising copy.

[0081] In one embodiment, in the process of querying the key features in the image features of the product image based on the text vector of the previous step, the image features used can be the image features obtained by the image encoder in the intermediate process of extracting the image features. For example, when the image encoder extracts the image features of the product image based on multiple layers of convolution, the image features obtained by the second convolution layer or the second convolution layer can be used; similarly, when the image encoder extracts the image features of the product image layer by layer based on multiple attention layers, the image features obtained by the second or third attention layer can also be used. When the image features are not standardized to the specific semantic space of the present application, they can be mapped to the semantic space through a fully connected layer. Since the deeper the operation depth of the image encoder in the process of extracting the image features of the product image, the more abstract the semantics represented by the obtained image features, and the clearer the operation depth, the richer the semantics represented by the obtained image features, in view of this, the image features in the intermediate process of the image encoder are used to query the key features, which can avoid missing the key information in the product image and ensure that the key features added to the previous step above have practical significance.

[0082] Step S1400: restore the text sequence of the advertisement copy according to the text vector sequence predicted by the decoder.

[0083] After the decoder predicts all the text vectors corresponding to the entire advertising copy, that is, obtains the text vector sequence corresponding to the text sequence of the advertising copy, then the vocabulary corresponding to the decoder can be queried one by one for each text vector, and the corresponding word in the vocabulary can be obtained as the corresponding text, and these texts can be sequentially spliced ​​into the advertising copy.

[0084] According to this embodiment, it can be known that in the process of generating advertising copy, it is only necessary to input the prefix information obtained according to the product image into the image encoder of the advertising copy creation model of the present application, so that the text corresponding to each word in the advertising copy can be obtained serially under the action of the autoregressive characteristics of the decoder, and the subsequent text is automatically predicted according to the previous context. On the basis of the text vector sequence of the product text, the key feature sequence of the product image is further combined, and the key feature sequence itself is the key information deeply mined in advance based on the image features of the product image with reference to the text feature sequence. Therefore, although it only depends on the product image, the semantic features contained in the prefix information input into the decoder at each step are richer and more accurate, which can ensure that the decoder can effectively predict the advertising copy corresponding to the product image of the target product with sufficient semantic information as a reference, so that the advertising copy can accurately describe the selling point characteristics of the target product.

[0085] According to the above embodiments, the present application has many advantages, including but not limited to:

[0086] First, this application can generate advertising copy corresponding to the target product by relying only on the image features provided by the product image of the target product. Since the product image provides more practical product characteristics, the image features obtained by semantic understanding of the product image can effectively and more comprehensively represent the selling point characteristics of the target product, so that effective advertising copy can be obtained based on these image features.

[0087] Secondly, when creating advertising copy, the present application does not need to rely on text information as the initial input of the advertising copy creation model, is not affected by artificial cultural background and creative style, and can create advertising copy with consistent style.

[0088] In addition, the algorithm implemented according to the principles of the present application can obtain more comprehensive selling point characteristics through product pictures and is not affected by differences in text styles. It only needs to use product pictures for training, so sample preparation is more efficient, the algorithm is easier to converge, and the quality of the created advertising copy is better.

[0089] Based on any embodiment of this application, please refer to Figure 3 The step S1200, extracting the image features of the product image through the image encoder, and obtaining the image feature vector of the product image mapped to the specific semantic space, includes:

[0090] Step S1210: divide the product image into multiple sub-images of equal size to form a sub-image sequence;

[0091] In this embodiment, Vision Transformer is used as an image encoder to encode the product image. According to the principle of Vision Transformer, the product image is divided into multiple sub-images of equal specifications according to a preset size, thereby obtaining a sub-image sequence.

[0092] Step S1220: Applying a picture encoder to extract image features from each sub-image in the sub-image sequence to obtain image feature information corresponding to each convolutional layer;

[0093] The sub-image sequence is input into the picture encoder for feature interaction. Vision Transformer is an encoding component based on the attention layer, which extracts the image feature information corresponding to each sub-image by performing deep feature interaction on each sub-image.

[0094] Step S1230: Process the image feature information into an image feature vector adapted to the input of the decoder so as to project it into a specific semantic space.

[0095] Finally, the image feature information corresponding to each sub-image obtained in the previous step can be converted into the image feature vector required for the input of a specific semantic space, so as to adapt to the input specification of the decoder and achieve the coordination and unification of the image feature vector output by the picture encoder and the input requirements of the decoder.

[0096] In this embodiment, by dividing the product image into sub-images and extracting features of the product image based on the local information corresponding to each sub-image, the image semantic features of the product image can be obtained at a finer granularity, which can more effectively guide the decoder to accurately predict the advertising copy.

[0097] Based on any embodiment of the present application, different from the previous embodiment, the step S1200, extracting the image features of the product image through the image encoder, and obtaining the image feature vector mapped to the specific semantic space of the product image, includes:

[0098] Step S1201: gradually extracting image features of the product image through multiple convolutional layers preset in the image encoder to obtain image feature information corresponding to each convolutional layer;

[0099] Taking the picture encoder implemented based on the basic model such as AlexNet and ResNet that applies the principle of multi-layer convolution as an example, the picture encoder contains multiple convolution layers, which extract image features from the product pictures from shallow to deep. Each convolution layer can obtain the image features extracted by this layer, and then input its image features into the next convolution layer to continue to extract the image features of a deeper layer. In the whole process, each convolution layer can output the corresponding image feature information, and the image feature information corresponding to any of the convolution layers can be obtained as needed later, so as to obtain the image features extracted at this layer.

[0100] Step S1202: fully connect and map the image features extracted by the last convolutional layer to a specific semantic space adapted to the input of the decoder to obtain an image feature vector.

[0101] The image feature information output by the last convolutional layer of the picture encoder can be mapped to a specific semantic space after passing through the fully connected layer, thereby obtaining an image feature vector that meets the decoder input requirements.

[0102] In this embodiment, the principle of multi-layer convolution is applied to extract the image features of the product image layer by layer, and finally output an image feature vector that meets the requirements of the decoder. The implementation cost is low and the feature extraction is accurate. The image encoder built on this basis, or the model built on the basis of the image encoder, including the advertising copy creation model, generative adversarial network, generator, discriminator, etc. of this application, is easier to train quickly to convergence.

[0103] Based on any embodiment of this application, please refer to Figure 4 The step S1300, providing the image feature vector as an initial input vector to a decoder for stepwise autoregressive decoding to obtain each text vector in a text sequence of the advertisement copy, comprises:

[0104] Step S1310, inputting the image feature vector as an initial input vector into the decoder to start decoding step by step;

[0105] The image feature vector output by the picture encoder is directly input to the decoder as an input vector corresponding to the prefix information at the initial moment, and the decoder then starts autoregressive step-by-step decoding.

[0106] Step S1320: After each decoding step, the decoder predicts the corresponding text vector according to the input vector, and transfers the text vector to the preset attention layer by autoregression;

[0107] The decoder decodes based on an autoregressive mechanism, and its decoding process can be regarded as a multi-step operation corresponding to multiple moments. Therefore, at each decoding step, the decoder applies its own inherent algorithm to predict the corresponding text vector based on the input vector corresponding to that step. The text vector is represented as the probability distribution of each word mapped to the vocabulary corresponding to the decoder, where the word corresponding to the maximum probability is the text required to construct the advertising text. According to the autoregressive mechanism of the decoder, the text vector is autoregressively transferred to the preset attention layer in the advertising copywriting model.

[0108] Step S1330: The attention layer queries the key feature vector in the image feature information in the process of extracting image features by the picture encoder according to the autoregressive text vector;

[0109] The attention layer has two inputs, one is the intermediate image feature information obtained from the image feature extraction process of the image encoder, and the other is the text vector output by the decoder and autoregressive. Based on this, QKV operation can be performed based on the image feature information and the text vector, and the key feature vector in the image feature information is obtained through the operation to achieve the extraction of key features in the image feature information. Since the referenced image feature information contains the image features of the product image extracted by the image encoder at a shallow level, its semantics are more original and relatively richer and more comprehensive, and it is expected to query more effective feature information, such as the selling point features of the target product represented by the product image.

[0110] Step S1340: Use the key feature vector as the input vector for the next decoding of the decoder to continue iteratively predicting the next text vector until the prediction of each text vector corresponding to the text sequence of the advertisement copy is completed.

[0111] The attention layer further inputs the key feature vector it obtains into the decoder as the input vector for the next decoding of the decoder. The decoder continues to predict the next text vector according to its own algorithm logic, and so on, iterating in a loop until the prediction of each text vector corresponding to the text sequence of the advertising copy is completed.

[0112] According to this embodiment, it can be understood that the attention layer built into the advertising copy creation model plays a role in updating the prefix information input by the decoder during the gradual decoding process of the decoder, and gradually provides the decoder with key features derived from the product image, so that the decoder can continuously predict the text vector based on the image semantics of the product image, while maintaining the consistency of the creative style, by continuously adding key features and comprehensively considering the deep semantics of the product image, thereby generating advertising copy with description content corresponding to the selling point characteristics presented in the product image.

[0113] Based on any embodiment of this application, please refer to Figure 5 The step S1330, in which the attention layer queries the key feature vector in the image feature information in the process of extracting image features by the picture encoder according to the autoregressive text vector, includes:

[0114] Step S1331: using the autoregressive text vector as the query vector, and the image feature information of the image encoder as the key vector and the value vector, the query vector and the key vector are interacted to determine the key vector of the product image;

[0115] Since the key vector is the image feature information of the product image, which is a feature representation of the product image, and the query vector is the text vector output by the decoder, which contains the semantic information corresponding to the previous text of the advertisement copy, in the embodiment using the multi-head attention mechanism in the attention layer, the query vector With key vector Match their respective learnable weights , After that, the dot product operation is performed to realize feature interaction, and the semantics of the previous text provided by the text vector is used as a reference to obtain the key vector Match the corresponding learnable weights The obtained results determine the key features of the image features in the product image, which can be expressed as a key feature vector.

[0116] For an embodiment in which the attention layer uses a single-head attention mechanism, the query vector No matching of learnable weights is required.

[0117] Step S1332, normalizing the key vector to calculate the attention weight and obtain a weight vector;

[0118] After obtaining the key features, the Softmax function can be used to normalize them, and the feature values ​​therein can be mapped to a confidence interval of [0,1] to obtain a weight vector for representing the weight of the criticality of each corresponding feature in the image feature information of the product image.

[0119] Step S1333: perform weighted summation of the value vector using the weight vector to obtain a key feature vector.

[0120] In order to extract the key feature vector from the image feature information of the product image, the value vector (The corresponding learnable weights can be matched in advance as needed ), that is, the image feature information, is multiplied by the weight vector to achieve the weighted summation of the eigenvalues ​​in the image feature information, thereby obtaining the final key feature vector, in which each feature value is adjusted under the action of the weight vector, thereby realizing the mining and representation of the key features in the product image.

[0121] In some embodiments, whether based on a single-head attention mechanism or a multi-head attention mechanism, the value vector The key vector can be reused directly , so as to reduce the weight parameters that need to be learned, so as to accelerate the convergence speed of the corresponding model during the training process. It is not difficult to understand that the various learnable weights are back-propagated and modified during the training process of the image encoder, and are solidified after the image encoder is trained to convergence.

[0122] In this embodiment, the attention layer performs attention operation on the image feature information of the product image by referring to the text vector predicted by the decoder, realizes deep mining of key features in the product image, and obtains key feature vectors, so that the key feature vectors can effectively represent the feature information in the product image that is valuable for generating advertising copy, and thus can further append the context information required for the next decoding of the decoder, gradually enhance the semantic reference required for decoding by the decoder, and more effectively guide the automatic generation of advertising copy.

[0123] On the basis of any embodiment of the present application, before the step S1200, i.e. the step of extracting the image features of the product image through the image encoder, the advertising copywriting creation model of the present application is trained by applying the generative adversarial idea to train it to a convergence state. Accordingly, the training process includes:

[0124] Step S2100: construct the image encoder and decoder into an advertising copywriting model, and construct a generative adversarial network with a preset discriminator as a generator, wherein the discriminator is pre-trained to a convergence state and is suitable for binary classification of the quality category of the advertising copy;

[0125] See also Figure 6 The generative adversarial network shown includes a generator and a discriminator. The generator directly adopts the advertising copywriting model of the present application. The discriminator is prepared to meet the needs of generative adversarial training. It has been trained in advance to obtain the ability to identify the quality category of the advertising copy corresponding to the text vector sequence output by the decoder of the advertising copywriting model of the present application by binary classification. That is, a text vector sequence corresponding to a text sequence of an advertising copy is provided, and the discriminator can judge whether the corresponding advertising copy is a qualified positive sample or an unqualified negative sample.

[0126] In one embodiment, the network architecture of the discriminator is as follows: Figure 7 As shown, it uses an image encoder and a decoder that are isomorphic to the image encoder and decoder used in the advertising copy creation model of the present application as two basic models, and builds these two basic models into a double-tower model. The image encoder extracts the image feature vector of the product image, and the decoder decodes the text vector sequence input therein to obtain another text vector sequence. Then, the image feature vector and the text vector sequence are spliced ​​into a merged feature vector through a splicing layer, and then mapped to the binary classification space of the classifier through a fully connected layer to obtain the classification probabilities corresponding to the two categories of positive samples and negative samples. The category with the largest classification probability is the quality category determined by the discriminator according to the input.

[0127] After the discriminator is pre-trained to a convergence state according to the above architecture, it can make a judgment based on the input. As a result, in the generative adversarial network, the text vector sequence generated by the generator decoding the image feature vector of the product image can be decoded to obtain another text vector sequence, which is then combined with the image feature vector extracted from the product image by the discriminator's image encoder to obtain a merged feature vector. The quality category is discriminated based on the merged feature vector to obtain the classification probability corresponding to each quality category mapped to the binary classification space.

[0128] In a generative adversarial network, the input of the discriminator is the text vector sequence output by the decoder of the generator. Taking into full consideration that the text vector sequence is a set of offline data, which will block the back propagation in the gradient update process during training, such an architecture is adopted to use the probability distribution of each word of the advertising copy generated by the generator as the input of the decoder in the discriminator, so that the generator can participate in the gradient update normally throughout the entire generative adversarial training process, including the stage when the generator provides the discriminator with the text vector sequence corresponding to the product image as a negative sample when the discriminator is trained in advance, and the generator can also participate in the gradient update normally.

[0129] In another embodiment, in the pre-training stage of the discriminator, not only the text vector sequence generated by the generator based on the product image as the training sample can be used as the input of the discriminator's decoder, but also the coding vector sequence of the advertising copy corresponding to the product image collected from the historical product advertisements of the advertising system can be used as the input of its decoder. The coding vector sequence can be obtained by self-encoding the advertising copy of the historical advertising product and normalizing it after adding uniformly distributed noise. The encoding method can be a one-hot encoding method, and the uniformly distributed noise can be a probability distribution in a preset confidence interval, such as [-3,3]. It can be seen that the input of the discriminator can be compatible with both the text vector sequence output by the generator and the coding vector sequence obtained by self-encoding, which facilitates the expansion of the total number of samples required for its training, so that both the hypothetical samples generated by the generator and the real samples collected in advance can be used to train the discriminator, making it easier for the discriminator to be trained to convergence.

[0130] Step S2200: freeze the weight of the discriminator, use the product image as the input of the generative adversarial network, calculate the loss value output by the discriminator with the label corresponding to the positive sample, and train the generator to a convergence state.

[0131] Before starting the training of the generative adversarial network, the discriminator can be configured to freeze its weights so that the weights are in an uncorrectable state. Then, the training of the entire generative adversarial network can be started. In essence, the training of the generator is implemented with the help of the generative adversarial network.

[0132] When implementing training, product images can be used as training samples. The product images are respectively input into the image encoder of the generator and the image encoder of the discriminator to extract image feature vectors. In the generator, the image feature vector obtained by its image encoder is further decoded by the decoder in the generator to obtain a text vector sequence corresponding to the advertising copy, and then directly input into the decoder of the discriminator for decoding, and the decoder decodes another text vector sequence accordingly. Subsequently, the discriminator performs feature splicing of the image feature vector obtained by its image encoder and another text vector sequence obtained by its decoder through its splicing layer to obtain a merged feature vector, and then performs binary classification after full connection to obtain the classification probability mapped to the binary classification space.

[0133] According to the generative adversarial idea, during the generator training stage, it is assumed that the text vector sequence generated by the generator is recognized by the discriminator as a positive sample that meets the expectations. Therefore, the loss value is calculated for the quality category corresponding to the positive sample, that is, the cross entropy loss assuming that the sample label is a positive sample (which can be expressed as 1) is calculated, and then the weight parameters of the generator are updated through the gradient descent algorithm.

[0134] In the process of training the generator, after obtaining the loss value corresponding to each training sample, a preset threshold can be used to compare it in order to determine whether the generator has reached a converged state. Specifically, when the loss value reaches the preset threshold, it indicates that the generator has been trained to a converged state, and the generator can be output as the advertising copy creation model required for online reasoning in this application. When the loss value does not reach the preset threshold, it indicates that the generator has not been trained to a converged state, and the weights of the generator can be updated, and then the next training sample can be called to continue iterative training of the generator until the generator is trained to a converged state to produce the advertising copy creation model required by this application.

[0135] According to the above training process, it can be known that by training the advertising copy creation model of this application with the help of generative adversarial thinking, the advertising copy creation model can be effectively trained to convergence quickly, so that it can obtain a more reliable advertising copy generation capability. In particular, when the discriminator also uses the image encoder and decoder used by the generator, by cleverly processing the input of the discriminator's decoder, the discriminator is quickly trained to convergence in advance so that it can obtain the ability to distinguish the quality of the text vector sequence generated by the generator, so that the discriminator can subsequently provide effective judgment criteria for the training of the generator, supervise the training of the generator, and improve the generator's ability to generate advertising copy through training, and finally output an advertising copy creation model that meets the requirements of this application.

[0136] Based on any embodiment of the present application, different from the previous embodiment, please refer to Figure 8 Before the step S1200, in which the image features of the product image are extracted by the image encoder, the advertising copywriting creation model of the present application is trained by applying the two-stage training idea to train it to a convergence state. Accordingly, the training process includes:

[0137] Step S3100: collect product images, advertisement texts and product category identifiers in historical product advertisements, and construct them into mapping relationship data in the data set;

[0138] The training samples required for the two-stage training of the advertising copy creation model of the present application can be obtained from the advertising system of the e-commerce platform. Specifically, the product images, advertising copy and product category identification in the historical product advertisements in the advertising system are obtained, and mapping relationship data are constructed for each product. These mapping relationship data are stored in the data set, and the training samples required for each training stage can be flexibly selected therefrom later.

[0139] Step S3200: using the product images in the mapping relationship data of the data set as training samples and inputting the image encoder for pre-training, using the corresponding preset product category identifier as the supervision label of the product image pre-training, and training the image encoder to a convergence state;

[0140] In the first stage of training, the image encoder is first trained to enable it to have the ability to extract image features. To this end, the image encoder can be connected to a classifier for training. In a single training, a product image in a mapping relationship data of the data set is used as the input of the image encoder, and the image encoder extracts image features from the product image to obtain an image feature vector, which is mapped to the classification space of the classifier through a fully connected layer to obtain the corresponding classification probability of each category as the classification result. Then, the product category identifier in the mapping relationship data is used as a supervision label to calculate the loss value of the classification result. When the loss reaches a preset threshold, it indicates that the image encoder has been trained to a convergence state, and the image encoder can be used as a component of the advertising copywriting creation model of the present application, and of course, it can also be used as a component of the generator and discriminator of the previous embodiment of the present application. When the loss value does not reach the preset threshold, it indicates that the image encoder has not been trained to a convergence state, and the weight update can be implemented on the image encoder, and then the next training sample is continued to be called, and the image encoder is continued to be iteratively trained until the image encoder is trained to a convergence state, thereby completing the pre-training of the image encoder.

[0141] Step S3300: input the advertisement text in the mapping relationship data of the data set as training samples into the decoder for unsupervised pre-training, and train the decoder to a convergence state;

[0142] Similarly, in the first stage of training, it is also necessary to perform pre-training of the decoder so that it has the ability to obtain text vectors according to the corresponding step-by-step decoding predictions of the input vector. Since the decoder used in this application is a decoder with an autoregressive mechanism, the input vector based on which it decodes can be used to self-supervise the text vector obtained by decoding, thereby realizing unsupervised training. Accordingly, the advertising text in the mapping relationship data of the data set can be used as a training sample, which is encoded into an embedded vector and then serialized into the decoder to provide the input vector required for step-by-step decoding. Similarly, it can be understood that the decoder is based on an autoregressive mechanism. After each step of decoding, it will append the text vector obtained by its own decoding to the input required for the next step of decoding, so as to realize context association in the decoding process.

[0143] The text vector obtained by the decoder for each word in the advertisement is a probability distribution obtained by mapping the classifier constructed by the Softmax function to each word in the preset vocabulary. Therefore, based on the next word in the advertisement, the cross entropy function is used to calculate the loss value of the text vector, and then the decoder is updated according to the gradient descent of the loss value. The decoder is trained by iterating multiple training samples, and the loss value can eventually reach a convergence state, and the training of the decoder can be terminated. As a result, the decoder acquires the ability to decode the text vector sequence corresponding to the advertisement according to the input vector.

[0144] Step S3400: construct the pre-trained image encoder and decoder and the attention layer into an advertising copy creation model, use the product image as input, and use the advertising copy corresponding to the product image as a supervision label to train the model to a convergence state.

[0145] After completing the pre-training of the image encoder and decoder, the image encoder can be connected to the decoder according to the network structure of the advertising copy creation model of the present application, and an attention layer can be connected between the image encoder and the decoder to construct the network architecture of the advertising copy creation model of the present application. Then, fine-tuning training can be implemented for the network architecture.

[0146] When implementing fine-tuning training, the product images in the mapping relationship data of the data set can be used as training samples for the input of the advertising copy creation model. Then, according to the algorithm principle of the advertising copy creation model of the present application, the image encoder first obtains the image feature vector of the product image, and then the decoder gradually decodes the text vector corresponding to each word in the advertising copy based on the image feature vector and the key features obtained by the attention layer under the autoregressive mechanism, so as to obtain the text vector sequence corresponding to the advertising copy.

[0147] Since the mapping relationship data of the data set stores advertising texts related to the same historical product advertisements corresponding to the product images used as training samples, a supervision label can be provided based on the advertising texts for use in the fine-tuning training process. The cross entropy loss of the text vector output by the decoder at each step is calculated to obtain the loss value. When the loss value reaches a preset threshold, the fine-tuning training is terminated to generate an advertising copy creation model. When the loss value does not reach the preset threshold, back propagation is performed to correct the weight parameters of the advertising copy creation model to make it close to convergence. Then, the next training sample in the data set is called to perform iterative training, and so on, until the advertising copy creation model is trained to a convergence state. In this way, the advertising copy creation model required for online reasoning in this application can be generated.

[0148] According to the above process, it can be seen that the two-stage training method is used to train the advertising copywriting creation model of this application, which is relatively simple and efficient, and it is easy to quickly train the model to a convergence state, and the implementation cost is relatively low.

[0149] See also Fig. 9 , an advertising copy creation device is provided to meet one of the purposes of the present application, which is a functional embodiment of the advertising copy creation method of the present application, and the device includes an image acquisition module 1100, an image encoding module 1200, a decoding output module 1300, and a copy restoration module 1400, wherein: the image acquisition module 1100 is used to obtain a product image of a target product; the image encoding module 1200 is used to extract the image features of the product image through an image encoder, and obtain an image feature vector that maps the product image to a specific semantic space; the decoding output module 1300 is used to provide the image feature vector as an initial input vector to a decoder to gradually autoregressively decode each text vector in a text sequence of the advertising copy, wherein the text vector of the previous step is used to query the key features in the image features of the product image as the input vector corresponding to the text vector of the next step; the copy restoration module 1400 is used to restore the text sequence of the advertising copy according to the text vector sequence predicted by the decoder.

[0150] Based on any embodiment of the present application, the image acquisition module 1100 includes: a request response unit, used to respond to an advertising copy generation request and determine the target product specified by the request; a query acquisition unit, used to query and obtain the product image corresponding to the target product from a product information library, wherein the product image is the default display image of the target product; and a format processing unit, used to complete format preprocessing of the product image.

[0151] Based on any embodiment of the present application, the image encoding module 1200 includes: an image segmentation unit, used to segment the product image into multiple sub-images of equal specifications to form a sub-image sequence; a feature extraction unit, used to apply an image encoder to extract image features for each sub-image in the sub-image sequence to obtain image feature information corresponding to each convolutional layer; a vector encoding unit, used to process the image feature information into an image feature vector adapted to the input of the decoder so that it is projected into a specific semantic space.

[0152] Based on any embodiment of the present application, and different from the previous embodiment, the picture encoding module 1200 includes: a convolution extraction unit, used to gradually extract the image features of the product picture through multiple convolution layers preset in the picture encoder, and obtain image feature information corresponding to each convolution layer; a vector encoding unit, used to fully connect and map the image features extracted by the last convolution layer to a specific semantic space adapted to the input of the decoder, and obtain an image feature vector.

[0153] Based on any embodiment of the present application, the decoding output module 1300 includes: a decoding start unit, which is used to input the image feature vector as the initial input vector into the decoder to start step-by-step decoding; a prediction regression unit, which is used for the decoder to predict the corresponding text vector based on the input vector after each decoding step, and autoregressively transfer the text vector to a preset attention layer; a query superposition unit, which is used for the attention layer to query the key feature vector in the image feature information in the process of extracting image features by the picture encoder based on the autoregressive text vector; and an iterative processing unit, which is used to use the key feature vector as the input vector for the next decoding of the decoder to continue to iteratively predict the next text vector until the prediction of each text vector corresponding to the text sequence of the advertising copy is completed.

[0154] Based on any embodiment of the present application, the query superposition unit includes: a feature interaction unit, which uses an autoregressive text vector as a query vector and image feature information of an image encoder as a key vector and a value vector, and interacts the query vector with the key vector to determine a key vector of the product image; a weight optimization unit, which normalizes the key vector to calculate an attention weight and obtain a weight vector; and a feature operation unit, which obtains a key feature vector by weighted summing the value vector with the weight vector.

[0155] On the basis of any embodiment of the present application, prior to the image encoding module 1200, it includes: a network construction module, which is used to construct the image encoder and decoder into an advertising copy creation model, as a generator and a preset discriminator to form a generative adversarial network, the discriminator is pre-trained to a convergence state, and is suitable for binary classification of the quality category of the advertising copy; a generation training module, which is used to freeze the weight of the discriminator, use the product image as the input of the generative adversarial network, calculate the loss value of the discriminator output with the label corresponding to the positive sample, and train the generator to a convergence state.

[0156] On the basis of any embodiment of the present application, different from the previous embodiment, prior to the image encoding module 1200, it includes: a sample construction module, which is used to collect product images, advertising texts and product category identifiers in historical product advertisements, and construct them into mapping relationship data in a data set; an encoder pre-training module, which is used to input the product images in the mapping relationship data of the data set as training samples into the image encoder for pre-training, and uses the corresponding preset product category identifier as the supervision label for the pre-training of the product image to train the image encoder to a convergence state; a decoder pre-training module, which is used to input the advertising texts in the mapping relationship data of the data set as training samples into the decoder for unsupervised pre-training, and train the decoder to a convergence state; a joint fine-tuning training module, which is used to construct the pre-trained image encoder and decoder with the attention layer into an advertising text creation model, and uses the product image as input, and uses the advertising texts corresponding to the product image as the supervision label to train the model to a convergence state.

[0157] In order to solve the above technical problems, the present application also provides a computer device. Fig.10 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a commodity search category identification method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the advertising copywriting method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] In this embodiment, the processor is used to execute Fig. 9 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the advertising copywriting creation device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0159] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the advertising copy creation method of any embodiment of the present application.

[0160] The present application also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any embodiment of the present application when executed by one or more processors.

[0161] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0162] To summarize, the present application converts the image features of the product image into sequence information, generates sequence information consisting of text vectors through the decoder, and then cleverly utilizes the autoregressive mechanism of the decoder to use the text vector of the previous step to determine the key features in the product image, and then appends it as the input of the decoder, and provides the image features of the product image to the decoder in a serialized manner for successive decoding, so that the decoder gradually predicts the text vector sequence corresponding to the product image, and finally obtains the corresponding advertising copy according to the text vector sequence. The advertising copy is obtained based on the image features of the product image, and therefore, the obtained advertising copy can more comprehensively reflect the selling point characteristics in the product image, and the created advertising copy is not affected by the creator's style and has uniform quality. The algorithm model implemented based on this is naturally easier to train, thereby effectively saving training costs.

[0163] Those skilled in the art will appreciate that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, altered, combined, or deleted. Further, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and schemes in the prior art that are similar to those disclosed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted.

[0164] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for creating advertising copy, characterized in that: include: Obtaining a product image of a target product, wherein the target product specified by the request is determined by responding to an advertisement copy prediction request, and after querying and obtaining a product image corresponding to the target product from a product information database, format preprocessing of the product image is completed; the product image provides a description of the selling point characteristics of the target product in the form of an image; A preset image encoder and decoder are constructed as an advertising copywriting model, and a generator and a preset discriminator are constructed as a generative adversarial network, wherein the discriminator is pre-trained to a convergence state and is suitable for binary classification of the quality category of the advertising copy, wherein a text vector sequence is generated by decoding the image feature vector of the product image by the generator, the text vector sequence is decoded to obtain another text vector sequence, and then the other text vector sequence obtained by decoding is combined with the image feature vector extracted from the product image by the image encoder of the discriminator to obtain a merged feature vector, and the quality category is discriminated according to the merged feature vector to obtain the classification probability corresponding to each quality category mapped to the binary classification space; Freeze the weight of the discriminator, use the product image as the input of the generative adversarial network, calculate the loss value of the discriminator output with the label corresponding to the positive sample, and train the generator to a convergence state; The image features of the product image are extracted by the image encoder to obtain an image feature vector of the product image mapped to a specific semantic space, wherein the product image is divided into a plurality of sub-images of equal size to form a sub-image sequence; the image encoder is applied to extract image features from each sub-image in the sub-image sequence to obtain image feature information corresponding to each convolutional layer; the image feature information is processed into an image feature vector input to the adaptive decoder to project it into a specific semantic space; The image feature vector is provided as an initial input vector to a decoder to gradually autoregressively decode each text vector in the text sequence of the advertisement copy, wherein the text vector of the previous step is used to query the key features in the image features of the product image as the input vector corresponding to the text vector of the next step; The text sequence of the advertisement copy is restored based on the text vector sequence predicted by the decoder.

2. The method for creating advertising copy according to claim 1, characterized in that: The image features of the product image are extracted by the image encoder, and the image feature vector of the product image mapped to a specific semantic space is obtained, including: The image features of the product image are gradually extracted through a plurality of convolutional layers preset in the image encoder to obtain image feature information corresponding to each convolutional layer; The image features extracted by the last convolutional layer are fully connected and mapped to a specific semantic space adapted to the input of the decoder to obtain an image feature vector.

3. The method for creating advertising copy according to claim 1, characterized in that: The image feature vector is provided as an initial input vector to the decoder to gradually autoregressively decode each text vector in the text sequence of the advertisement copy, including: Input the image feature vector as an initial input vector to the decoder to start step-by-step decoding; After each decoding step, the decoder predicts the corresponding text vector based on the input vector, and transfers the text vector to the preset attention layer by autoregression; The attention layer queries the key feature vectors in the image feature information extracted by the image encoder based on the autoregressive text vector; The key feature vector is used as the input vector for the next decoding of the decoder to continue iteratively predicting the next text vector until the prediction of each text vector corresponding to the text sequence of the advertisement copy is completed.

4. The method for creating advertising copy according to claim 3, characterized in that: The attention layer queries the key feature vectors in the image feature information extracted by the image encoder based on the autoregressive text vector, including: The autoregressive text vector is used as the query vector, and the image feature information of the image encoder is used as the key vector and value vector. The query vector and the key vector are interacted to determine the key vector of the product image. Normalize the key vector to calculate the attention weight and obtain a weight vector; The key feature vector is obtained by weighted summing the value vector with the weight vector.

5. The method for creating advertising copy according to claim 1, characterized in that: The step of replacing the step of constructing the preset picture encoder and decoder as the advertising copywriting model with the generator and the preset discriminator as the generative adversarial network is performed, including: Collect product images, advertising copy, and product category identifiers from historical product advertisements and construct them into mapping relationship data in the data set; The product images in the mapping relationship data of the data set are used as training samples to input the image encoder for pre-training, and the corresponding preset product category identifiers are used as supervision labels for the pre-training of the product images, and the image encoder is trained to a convergence state; Using the advertisement copy in the mapping relationship data of the data set as training samples, inputting the decoder to perform unsupervised pre-training, and training the decoder to a convergence state; The pre-trained image encoder and decoder are constructed with the attention layer into an advertising copy creation model. The product image is used as input, and the advertising copy corresponding to the product image is used as the supervision label to train the model to a convergence state.

6. An advertising copywriting device, characterized in that: include: The image acquisition module is used to acquire the product image of the target product, wherein the target product specified by the request is determined by responding to the advertising copy prediction request, and the product image corresponding to the target product is obtained from the product information database, and then the format preprocessing of the product image is completed; the product image provides a description of the selling point characteristics of the target product in the form of an image; The image encoding module is used to construct a preset image encoder and decoder into an advertising copy creation model, and to construct a generative adversarial network as a generator and a preset discriminator. The discriminator is pre-trained to a convergence state and is suitable for binary classification to discriminate the quality category of the advertising copy, wherein the text vector sequence generated by the image feature vector of the product image decoded by the generator is decoded to obtain another text vector sequence, and then the other text vector sequence obtained by decoding is combined with the image feature vector extracted from the product image by the image encoder of the discriminator to obtain a merged feature vector, and the quality category is discriminated according to the merged feature vector to obtain a text vector mapped to the binary classification space. The classification probability corresponding to each quality category between the two images is obtained; the weight of the discriminator is frozen, the product image is used as the input of the generative adversarial network, the loss value of the discriminator output is calculated with the label corresponding to the positive sample, and the generator is trained to a convergence state; the image features of the product image are extracted through the image encoder, and the image feature vector of the product image mapped to a specific semantic space is obtained, wherein the product image is divided into a plurality of sub-images of equal specifications to form a sub-image sequence; the image encoder is used to extract image features from each sub-image in the sub-image sequence to obtain image feature information corresponding to each convolution layer; the image feature information is processed into an image feature vector of the input of the adaptive decoder so that it is projected on a specific semantic space; A decoding output module is used to provide the image feature vector as an initial input vector to a decoder to gradually autoregressively decode each text vector in the text sequence of the advertisement copy, wherein the text vector of the previous step is used to query the key features in the image features of the product image as the input vector corresponding to the text vector of the next step; The text restoration module is used to restore the text sequence of the advertising copy based on the text vector sequence predicted by the decoder.

7. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 5 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images