Ad creative selection and model training method and device, equipment and storage medium

Abstract

Embodiments of the present application disclose an advertisement creative selection and model training method and device, equipment and a storage medium. The present application relates to the field of artificial intelligence. The method comprises: obtaining candidate advertisement creative data corresponding to a target object; wherein the candidate advertisement creative data comprises an advertisement picture and an advertisement script; obtaining a sparse feature vector and a picture feature vector corresponding to the candidate advertisement creative data, and obtaining a recommendation probability value corresponding to the candidate advertisement creative data based on the sparse feature vector, the picture feature vector and a pre-trained creative selection model; and selecting target advertisement creative data according to the recommendation probability value. That is, in the embodiments of the present application, the creative selection model can be used to automatically select target advertisement creative data from the collected candidate advertisement creative data, and the picture and the text are fused and processed, the problem of creative optimization based on multi-modal in the advertisement system is solved, and the method has good universality.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for selecting advertising creative ideas and training models. Background Technology

[0002] With the continuous development of artificial intelligence technology, image processing technology and natural language processing technology have been applied to the advertising industry.

[0003] Current methods for selecting advertising creatives generally utilize image processing technology to manipulate image elements within advertising materials and natural language processing models to identify and process text elements. However, these methods are currently domain-specific and lack universality. Furthermore, they cannot directly select the best advertising creative elements for users, thus degrading the user experience. Summary of the Invention

[0004] This invention provides a method, apparatus, medium, and electronic device for selecting advertising creatives and training models, so as to automatically and accurately select the optimal target advertising creative data from candidate advertising creative data.

[0005] In a first aspect, embodiments of the present invention provide a method for selecting advertising creatives based on multimodality, comprising:

[0006] Obtain candidate ad creative data corresponding to the target item; wherein, the candidate ad creative data includes ad images and ad copy;

[0007] Obtain the sparse feature vector and image feature vector corresponding to the candidate ad creative data, and based on the sparse feature vector, the image feature vector and the pre-trained creative selection model, obtain the recommendation probability value corresponding to the candidate ad creative data;

[0008] Select target ad creative data based on the recommended probability value;

[0009] The creative selection model is used to: fuse the sparse feature vector and the image feature vector using a self-attention mechanism, and output the recommendation probability value based on the fusion result.

[0010] Secondly, embodiments of the present invention also provide a model training method, the method comprising:

[0011] Acquire training sample data, which includes sample ad creative data corresponding to sample items and standard recommendation probability values ​​corresponding to the sample ad creative data. The sample ad creative data includes ad images and ad copy.

[0012] Obtain the sparse feature vector and image feature vector corresponding to the sample ad creative data, and based on the sparse feature vector, the image feature vector and the creative selection model to be trained, obtain the predicted recommendation probability value corresponding to the sample ad creative data;

[0013] The loss function is determined based on the standard recommendation probability value and the predicted recommendation probability value. The network parameters in the creative selection model are adjusted based on the loss function, and training is stopped when the preset iteration stopping condition is met.

[0014] The creative selection model is used to: fuse the sparse feature vector and the image feature vector using a self-attention mechanism, and output the predicted recommendation probability value based on the fusion result.

[0015] Thirdly, embodiments of the present invention also provide an advertising creative data selection device, the device comprising:

[0016] The data acquisition module is used to acquire candidate advertising creative data corresponding to the target item; wherein, the candidate advertising creative data includes advertising images and advertising copy;

[0017] The probability value acquisition module is used to obtain the sparse feature vector and image feature vector corresponding to the candidate ad creative data, and obtain the recommendation probability value corresponding to the candidate ad creative data based on the sparse feature vector, the image feature vector and the pre-trained creative selection model.

[0018] The data selection module is used to select target ad creative data based on the recommendation probability value;

[0019] The creative selection model is used to: fuse the sparse feature vector and the image feature vector using a self-attention mechanism, and output the recommendation probability value based on the fusion result.

[0020] Fourthly, embodiments of the present invention also provide a model training apparatus, the apparatus comprising:

[0021] The sample data acquisition module is used to acquire training sample data, which includes sample advertising creative data corresponding to sample items and standard recommendation probability values ​​corresponding to the sample advertising creative data. The sample advertising creative data includes advertising images and advertising copy.

[0022] The vector acquisition module is used to acquire the sparse feature vector and image feature vector corresponding to the sample ad creative data, and based on the sparse feature vector, the image feature vector and the creative selection model to be trained, to obtain the predicted recommendation probability value corresponding to the sample ad creative data.

[0023] The model training module is used to determine a loss function based on the standard recommendation probability value and the predicted recommendation probability value, adjust the network parameters in the creative selection model based on the loss function, and stop training when a preset iteration stopping condition is met.

[0024] The creative selection model is used to: fuse the sparse feature vector and the image feature vector using a self-attention mechanism, and output the predicted recommendation probability value based on the fusion result.

[0025] Fifthly, embodiments of the present invention also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the multimodal advertising creative selection method or model training method as described in any of the embodiments of the present invention.

[0026] Sixthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multimodal advertising creative selection method or model training method as described in any of the embodiments of the present invention.

[0027] In this embodiment of the invention, candidate advertising creative data corresponding to the target item is obtained; wherein, the candidate advertising creative data includes advertising images and advertising copy; sparse feature vectors and image feature vectors corresponding to the candidate advertising creative data are obtained, and based on the sparse feature vectors, image feature vectors, and a pre-trained creative selection model, a recommendation probability value corresponding to the candidate advertising creative data is obtained; target advertising creative data is selected according to the recommendation probability value; wherein, the creative selection model is used to: fuse the sparse feature vectors and image feature vectors using a self-attention mechanism, and output a recommendation probability value based on the fusion result. That is, in this embodiment of the invention, the creative selection model can be used to automatically select target advertising creative data from the collected candidate advertising creative data. This creative selection model can use a self-attention mechanism to fuse the sparse feature vectors, i.e., ID-type features, and image feature vectors, thereby fusing images and text, solving the creative optimization problem based on multimodality in advertising systems, and has good universality. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1A flowchart illustrating a multimodal advertising creative selection method provided in an embodiment of the present invention;

[0030] Figure 2 A schematic diagram of the creative selection model provided in an embodiment of the present invention;

[0031] Figure 3 This is a flowchart of a method for obtaining image feature vectors provided in an embodiment of the present invention;

[0032] Figure 4 A flowchart of a method for obtaining candidate advertising creative data provided in an embodiment of the present invention;

[0033] Figure 5 A flowchart illustrating another method for obtaining candidate ad creative data provided in an embodiment of the present invention;

[0034] Figure 6 This is a flowchart of optimizing candidate advertising creative data provided in an embodiment of the present invention;

[0035] Figure 7 A flowchart illustrating the process of selecting and optimizing advertising creative data provided in an embodiment of the present invention;

[0036] Figure 8 A flowchart of a model training method provided in an embodiment of the present invention;

[0037] Figure 9 This is a schematic diagram of the structure of an advertising creative data selection device provided in an embodiment of the present invention;

[0038] Figure 10 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present invention;

[0039] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0040] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0041] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0042] Figure 1 This is a flowchart illustrating a multimodal advertising creative selection method provided by an embodiment of the present invention. This embodiment can automatically and accurately select the optimal target advertising creative data from candidate advertising creative data. This method can be executed by the advertising creative data selection device in this embodiment, which can be implemented using software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps:

[0043] S110, Obtain candidate ad creative data corresponding to the target item.

[0044] The target item refers to the item for which corresponding ad creative data needs to be generated or selected. There can be multiple candidate ad creative data sets, each describing an ad creative solution for the target item. The candidate ad creative data sets include ad images and ad copy.

[0045] Specifically, on product pages displayed on some product websites and mobile apps, the corresponding advertising material data (ad copy and ad images) can be obtained through the product details page. By performing text recognition and text extraction on the content of the details page, the advertising copy for the target product can be obtained; by performing image recognition and object detection on the content of the details page, the ad image for the target product can be obtained. Alternatively, the advertising material data corresponding to the target product can be obtained from advertising creative materials provided by advertising companies. Further, the advertising material data corresponding to the target product can be filtered according to requirements to obtain candidate advertising creative data for the target product. For example, the target product is a mobile phone. On a mobile phone website, view the content of the details page for various mobile phones. The top of the details page displays images of the mobile phone from various angles, and below the images are the corresponding advertising copy. By performing text recognition and text extraction on the content displayed on the mobile phone details page, the advertising copy at the bottom of the details page can be extracted. The mobile phone images from various angles on the details page can be located to obtain the ad image. If the extracted ad image size is not suitable, the image can be intelligently cropped to obtain the final ad image. After obtaining the advertising creative data, the data is filtered according to requirements (such as click volume, sensitive words, etc.) to obtain candidate advertising creative data.

[0046] S120: Obtain the sparse feature vector and image feature vector corresponding to the candidate ad creative data, and obtain the recommendation probability value corresponding to the candidate ad creative data based on the sparse feature vector, image feature vector and pre-trained creative selection model.

[0047] The creative selection model is used to: output a first feature vector based on a sparse feature vector; fuse the sparse feature vector and the image feature vector using a self-attention mechanism; and output a recommendation probability value based on the fusion result. In this scheme, the creative selection model includes a Multilayer Perceptron (MLP) neural network module, a self-attention module, and an output module. The MLP module outputs a first feature vector based on the sparse feature vector; the self-attention module outputs a second feature vector based on the sparse feature vector and the image feature vector; and the output module outputs a recommendation probability value based on the first and second feature vectors.

[0048] The sparse feature vector is a vector used to reflect multiple types of sparse features. In this scheme, sparse features include item features, user features, and creative features. Item features include item identifiers, ad placement identifiers, brand identifiers, and item category target identifiers. User features include user age, gender, and preferences. Creative features include background template features, copy features, and image features. Background template features include background template identifiers, template style, template layout, and main template color. Copy features include main copy, secondary copy, and speech bubble (a type of copy used to indicate that an item is on sale or selling well), which can be obtained from the ad copy in the ad creative data. Image features include whether the ad image contains text, whether it contains people, and the creative type. The image feature vector is a vector used to reflect the image features of the ad image, which can be obtained from the ad images in the ad creative data. The MLP module maps multiple input feature vectors to a single output feature vector. The self-attention module can quickly extract important features from the sparse feature vector. Optionally, in this scheme, the self-attention module includes the multi-head self-attention module from the Transformer model. The Transformer model is a neural network model that learns the context of data by tracking relationships within sequence data. It includes a multi-head self-attention module. This module extracts feature information from multiple dimensions and boasts high parallelism, enabling it to combine information from different dimensions to capture dependencies across various ranges within the sequence.

[0049] By inputting the sparse feature matrix and image feature vector into the creative selection model, the recommendation probability value corresponding to the candidate ad creative data can be obtained. Figure 2 This is a structural diagram of the creative selection model provided in an embodiment of the present invention. Figure 2As shown, the sparse feature matrix and image feature vector are input into the creative selection model. A vector transformation table can be used to convert the sparse feature matrix into sparse feature vectors. The sparse feature vectors and image feature vectors are then input into the multi-head self-attention module, and the sparse feature vectors are input into the MLP module. Finally, based on the second feature vector output by the multi-head self-attention module and the first feature vector output by the MLP module, the recommendation probability value corresponding to the candidate ad creative data is predicted (output module).

[0050] Before using the creative selection model, it needs to be trained. Specifically, a large number of existing advertising creative schemes are collected, and advertising creative data (background template information, item information, copywriting information, and image information, etc.) is extracted from these schemes. The labeled sparse feature vectors and image feature vectors are used as sample data, and the recommendation probability values ​​(1 or 0) of the labeled sparse feature vectors and image feature vectors are used as sample labels. The sample data is input into the creative selection model to obtain the recommendation probability values ​​predicted by the model corresponding to the sample data. Then, the loss function is calculated using the sample labels and the predicted recommendation probability values. Based on the calculation results, the model parameters of the creative selection model are continuously adjusted and trained to obtain the trained creative selection model.

[0051] The acquisition, storage, use, and processing of data in this application are all authorized by the user and comply with relevant national laws and regulations.

[0052] S130, Select target ad creative data based on the recommendation probability value.

[0053] The target ad creative data is the optimal ad creative data selected from multiple candidate ad creative data. Target ad creative data includes ad copy, ad images, background templates (template style, background color, layout, etc.), and other data related to the target item's advertising strategy. The recommendation probability value is the probability value output by the creative selection model to recommend the corresponding candidate ad creative data to the user. The higher the recommendation probability value, the higher the quality of the corresponding candidate ad creative data in the creative selection model's opinion. Specifically, a preset probability value can be set according to specific needs. When the recommendation probability value is greater than the preset probability value, the corresponding candidate ad creative data is determined as the target ad creative data. Alternatively, the candidate ad creative data with the highest recommendation probability value can be directly selected as the target ad creative data.

[0054] After selecting the target ad creative data, this solution may optionally include the following steps A1-A2:

[0055] Step A1: Obtain the first encoding information of the ad image and the second encoding information of the ad copy in the target ad creative data, and generate the Uniform Resource Locator (URL) corresponding to the target ad creative data based on the first encoding information and the second encoding information.

[0056] A Uniform Resource Identifier (URL) is a concise representation of the location and access method of a resource obtained from the Internet; it is the address of a standard resource on the Internet. Every file on the Internet has a unique URL, which contains information indicating the file's location and how the browser should handle it. URL encoding an ad image using a Uniform Resource Identifier yields the first encoded information, while URL encoding the ad copy yields the second encoded information. Using both the first and second encoded information, an image URL can be generated, which is the URL corresponding to the target ad creative data. The address of the ad creative data can be directly accessed through the URL corresponding to the target ad creative data.

[0057] Step A2: Upon receiving an access request for a URL from the client, retrieve the ad image and ad copy from the target ad creative data based on the URL, perform a combined image operation on the retrieved ad image and ad copy to obtain the target ad creative image, and send the target ad creative image to the client for display.

[0058] Specifically, the URL corresponding to the target ad creative data allows direct access to the ad creative data's address. When a URL access request is received from the client, the ad image and ad copy pointed to by that URL are retrieved. Further, the retrieved ad image and ad copy are combined to obtain a more specific ad image, which is then used as the target ad creative image. For example, image processing software can be used to merge the ad copy and ad image, incorporating information such as the target item's category. During the image merging process, the position and size of the ad copy and ad image can be adjusted appropriately according to specific needs and the actual environment to ultimately obtain the target ad creative image.

[0059] By generating the URL corresponding to the target ad creative data through the above steps, we can save resources occupied by image storage, and the ad image can also be updated at any time as the URL encoding changes, thus improving the efficiency of providing ad creative data to users.

[0060] The technical solution of this embodiment obtains candidate advertising creative data corresponding to the target item; wherein, the candidate advertising creative data includes advertising images and advertising copy. Sparse feature vectors and image feature vectors corresponding to the candidate advertising creative data are obtained, and based on the sparse feature vectors, image feature vectors, and a pre-trained creative selection model, a recommendation probability value corresponding to the candidate advertising creative data is obtained. The creative selection model includes a multilayer perceptron neural network (MLP) module, a self-attention module, and an output module. The MLP module outputs a first feature vector based on the sparse feature vectors, and the self-attention module outputs a second feature vector based on the sparse feature vectors and image feature vectors. The output module outputs a recommendation probability value based on the first and second feature vectors. Target advertising creative data is selected based on the recommendation probability value. This embodiment's solution can automatically select target advertising creative data from the collected candidate advertising creative data using the creative selection model. The self-attention module included in this creative selection model can fuse the ID-type features of the sparse feature vectors with the image feature vectors, thereby fusing images and text, solving the multimodal creative optimization problem in advertising systems, and has good universality.

[0061] Figure 3 This is a flowchart of a method for obtaining image feature vectors according to an embodiment of the present invention. This embodiment refines the method for obtaining image feature vectors based on the above embodiment. Figure 3 As shown, the method in this embodiment specifically includes the following steps:

[0062] S210, input the ad images from the candidate ad creative data into the pre-trained residual neural network model.

[0063] The candidate ad creative data includes ad copy and ad images. For example... Figure 2 As shown, the image feature vector needs to be input into the multi-head self-attention module. Therefore, the ad images in the candidate ad creative data need to be input into the pre-trained residual neural network model to obtain the image feature vector. In this embodiment, optionally, the residual neural network model is trained, including the following steps B1-B3:

[0064] Step B1: Obtain the sample image and its corresponding category label.

[0065] The category labels are the product terms of the sample items contained in the sample images. Product terms are words used to characterize the category of the sample items and do not contain brand information. Existing methods for training residual neural network models use the category terms of the items as category labels. The category terms include brand information, while product terms can be used to indicate the category of the item but do not contain brand information. For example, if the sample item is a mobile phone from brand A, the corresponding category terms would include both "mobile phone" and "brand A," and the corresponding product terms would only include "mobile phone." The sample images are advertising images from existing advertising campaigns for the sample items.

[0066] Specifically, sample images and their corresponding category tags can be obtained from product detail pages on websites or mobile apps. Alternatively, sample images and their corresponding category tags can be obtained from advertising creative libraries provided by advertisers or professionals. For example, based on the exposure volume of all items in the product detail page / ad creative library, sorted from high to low, the item category information corresponding to the top 10,000 items in terms of exposure volume can be selected, and this category information can be used as the sample tag. Using product terms as sample tags is suitable for large-scale, multi-classification tasks, which can improve the generalization of image feature vectors generated by residual neural network models and further avoid the inadequacy of image content representation due to overly concentrated item category information. Moreover, in practical applications, for advertising data, item category information is often more important than item brand information. For example, advertising campaigns for mobile phones and clothing are very different. Mobile phone advertising campaigns may need to emphasize the advertising copy (description of the phone's performance, etc.), while clothing advertising campaigns may need to emphasize the advertising image. However, for the same type of item, even for different brands, the advertising campaigns are mostly the same. For example, the differences in advertising campaigns for different mobile phone brands might only lie in the content described in the advertising copy. Therefore, using product terms of sample items as category tags can more closely reflect reality and further improve the accuracy of the recommendation probability values ​​of the creative selection model.

[0067] Step B2: Input the sample image into the residual neural network model and obtain the predicted classification output by the residual neural network model.

[0068] Residual neural network (RNN) models are a type of convolutional neural network (CNN) model, such as the ResNet model. RNN models are widely used for image classification and object recognition. RNN models are easy to optimize; accuracy can be improved by increasing the network depth. The residual blocks within RNNs use skip connections, mitigating the vanishing gradient problem that arises when increasing depth in deep neural networks. Specifically, sample images are input into the RNN model, which then calculates and infers to predict the category corresponding to the sample image. In this embodiment, optionally, the tail of the RNN model includes three fully connected layers for outputting 32-dimensional, 128-dimensional, and 256-dimensional vectors, respectively; that is, these three fully connected layers are added to the tail of the existing ResNet model.

[0069] Specifically, residual neural network models include convolutional layers, pooling layers, activation functions, and fully connected layers. The operations of convolutional layers, pooling layers, and activation functions map the original data to the feature space of the hidden layers, obtaining feature vectors. Fully connected layers map the feature vectors, representing distributed features, to the sample label space. Fully connected layers can extract features from the feature vectors and classify sample images based on these feature vectors. Depending on the size of the sample data and the classification requirements, different dimensions of the output vector can be set for the residual neural network model. Using a residual neural network model with three fully connected layers at the tail, containing output vectors of 32, 128, and 256 dimensions, the category of a sample image can be predicted flexibly and accurately according to business needs and the size of the sample data.

[0070] Step B3: Determine the loss function based on the predicted classification and classification label, adjust the network parameters in the residual neural network model based on the loss function, and stop training when the preset iteration stopping condition is met.

[0071] In this process, prediction classification involves inputting sample images into a residual neural network model, which then calculates and predicts the category of the sample image. The classification label represents the true category of the sample image.

[0072] The loss function calculates the "gap" between the predicted classification and the sample label. Based on the loss function determined by the predicted classification and label, the network parameters in the residual neural network model can be continuously adjusted to make the predicted classification and label increasingly closer until a preset iteration stopping condition is met, at which point training stops. This preset stopping condition includes the residual neural network model's prediction accuracy reaching a preset accuracy range. In this embodiment, the preset accuracy range may optionally include [75%, 80%]. Specifically, the higher the prediction accuracy of the residual neural network model, the more accurate the predicted category of the sample image. However, this also increases the computational complexity of the residual neural network model, leading to slower computation speeds and potential overfitting in practical applications. Therefore, to improve the prediction accuracy of the residual neural network model and avoid excessive computational complexity, the preset accuracy range can be set to [75%, 80%]. Training of the residual neural network model can be stopped when the prediction accuracy is not less than 75% and not greater than 80%.

[0073] In the above steps, using product terms as category labels for sample items can improve the accuracy of the recommendation probability value of the creative selection model. By utilizing a residual neural network model with three fully connected layers at the tail containing 32-dimensional, 128-dimensional, and 256-dimensional output vectors, the category of the sample image can be predicted flexibly and accurately based on business needs and the size of the sample data, further improving the accuracy of the recommendation probability value of the creative selection model.

[0074] S220: Obtain the image feature vector output by the residual neural network model.

[0075] In this approach, the image feature vector is a vector representing some features of an image. The image feature vector output by the residual neural network model can be used to represent the image category. For example, when a picture of a mobile phone is input into the residual neural network model, the model predicts the image's feature as "Category: Mobile Phone Image" and outputs an image feature vector with the "Mobile Phone" feature. Further, the image feature vector output by the residual neural network model is obtained.

[0076] The technical solution of this embodiment inputs the advertising images from the candidate advertising creative data into a pre-trained residual neural network model; and obtains the image feature vector output by the residual neural network model. This embodiment's solution can flexibly and accurately predict the category of the sample image based on business needs and the size of the sample data, and further uses the product terms of the sample items as classification tags, thereby improving the accuracy of the recommendation probability value of the creative selection model.

[0077] Figure 4 This is a flowchart illustrating a method for obtaining candidate advertising creative data according to an embodiment of the present invention. This embodiment refines the method for obtaining candidate advertising creative data based on the above embodiments. Figure 4 As shown, the method in this embodiment specifically includes the following steps:

[0078] S310, Obtain multiple advertising material data corresponding to the target item, including advertising copy and advertising images.

[0079] Specifically, on product pages displayed on some websites and mobile apps, the corresponding advertising material data (ad copy and images) can be obtained through the product details page. The advertising copy is obtained by performing text recognition and text extraction on the content of the details page; the advertising image is obtained by performing image recognition and image cropping on the content of the details page. Alternatively, the advertising material data corresponding to the product can be obtained from advertising creative materials provided by advertising companies.

[0080] S320: Based on the online click data corresponding to each advertising creative data, select at least one advertising copy and at least one advertising image from each advertising creative data.

[0081] Online click data represents the number of clicks on advertising creative data, reflecting users' liking for the advertising creative. The more clicks an advertising creative receives, the more likely it is to be liked by more people. Therefore, based on the online click data corresponding to each advertising creative data set, at least one advertising copy and at least one advertising image can be selected from each advertising creative data set. In this embodiment, optionally, selecting at least one advertising copy and at least one advertising image from each advertising creative data set includes the following steps C1-C2:

[0082] Step C1: For each ad creative data, determine the score of the ad creative data based on the average online click-through rate and the cumulative number of times the ad creative data has been selected.

[0083] The online click-through rate of ad creative data can be determined by the number of views and clicks on the target item's details page. Specifically, a multi-armed bandits (MAB) model can be used, employing an upper confidence bound (UCB) algorithm, to offline calculate the score of ad creative data for the past month:

[0084]

[0085] in, n represents the average online click-through rate of the advertising creative data. j This represents the cumulative number of times the current ad creative data has been selected, where n represents the number of ad creative data. The higher the average online clicks and the higher the cumulative number of times the ad creative data has been selected, the higher the ad creative data score. Conversely, the lower the average online clicks and the lower the cumulative number of times the ad creative data has been selected, the lower the ad creative data score.

[0086] Furthermore, based on the above formula, the scores for the advertising copy and advertising images in the advertising creative are calculated separately. Specifically, after selecting at least one advertising copy and at least one advertising image from each advertising creative data, the selected advertising copy forms a copy group, and the score for each advertising copy in the file group is calculated. When calculating the score for the advertising copy, n represents the average online click-through rate of the ad copy within the copywriting group. j This represents the cumulative number of times the current ad copy has been selected, where n represents the number of ad copy entries in the ad copy group. Further, the selected ad images are grouped into image groups, and the score for each ad image in each image group is calculated. When calculating the score for each ad image, n represents the average online click-through rate of the ad images in the image group. j It represents the cumulative number of times the current ad image has been selected, and n represents the number of ad images in the image group.

[0087] Step C2: Based on the scores of each advertising creative data, select at least one advertising copy and at least one advertising image from each advertising creative data.

[0088] Specifically, after obtaining the scores for the ad images and ad copy, at least one ad image is selected from the ad creative data based on the ad image score. At least one ad copy is selected from the ad creative data based on the ad copy score.

[0089] By using the above steps, the score of the advertising creative data can be accurately calculated. Combining the score of the advertising creative data with the optimization of the advertising creative data can accurately and quickly select suitable advertising creative data and avoid the combination explosion problem that may occur during the combination of advertising creative data.

[0090] S330: Combine the selected advertising copy and advertising images to obtain at least one candidate advertising creative data.

[0091] After selecting at least one ad copy and at least one ad image, the selected ad copy and ad image are further combined in pairs to obtain at least one candidate ad creative data. For example, based on the scores of each ad creative data, the ad copy selected from each ad creative data is copy A and copy B. The ad image selected from each ad creative data is image C. Then, based on the ad copy and ad image, the candidate ad creative data can be AC ​​and BC.

[0092] The technical solution of this embodiment acquires multiple advertising material data corresponding to the target item. The advertising material data includes advertising copy and advertising images. Based on the online click data corresponding to each advertising material data, at least one advertising copy and at least one advertising image are selected from each advertising material data. The selected advertising copy and advertising image are combined to obtain at least one candidate advertising creative data. This embodiment's solution can accurately calculate the score of the advertising material data. By combining the scores of the advertising material data, suitable advertising material data can be selected accurately and quickly, avoiding the combination explosion problem that occurs during the combination of advertising material data. Furthermore, the candidate advertising creative data obtained by combining the selected advertising copy and advertising image is more accurate and better suited to user preferences.

[0093] Figure 5 This is a flowchart illustrating another method for obtaining candidate ad creative data provided by an embodiment of the present invention. This embodiment refines the method for obtaining candidate ad creative data based on the above embodiments. Figure 5 As shown, the method in this embodiment specifically includes the following steps:

[0094] S410, Identify and extract advertising copy from the item details page and / or advertising creative materials of the target item.

[0095] Specifically, in the product pages displayed on some product websites and mobile apps, the advertising material data corresponding to the product can be obtained through the product details page. Furthermore, the advertising copy can be extracted from the advertising material data. In this embodiment, optionally, identifying and extracting the advertising copy from the target product's product details page and / or advertising creative materials includes the following steps D1-D3:

[0096] Step D1: Based on a preset character recognition model, identify the candidate text from the item details page and / or advertising creative materials of the target item.

[0097] The character recognition model includes the Optical Character Recognition (OCR) model. Specifically, in the item details page and / or advertising creative materials of the target product, the OCR model can be used to recognize and extract the text in the item details page and / or advertising creative materials to obtain the advertising text, and the recognized advertising text is used as the candidate text.

[0098] Step D2: Based on the first thesaurus containing preset benefit words, select benefit-point copy from the candidate copy.

[0099] Among them, "benefit point vocabulary" refers to the words used in advertising copy to express the characteristics, benefits / advantages, consumer benefits, emotions / values, etc., of an item to the user. A pre-defined benefit point vocabulary list is a table that records the benefit point vocabulary of the target item, set according to specific needs and the actual environment. For example, if the target item is a camera, the first vocabulary list would include:

[0100] Item characteristics: small size, high pixel count

[0101] Product benefits / advantages: Easily takes clear and beautiful photos.

[0102] Consumer benefits: Portable and easy to operate

[0103] Emotions / Values: Recording life, showcasing the most authentic world.

[0104] Furthermore, after obtaining the candidate texts, the benefit-point words are selected from the candidate texts based on the first word list.

[0105] Step D3: Based on the preset word count limit and / or a second vocabulary list containing preset non-selling point words, select selling point copy from the remaining copy after removing the benefit copy from the candidate copy.

[0106] Among them, selling point copy is valuable copy that can enhance users' purchase interest and promote product sales. Selling point copy uses simple language to describe the product's selling points, therefore it has a certain word limit. Specifically, selling point copy can be filtered from the remaining copy after removing benefit-related copy based on a preset word limit. However, in some cases, advertising copy that meets the word limit may not necessarily be selling point copy. In this case, selling point copy can be filtered from the remaining copy after removing benefit-related copy based on a second vocabulary list containing preset non-selling point words. To further refine the filtering of selling point copy, it can be further filtered from the remaining copy after removing benefit-related copy based on both the preset word limit and the second vocabulary list. For example, if the preset word limit is 5, then firstly, remove the benefit-point copy from the candidate copy and delete the remaining copy that has more than 5 words. Then, filter the remaining copy according to the second word list and delete the copy that contains non-selling point words, and finally get the selling point copy.

[0107] In the above steps, an OCR model can be used to accurately and quickly mine and identify the advertising copy in the product details page. The final benefit copy and selling point copy can be obtained through the first and second word lists, which can be used to solve the problem of insufficient online copywriting materials.

[0108] S420: Locate and crop the item image in the item details page of the target item to obtain the advertising image.

[0109] This process involves locating and representing item images on the item details page to accurately pinpoint their location. Specifically, since the location of the target item image on the details page is uncertain, a saliency algorithm can be used to divide the page into several specific regions with unique characteristics (such as text and image areas). The item image is then identified within these segmented image regions, and its size is analyzed for intelligent cropping. For example, if the main item in an image is too small for the user to clearly see, the image can be cropped to an appropriate size. Furthermore, the cropped image is used as an advertising image.

[0110] S430: Based on the online click data corresponding to each advertising creative data, select at least one advertising copy and at least one advertising image from each advertising creative data.

[0111] S440: Combine the selected advertising copy and advertising images to obtain at least one candidate advertising creative data.

[0112] The technical solution of this embodiment identifies and extracts advertising copy from the item details page and / or advertising creative materials of the target item; locates and crops the item images in the item details page of the target item to obtain advertising images; selects at least one advertising copy and at least one advertising image from each advertising creative data according to the online click data corresponding to each advertising creative data; and combines the selected advertising copy and advertising images to obtain at least one candidate advertising creative data. The technical solution of this embodiment can accurately and quickly mine and identify advertising copy in the item details page, solving the problem of insufficient online copy materials through selling point copy. By intelligently cropping the advertising images, a target item image that highlights the target item is obtained, allowing the advertising image to fully display the target item.

[0113] Figure 6 This is a flowchart illustrating the optimization process for candidate ad creative data provided in this embodiment of the invention. This embodiment further refines the method for optimizing candidate ad creative data based on the aforementioned embodiments. For example... Figure 6 As shown, the method in this embodiment specifically includes the following steps:

[0114] S510: Combine the selected advertising copy and advertising images to obtain at least one copy-image combination.

[0115] Specifically, after selecting at least one ad copy and at least one ad image from each ad creative data set, the selected ad copy and ad image are paired to obtain at least one copy-image combination. For example, based on the scores of each ad creative data set, the selected ad copy is copy A and copy B. The selected ad image is image C. Then, based on the ad copy and ad image, the copy-image combinations can be AC ​​and BC.

[0116] S520, combine at least one text image combination with at least one preset background template to obtain at least one creative combination.

[0117] The preset background templates are layout templates for text and image combinations with a fixed style, pre-set according to the characteristics of the items and specific needs. Specifically, after obtaining the text and image combination, at least one background template can be selected for the text and image combination based on the category information and characteristics of the items. The text and image combination is then paired with at least one preset background template to obtain at least one creative combination.

[0118] S530, based on preset screening factors, selects at least one creative combination from each creative combination as candidate advertising creative data.

[0119] The preset filtering factors include the category information of the target item and / or the color information of the ad images and background templates in each creative combination. Specifically, to obtain ad creative data more suitable for the target item, candidate ad creative data needs to be filtered based on the item's category information and / or the color information of the ad images and background templates in each creative combination. This color information can include the dominant color. For example, the K-Means clustering algorithm can be used to perform cluster analysis and extract the dominant color of the image to identify its dominant color.

[0120] When selecting at least one creative combination as candidate ad creative data from various creative combinations based on the category information of the target item, specifically, the background template style corresponding to the target item can be determined according to the category information of the target item and the pre-set correspondence between the item category and the background template style, and then the creative combination that matches the background template style can be selected from each creative combination.

[0121] When selecting at least one creative combination as candidate ad creative data based on the color information of the ad image and background template in each creative combination, specifically, it can be based on the main color of the ad image and the main color of the background template, using the HSV color model, and selecting at least one creative combination as candidate ad creative data according to the method of giving priority to the use of adjacent color matching and contrasting color matching.

[0122] S540, if the size of the target item area in the ad image contained in the candidate ad creative data is less than a preset threshold, the target item area is cropped, and the cropped target item image is used to update the ad image contained in the candidate ad creative data.

[0123] The preset threshold can be set in advance according to specific needs. Specifically, if the size of the target item area in the ad image is smaller than the preset threshold, the target item may be too small for users to see clearly. Object detection algorithms can be used to identify the target item area in the ad image. Furthermore, the ad image can be intelligently cropped to obtain a target item image that highlights the target item, and the ad images included in the candidate ad creative data can be updated based on this target item image.

[0124] S550 refines the target item area in the advertisement image based on the color information of the advertisement image in the candidate advertisement creative data.

[0125] The retouching process includes adjusting brightness, contrast, and saturation at least one of these. Specifically, after obtaining the updated candidate ad creative data containing the ad image, image analysis is performed on the ad image based on the color of the ad image and the color of the target item area. When the color of the target item area is too dark, it may result in the target item area not being prominent enough. In this case, the brightness of the target item area can be increased to highlight the target item and attract users to click or trigger the target item. Similarly, when the color contrast between the target item area and the ad image is weak, it may result in the target item area blending into the background of the ad image. In this case, the contrast can be adjusted to highlight the target item. When the color saturation between the target item area and the ad image is weak, the saturation can be adjusted to highlight the target item. In this embodiment, optionally, the retouching process for the target item area in the ad image based on the color information of the ad image in the candidate ad creative data includes the following steps E1-E2:

[0126] Step E1: Determine whether the ad image is a color image or a black and white image based on the pixel values ​​of the pixels contained in the ad image in the candidate ad creative data.

[0127] The pixel information in an image reflects its color information. Specifically, the pixel values ​​of the pixels in the advertisement image are statistically analyzed. If there are pixels with a value greater than 190 (pixel values ​​range from 0-255), and the number of pixels with a value greater than 190 exceeds 50% of the total pixels, the advertisement image is determined to be a white image. If the number of pixels with a value greater than 190 does not exceed 15% of the total pixels, and the number of pixels with a value less than or equal to 55 exceeds 50% of the total pixels, the advertisement image is determined to be a black image. Otherwise, the advertisement image is determined to be a color image. When an advertisement image is determined to be a white image, no color enhancement processing is performed.

[0128] Step E2: If the advertisement image is a color image, the target item area in the advertisement image is retouched based on the first preset brightness parameter value, the first preset contrast parameter value, and the first preset saturation parameter value.

[0129] The first preset brightness parameter value, first preset contrast parameter value, and first preset saturation parameter value can be preset according to specific needs. For example, when the advertisement image is a color image, the first preset brightness parameter value can be set to 15, the first preset contrast parameter value to 10, and the first preset saturation parameter value to 10. Furthermore, based on the aforementioned first preset brightness parameter value, first preset contrast parameter value, and first preset saturation parameter value, the target item area in the advertisement image is refined.

[0130] Step E3: When the advertisement image is a black and white image, the target item area in the advertisement image is retouched based on the second preset brightness parameter value, the second preset contrast parameter value, and the second preset saturation parameter value.

[0131] The second preset brightness parameter value is greater than the first preset brightness parameter value, the second preset contrast parameter value is greater than the first preset contrast parameter value, and the second preset saturation parameter value is greater than the first preset saturation parameter value. These values ​​can be preset according to specific needs. When the advertising image is a black and white image, since black images do not provide significant image enhancement, strong enhancement processing can be applied. For example, the second preset brightness parameter value can be set to 20, the second preset contrast parameter value to 15, and the second preset saturation parameter value to 15. Further, based on the aforementioned second preset brightness parameter value, second preset contrast parameter value, and second preset saturation parameter value, enhancement processing is applied to the target item area in the advertising image.

[0132] By following the steps above, the colors of the advertising images can be enhanced to make the advertised items more attractive and fully utilize the advertising images to showcase the items.

[0133] Figure 7 This is a flowchart illustrating the process of selecting and optimizing advertising creative data according to an embodiment of the present invention. Figure 7 As shown, the advertising copy for the target item is identified and extracted from the item details page and / or advertising creative library using the OCR model in the creative mining module. The advertising image for the target item is identified using the image segmentation model and object detection model in the creative mining module. The scores for the advertising copy and advertising image are calculated using MAB (Multi-Aspect Ratio). Based on the scores, at least one advertising copy and at least one advertising image are selected from each advertising creative data set. Candidate advertising creative data is obtained through the creative element combination module, and then input into the creative selection module. The output value of the creative selection model in the creative selection module is used to obtain the target advertising creative data.

[0134] This embodiment combines selected advertising copy and images to obtain at least one copy-image combination; it then combines this at least one copy-image combination with at least one preset background template to obtain at least one creative combination; based on preset screening factors, it selects at least one creative combination as candidate advertising creative data; if the size of the target item area in the advertising image included in the candidate advertising creative data is less than a preset threshold, the target item area is cropped, and the cropped target item image is used to update the advertising image included in the candidate advertising creative data; finally, it refines the target item area in the advertising image based on the color information of the advertising image in the candidate advertising creative data. This embodiment can filter creative combinations based on the category information and color information of the target item, resulting in more aesthetically pleasing candidate advertising creative data with colors closer to those of a human designer. Finally, it refines the advertising image based on its color, making the advertising item in the image more attractive and fully leveraging the advertising image's role in showcasing the item.

[0135] Figure 8 This is a flowchart illustrating a model training method provided in an embodiment of the present invention. This embodiment can train an initial model to obtain a creative selection model. This method can be executed by the model training device in this embodiment of the present invention, which can be implemented in software and / or hardware, such as... Figure 8 As shown, the method specifically includes the following steps:

[0136] S610, acquire training sample data.

[0137] The training sample data includes sample ad creative data corresponding to sample items and standard recommendation probability values ​​corresponding to the sample ad creative data. The sample ad creative data includes ad images and ad copy. In an optional implementation, sample data can be obtained from a historical creative database that stores sample ad creative data and corresponding standard recommendation probability values. For example, based on big data and data analysis algorithms, some ad creative data and their corresponding standard recommendation probability values ​​can be determined, and these ad creative data and their corresponding standard probability recommendation values ​​can be stored in the historical creative database. Further, sample data can be obtained from the historical creative database.

[0138] S620: Obtain the sparse feature vector and image feature vector corresponding to the sample ad creative data, and based on the sparse feature vector, image feature vector and the creative selection model to be trained, obtain the predicted recommendation probability value corresponding to the sample ad creative data.

[0139] Here, the sparse feature vector is a vector reflecting multiple types of sparse features. The image feature vector is a vector reflecting the image features of the ad image, which can be obtained from the ad images in the sample ad creative data. The predicted recommendation probability value is the recommendation probability value corresponding to the sparse feature vector and the image feature vector, calculated by the untrained creative selection model based on the sparse feature vector and the image feature vector. Specifically, the sparse feature vector and the image feature vector are input into the creative selection model to be trained, and the model can output the predicted recommendation probability value corresponding to the sample ad creative data after calculation.

[0140] S630 determines the loss function based on the standard recommendation probability value and the predicted recommendation probability value, adjusts the network parameters in the creative selection model based on the loss function, and stops training when the preset iteration stopping condition is met.

[0141] The loss function maps the values ​​of a random event or its related random variables to non-negative real numbers to represent the "risk" or "loss" of that random event. The network parameters in the model are configuration variables within the model, and their values ​​can be adjusted according to the loss function. The creative selection model uses a self-attention mechanism to fuse sparse feature vectors and image feature vectors, and outputs a predicted recommendation probability value based on the fusion result. Optionally, in this scheme, the creative selection model includes a Multilayer Perceptron Neural Network (MLP) module, a self-attention module, and an output module. The MLP module outputs a first feature vector based on the sparse feature vector; the self-attention module outputs a second feature vector based on the sparse feature vector and the image feature vector; and the output module outputs the predicted recommendation probability value based on the first and second feature vectors.

[0142] Specifically, sparse feature vectors and image feature vectors are input into the creative selection model to be trained, resulting in corresponding predicted recommendation probabilities. At this point, there is a significant "gap" between the predicted recommendation probabilities and the standard recommendation probabilities. The creative selection model is continuously optimized based on the loss function and this "gap." By adjusting the network parameters of the creative selection model, the "gap" between the predicted and standard recommendation probabilities is continuously reduced. When a preset stopping condition is met, the trained creative selection model is obtained.

[0143] The technical solution of this embodiment can acquire training sample data, obtain sparse feature vectors and image feature vectors corresponding to sample ad creative data, and obtain predicted recommendation probability values ​​corresponding to the sample ad creative data based on the sparse feature vectors, image feature vectors, and the creative selection model to be trained. A loss function is determined based on the standard recommendation probability value and the predicted recommendation probability value. The network parameters in the creative selection model are adjusted based on the loss function, and training stops when a preset iteration stopping condition is met. The technical solution of this embodiment can continuously optimize the creative selection model, making the predicted recommendation probability value output by the creative selection model closer to the standard recommendation probability value, thereby improving the accuracy of the predicted recommendation probability value.

[0144] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0145] Figure 9 This is a schematic diagram of an advertising creative data selection device provided in an embodiment of the present invention. This embodiment can automatically and accurately select the optimal target advertising creative data from candidate advertising creative data. The device can be implemented using software and / or hardware, and can be integrated into any device that provides advertising creative data selection functionality, such as… Figure 9 As shown, the device for selecting advertising creative data specifically includes:

[0146] The data acquisition module 910 is used to acquire candidate advertising creative data corresponding to the target item; wherein, the candidate advertising creative data includes advertising images and advertising copy;

[0147] The probability value acquisition module 920 is used to obtain the sparse feature vector and image feature vector corresponding to the candidate ad creative data, and obtain the recommendation probability value corresponding to the candidate ad creative data based on the sparse feature vector, the image feature vector and the pre-trained creative selection model.

[0148] Data selection module 930 is used to select target advertising creative data based on the recommendation probability value;

[0149] The creative selection model is used to: fuse the sparse feature vector and the image feature vector using a self-attention mechanism, and output the recommendation probability value based on the fusion result.

[0150] The creative selection model includes a multilayer perceptron neural network (MLP) module, a self-attention module, and an output module; wherein:

[0151] The MLP module is used to output a first feature vector based on the sparse feature vector;

[0152] The self-attention module is used to output a second feature vector based on the sparse feature vector and the image feature vector;

[0153] The output module is used to output the recommendation probability value based on the first feature vector and the second feature vector.

[0154] Optionally, the probability value acquisition module 920 is specifically used for:

[0155] The ad images from the candidate ad creative data are input into a pre-trained residual neural network model;

[0156] Obtain the image feature vector output by the residual neural network model.

[0157] Optionally, the probability value acquisition module 920 is also used for:

[0158] Obtain sample images and corresponding category tags for the sample images; wherein the category tags are product terms for the sample items contained in the sample images, and the product terms are words used to characterize the type of the sample items and do not contain brand information;

[0159] Input the sample image into the residual neural network model to obtain the predicted classification output by the residual neural network model;

[0160] A loss function is determined based on the predicted classification and the classification label. The network parameters in the residual neural network model are adjusted based on the loss function, and training stops when a preset iteration stopping condition is met.

[0161] Optionally, the preset iteration stopping condition includes the prediction accuracy of the residual neural network model reaching a preset accuracy range, which may include [75%, 90%].

[0162] Optionally, the tail of the residual neural network model includes three fully connected layers for outputting 32-dimensional vectors, 129-dimensional vectors, and 256-dimensional vectors, respectively.

[0163] Optionally, the self-attention module includes the multi-head self-attention module in the Transformer model.

[0164] Optionally, the data acquisition module 910 is specifically used for:

[0165] Acquire multiple advertising creative data corresponding to the target item, wherein the advertising creative data includes advertising copy and advertising images;

[0166] Based on the online click data corresponding to each of the aforementioned advertising material data, at least one advertising copy and at least one advertising image are selected from each of the aforementioned advertising material data;

[0167] The selected advertising copy and images are combined to obtain at least one candidate advertising creative data.

[0168] Optionally, the data acquisition module 910 is also used for:

[0169] Identify and extract advertising copy from the item details page and / or advertising creative materials of the target item;

[0170] The item image on the item details page of the target item is located and cropped to obtain the advertising image.

[0171] Optionally, the data acquisition module 910 is also used for:

[0172] Based on a preset character recognition model, candidate text is identified from the item details page and / or advertising creative materials of the target item;

[0173] Based on a first thesaurus containing preset benefit-point words, benefit-point texts are selected from the candidate texts;

[0174] Based on preset word count limits and / or a second vocabulary list containing preset non-selling point words, selling point copy is selected from the remaining copy after removing the benefit-point copy from the candidate copy.

[0175] Optionally, the data acquisition module 910 is also used for:

[0176] For each of the aforementioned ad creative data, the score of the ad creative data is determined based on the average online click-through rate of the ad creative data and the cumulative number of times the ad creative data has been selected;

[0177] Based on the scores of each of the aforementioned advertising creative data, at least one advertising copy and at least one advertising image are selected from each of the aforementioned advertising creative data.

[0178] Optionally, the data acquisition module 910 is also used for:

[0179] Combine the selected advertising copy and advertising images to obtain at least one copy-image combination;

[0180] The at least one text image combination is combined with at least one preset background template to obtain at least one creative combination;

[0181] Based on preset screening factors, at least one creative combination is selected from each of the creative combinations as candidate advertising creative data; wherein, the preset screening factors include the category information of the target item, and / or the color information of the advertising image and background template in each of the creative combinations.

[0182] Optionally, the data acquisition module 910 is also used for:

[0183] If the size of the target item area in the ad image contained in the candidate ad creative data is less than a preset threshold, the target item area is cropped, and the cropped target item image is used to update the ad image contained in the candidate ad creative data.

[0184] Optionally, the data acquisition module 910 is also used for:

[0185] Based on the color information of the advertising image in the candidate advertising creative data, the target item area in the advertising image is refined; wherein, the refining process includes at least one of adjusting brightness, adjusting contrast, and adjusting saturation.

[0186] Optionally, the data acquisition module 910 is also used for:

[0187] Based on the pixel values ​​of the pixels contained in the ad image in the candidate ad creative data, determine whether the ad image is a color image or a black and white image;

[0188] When the advertisement image is a color image, the target item area in the advertisement image is retouched based on the first preset brightness parameter value, the first preset contrast parameter value, and the first preset saturation parameter value.

[0189] When the advertisement image is a black and white image, the target item area in the advertisement image is retouched based on the second preset brightness parameter value, the second preset contrast parameter value, and the second preset saturation parameter value.

[0190] Wherein, the second preset brightness parameter value is greater than the first preset brightness parameter value, the second preset contrast parameter value is greater than the first preset contrast parameter value, and the second preset saturation parameter value is greater than the first preset saturation parameter value.

[0191] Optionally, the device is also used for:

[0192] Obtain the first encoding information of the ad image and the second encoding information of the ad copy in the target ad creative data, and generate the Uniform Resource Locator URL corresponding to the target ad creative data based on the first encoding information and the second encoding information;

[0193] Upon receiving an access request for the URL from the client, the system retrieves the ad image and ad copy from the target ad creative data based on the URL, performs a composite image operation on the retrieved ad image and ad copy to obtain the target ad creative image, and sends the target ad creative image to the client for display.

[0194] The above-described products can perform the methods provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for performing the methods.

[0195] Figure 10 This is a schematic diagram of a model training device provided in an embodiment of the present invention. The device can be implemented in software and / or hardware, and can be integrated into any device that provides model training functionality, such as… Figure 10 As shown, the model training device specifically includes:

[0196] The sample data acquisition module 1010 is used to acquire training sample data, which includes sample advertising creative data corresponding to the sample items and standard recommendation probability values ​​corresponding to the sample advertising creative data. The sample advertising creative data includes advertising images and advertising copy.

[0197] The vector acquisition module 1020 is used to acquire the sparse feature vector and image feature vector corresponding to the sample ad creative data, and obtain the predicted recommendation probability value corresponding to the sample ad creative data based on the sparse feature vector, the image feature vector and the creative selection model to be trained.

[0198] The model training module 1030 is used to determine a loss function based on the standard recommendation probability value and the predicted recommendation probability value, adjust the network parameters in the creative selection model based on the loss function, and stop training when a preset iteration stopping condition is met.

[0199] The creative selection model is used to: employ a self-attention mechanism to fuse the sparse feature vector and the image feature vector, and output the predicted recommendation probability value based on the fusion result.

[0200] The creative selection model includes a multilayer perceptron neural network (MLP) module, a self-attention module, and an output module; wherein:

[0201] The MLP module is used to output a first feature vector based on the sparse feature vector;

[0202] The self-attention module is used to output a second feature vector based on the sparse feature vector and the image feature vector;

[0203] The output module is used to output the predicted recommendation probability value based on the first feature vector and the second feature vector.

[0204] The above-described products can perform the methods provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for performing the methods.

[0205] Figure 11This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 11 A block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present invention is shown. Figure 11 The computer device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0206] like Figure 11 As shown, the computer device 12 is represented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0207] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0208] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0209] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 11 Not shown; usually referred to as a "hard drive"). Although Figure 11 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0210] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0211] The computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via the input / output (I / O) interface 22. Furthermore, in this embodiment, the display 24 of the computer device 12 is not an independent entity, but is embedded in a mirror, so that when the display surface of the display 24 is not displayed, the display surface of the display 24 and the mirror surface visually blend together. Moreover, the computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via the network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer device 12 via the bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0212] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a multimodal advertising creative selection method provided in this embodiment of the invention: acquiring candidate advertising creative data corresponding to a target item; wherein, the candidate advertising creative data includes advertising images and advertising copy; acquiring sparse feature vectors and image feature vectors corresponding to the candidate advertising creative data, and obtaining recommendation probability values ​​corresponding to the candidate advertising creative data based on the sparse feature vectors, the image feature vectors, and a pre-trained creative selection model; wherein, the creative selection model is used to: fuse the sparse feature vectors and the image feature vectors using a self-attention mechanism, and output the recommendation probability value based on the fusion result.

[0213] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a multimodal advertising creative selection method as provided in all embodiments of this invention: acquiring candidate advertising creative data corresponding to a target item; wherein the candidate advertising creative data includes advertising images and advertising copy; acquiring sparse feature vectors and image feature vectors corresponding to the candidate advertising creative data, and obtaining a recommendation probability value corresponding to the candidate advertising creative data based on the sparse feature vectors, the image feature vectors, and a pre-trained creative selection model; wherein the creative selection model is used to: employ a self-attention mechanism to fuse the sparse feature vectors and the image feature vectors, and output the recommendation probability value based on the fusion result.

[0214] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0215] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0216] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0217] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0218] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A multi-modal based ad creative selection method, characterized in that, include: Obtain multiple candidate ad creative data corresponding to the target item; wherein, the candidate ad creative data includes ad images and ad copy; Obtain the sparse feature vector and image feature vector corresponding to the candidate ad creative data, and based on the sparse feature vector, the image feature vector and the pre-trained creative selection model, obtain the recommendation probability value corresponding to the candidate ad creative data; Select target ad creative data based on the recommended probability value; The creative selection model is used to: fuse the sparse feature vector and the image feature vector using a self-attention mechanism, and output the recommendation probability value based on the fusion result; the creative selection model includes a multilayer perceptron neural network (MLP) module, a self-attention module, and an output module; wherein: The MLP module is used to output a first feature vector based on the sparse feature vector; the sparse feature vector is a vector that reflects multiple types of sparse features; the sparse features include item features, user features, and creative features; the item features include item identifier, ad slot identifier, brand identifier, and item category target identifier; the user features include the user's age, gender, and preferences; the creative features include background template features, copywriting features, and image features; The self-attention module is used to output a second feature vector based on the sparse feature vector and the image feature vector; The output module is used to output the recommendation probability value based on the first feature vector and the second feature vector.

2. The method of claim 1, wherein, The step of obtaining the image feature vector corresponding to the candidate ad creative data includes: The ad images from the candidate ad creative data are input into a pre-trained residual neural network model; Obtain the image feature vector output by the residual neural network model.

3. The method of claim 2, wherein, The training method for the residual neural network model includes: Obtain sample images and corresponding category tags for the sample images; wherein the category tags are product terms for the sample items contained in the sample images, and the product terms are words used to characterize the type of the sample items and do not contain brand information; The sample image is input into the residual neural network model to be trained, and the predicted classification output by the residual neural network model is obtained. A loss function is determined based on the predicted classification and the classification label. The network parameters in the residual neural network model are adjusted based on the loss function, and training stops when a preset iteration stopping condition is met.

4. The method of claim 2, wherein, The tail of the residual neural network model contains three fully connected layers for outputting 32-dimensional, 128-dimensional, and 256-dimensional vectors, respectively.

5. The method according to any one of claims 1-4, characterized in that, The acquisition of candidate ad creative data corresponding to the target item includes: Acquire multiple advertising creative data corresponding to the target item, wherein the advertising creative data includes advertising copy and advertising images; Based on the online click data corresponding to each of the aforementioned advertising material data, at least one advertising copy and at least one advertising image are selected from each of the aforementioned advertising material data; The selected advertising copy and images are combined to obtain at least one candidate advertising creative data.

6. The method of claim 5, wherein, The acquisition of multiple advertising creative data corresponding to the target item includes: Identify and extract advertising copy from the item details page and / or advertising creative materials of the target item; The item image on the item details page of the target item is located and cropped to obtain the advertising image.

7. The method of claim 6, wherein, The advertising copy includes benefit-point copy and selling-point copy; the process of identifying and extracting the advertising copy from the item details page and / or advertising creative materials of the target item includes: Based on a preset character recognition model, candidate text is identified from the item details page and / or advertising creative materials of the target item; Based on a first thesaurus containing preset benefit-point words, benefit-point texts are selected from the candidate texts; Based on preset word count limits and / or a second vocabulary list containing preset non-selling point words, selling point copy is selected from the remaining copy after removing the benefit-point copy from the candidate copy.

8. The method of claim 5, wherein, The step of selecting at least one ad copy and at least one ad image from each of the ad creative data based on the online click data corresponding to each of the ad creative data includes: For each of the aforementioned ad creative data, the score of the ad creative data is determined based on the average online click-through rate of the ad creative data and the cumulative number of times the ad creative data has been selected; Based on the scores of each of the aforementioned advertising creative data, at least one advertising copy and at least one advertising image are selected from each of the aforementioned advertising creative data.

9. The method of claim 5, wherein, The process of combining the selected advertising copy and images to obtain at least one candidate advertising creative data includes: Combine the selected advertising copy and advertising images to obtain at least one copy-image combination; The at least one text image combination is combined with at least one preset background template to obtain at least one creative combination; Based on preset screening factors, at least one creative combination is selected from each of the creative combinations as candidate advertising creative data; wherein, the preset screening factors include the category information of the target item and / or the color information of the advertising image and background template in each of the creative combinations.

10. The method of claim 5, wherein, After obtaining the candidate ad creative data, the method further includes: Based on the color information of the advertising image in the candidate advertising creative data, the target item area in the advertising image is refined; wherein, the refining process includes at least one of adjusting brightness, adjusting contrast, and adjusting saturation.

11. The method of claim 10, wherein, The step of refining the target item area in the advertisement image based on the color information of the advertisement image in the candidate advertisement creative data includes: Based on the pixel values ​​of the pixels contained in the ad image in the candidate ad creative data, determine whether the ad image is a color image or a black and white image; When the advertisement image is a color image, the target item area in the advertisement image is retouched based on the first preset brightness parameter value, the first preset contrast parameter value, and the first preset saturation parameter value. When the advertisement image is a black and white image, the target item area in the advertisement image is retouched based on the second preset brightness parameter value, the second preset contrast parameter value, and the second preset saturation parameter value. Wherein, the second preset brightness parameter value is greater than the first preset brightness parameter value, the second preset contrast parameter value is greater than the first preset contrast parameter value, and the second preset saturation parameter value is greater than the first preset saturation parameter value.

12. The method of any one of claims 1-4, wherein, After selecting the target ad creative data, the method further includes: Obtain the first encoding information of the ad image and the second encoding information of the ad copy in the target ad creative data, and generate the Uniform Resource Locator URL corresponding to the target ad creative data based on the first encoding information and the second encoding information; Upon receiving an access request for the URL from the client, the system retrieves the ad image and ad copy from the target ad creative data based on the URL, performs a composite image operation on the retrieved ad image and ad copy to obtain the target ad creative image, and sends the target ad creative image to the client for display.

13. A model training method, comprising: The method includes: Acquire training sample data, which includes sample ad creative data corresponding to sample items and standard recommendation probability values ​​corresponding to the sample ad creative data. The sample ad creative data includes ad images and ad copy. Obtain the sparse feature vector and image feature vector corresponding to the sample ad creative data, and based on the sparse feature vector, the image feature vector and the creative selection model to be trained, obtain the predicted recommendation probability value corresponding to the sample ad creative data; The loss function is determined based on the standard recommendation probability value and the predicted recommendation probability value. The network parameters in the creative selection model are adjusted based on the loss function, and training is stopped when the preset iteration stopping condition is met. The creative selection model is used to: fuse the sparse feature vector and the image feature vector using a self-attention mechanism, and output the predicted recommendation probability value based on the fusion result; the creative selection model includes a multilayer perceptron neural network (MLP) module, a self-attention module, and an output module; wherein: The MLP module is used to output a first feature vector based on the sparse feature vector; the sparse feature vector is a vector that reflects multiple types of sparse features; the sparse features include item features, user features, and creative features; the item features include item identifier, ad slot identifier, brand identifier, and item category target identifier; the user features include the user's age, gender, and preferences; the creative features include background template features, copywriting features, and image features; The self-attention module is used to output a second feature vector based on the sparse feature vector and the image feature vector; The output module is used to output the predicted recommendation probability value based on the first feature vector and the second feature vector.

14. An advertisement creative data selection apparatus characterized by comprising: The device includes: The data acquisition module is used to acquire multiple candidate advertising creative data corresponding to the target item; wherein, the candidate advertising creative data includes advertising images and advertising copy; The probability value acquisition module is used to obtain the sparse feature vector and image feature vector corresponding to the candidate ad creative data, and obtain the recommendation probability value corresponding to the candidate ad creative data based on the sparse feature vector, the image feature vector and the pre-trained creative selection model. The data selection module is used to select target ad creative data based on the recommendation probability value; The creative selection model is used to: fuse the sparse feature vector and the image feature vector using a self-attention mechanism, and output the recommendation probability value based on the fusion result; the creative selection model includes a multilayer perceptron neural network (MLP) module, a self-attention module, and an output module; wherein: The MLP module is used to output a first feature vector based on the sparse feature vector; the sparse feature vector is a vector that reflects multiple types of sparse features; the sparse features include item features, user features, and creative features; the item features include item identifier, ad slot identifier, brand identifier, and item category target identifier; the user features include the user's age, gender, and preferences; the creative features include background template features, copywriting features, and image features; The self-attention module is used to output a second feature vector based on the sparse feature vector and the image feature vector; The output module is used to output the recommendation probability value based on the first feature vector and the second feature vector.

15. A model training apparatus, comprising: The device includes: The sample data acquisition module is used to acquire training sample data, which includes sample advertising creative data corresponding to sample items and standard recommendation probability values ​​corresponding to the sample advertising creative data. The sample advertising creative data includes advertising images and advertising copy. The vector acquisition module is used to acquire the sparse feature vector and image feature vector corresponding to the sample ad creative data, and based on the sparse feature vector, the image feature vector and the creative selection model to be trained, to obtain the predicted recommendation probability value corresponding to the sample ad creative data. The model training module is used to determine a loss function based on the standard recommendation probability value and the predicted recommendation probability value, adjust the network parameters in the creative selection model based on the loss function, and stop training when a preset iteration stopping condition is met. The creative selection model is used to: fuse the sparse feature vector and the image feature vector using a self-attention mechanism, and output the predicted recommendation probability value based on the fusion result; the creative selection model includes a multilayer perceptron neural network (MLP) module, a self-attention module, and an output module; wherein: The MLP module is used to output a first feature vector based on the sparse feature vector; the sparse feature vector is a vector that reflects multiple types of sparse features; the sparse features include item features, user features, and creative features; the item features include item identifier, ad slot identifier, brand identifier, and item category target identifier; the user features include the user's age, gender, and preferences; the creative features include background template features, copywriting features, and image features; The self-attention module is used to output a second feature vector based on the sparse feature vector and the image feature vector; The output module is used to output the predicted recommendation probability value based on the first feature vector and the second feature vector.

16. An electronic device, comprising: The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the multimodal advertising creative selection method according to any one of claims 1-12, or the model training method according to claim 13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the multimodal advertising creative selection method according to any one of claims 1-12, or the model training method according to claim 13.

Images