A recommendation method, training method, apparatus, device, and news recommendation system

Abstract

This application discloses a recommendation method, training method, apparatus, device, and recommendation system for leveraging the influence of news interfaces on users to increase user click-through rates on news. The method includes: acquiring multiple images, each image containing a candidate interface and candidate content presented through the candidate interface; acquiring image feature data for each image; determining the input to a prediction model based on the target user's user feature data and the image feature data, and then predicting the target user's preference level for each image using the prediction model; finally, selecting candidate content and / or candidate interfaces from the candidate interfaces and candidate content contained in the multiple images based on the preference level, and then recommending the selected candidate content or candidate interface to the user.

Description

Technical Field

[0001] This application relates to the field of recommendation technology, and in particular to a recommendation method, training method, apparatus, device, and recommendation system. Background Technology

[0002] Today, various news mobile applications have changed the way people traditionally read news. Major news platforms generate massive amounts of news constantly. Therefore, when using these apps, users are recommended all sorts of news content. If the recommended content is not of interest to the user, the click-through rate will decrease. To improve click-through rates, personalized news recommendation systems have emerged. These systems use machine learning to identify users' interests and recommend more relevant news content, thereby increasing click-through rates.

[0003] However, current news recommendation systems only focus on identifying news content that users are interested in, ignoring the impact of the news interface used to recommend the news content on users, thus failing to further improve the click-through rate of the news. Summary of the Invention

[0004] This application provides a recommendation method, training method, apparatus, device, and recommendation system for increasing user click-through rates on news articles by leveraging the influence of news interfaces on users.

[0005] In a first aspect, embodiments of this application provide a recommendation method, comprising: acquiring multiple images, each image containing a candidate interface and candidate content presented through the candidate interface, wherein the image can be understood as an image presenting candidate content through the candidate interface; the candidate content can be not only news content, but also short videos, product information, and other content; correspondingly, the candidate interface can be not only a news interface, but also an interface for presenting short videos, an interface for presenting product information; acquiring image feature data for each image; the image feature data may include global visual impression feature data and / or local visual impression feature data, wherein the global visual impression feature data can be understood as feature data extracted from the entire image, and the local visual impression feature data can be understood as feature data extracted from a local region of the image; and based on the user feature data of the target user... Based on image feature data, a prediction model is used to predict the target user's preference for each image. The input to the prediction model is determined based on user feature data and image feature data. The user feature data includes the user's age, city, and historical data related to news. Specifically, the historical data related to news can include the type of news the user browsed, the type of news the user clicked, the time the user clicked the news, and the location when the user clicked the news. Based on the preference level, candidate content and / or candidate interfaces are selected from candidate interfaces and candidate screens contained in multiple images for recommendation. Specifically, the preference-based program can select only candidate content from multiple images for recommendation, or only candidate interfaces from multiple images for recommendation, or simultaneously select candidate content and candidate interfaces from multiple images for recommendation.

[0006] Since an image contains both candidate content and candidate interface, a prediction model trained based on the image feature data can accurately predict a user's preference for an image while simultaneously considering the impact of both candidate content and candidate interface on the user. This helps to recommend content that the user is interested in by using candidate interfaces that the user is interested in, thereby increasing the click-through rate of the recommended content.

[0007] As one feasible approach, each image comprises multiple regions. Specifically, the image can be divided into multiple regions using various methods. For example, as described above, a news article may include the headline, author, and category of the news, as well as accompanying images. Therefore, the coordinates of each of these parts can be obtained according to the news layout, and then the image can be divided into multiple regions based on these coordinates. The image feature data of each image includes multiple local feature vectors, each of which represents a region.

[0008] In this implementation, the image is divided into multiple regions, and the local feature vector representing each region is used as the image feature data. This allows for better extraction of local features of the image, thereby improving the accuracy of predicting the user's preference for the image.

[0009] As an achievable approach, based on user feature data and image feature data of the target user, and using a predictive model to predict the target user's preference for each image, the following steps are taken: For each image, N word vectors are obtained based on the candidate content in each image, with each word vector representing a word in the candidate content, where N is a positive integer; the candidate content includes N words, and for each word, a word vector can be generated using a text representation model; similar to image representation models, text representation models can be understood as models obtained through pre-training, and there are various types of models, such as the BERT model; since the title of news content can well reflect the main information of the news content, when the candidate content is news content, the title of the news content can be segmented to obtain N words, and then N word vectors representing the N words are obtained through a text representation model; for each word vector, based on each word vector and multiple local feature vectors, and using an attention mechanism model, the attention weights of the multiple local feature vectors are calculated, and an attention weight table is provided. This study demonstrates the degree to which target users pay attention to the regions represented by local feature vectors when reading each word vector. The attention mechanism is a method that dynamically controls the attention level to different parts or specific parts of the neural network model by calculating the attention weights of each part and merging them into an attention vector. Based on the attention weights of multiple local feature vectors, each word vector and multiple local feature vectors are fused to obtain a first fused feature vector, with each word vector corresponding to one first fused feature vector. Specifically, multiple local feature vectors can be weighted using their respective attention weights, and the weighted result is added to the word vector to obtain the first fused feature vector. Based on the user feature vector and the N first fused feature vectors corresponding to N word vectors, a prediction model is used to predict the target user's preference for each image. The input to the prediction model is determined based on the user feature vector and the N first fused feature vectors, with the user feature vector representing the target user's user feature data.

[0010] In this implementation, attention weights are calculated for each of the multiple local feature vectors using an attention mechanism model. Since the attention weights represent the degree to which the target user pays attention to the region represented by the local feature vector when reading each word vector, each word vector and multiple local feature vectors are fused based on their respective attention weights. The resulting first fused feature vector can reflect the impression features left on the user by the words and regions in the image. In this way, the accuracy of predicting the user's preference level using the first fused feature vector can be improved.

[0011] As an achievable approach, based on the user feature vector and N first fusion feature vectors corresponding to N word vectors, and using a prediction model to predict the target user's preference for each image, the following steps are taken: For each image, the N first fusion feature vectors corresponding to the N word vectors are processed using a self-attention mechanism to obtain N semantic enhancement feature vectors. Each first fusion feature vector corresponds to one semantic enhancement feature vector. The self-attention mechanism is an improvement on the attention mechanism, which reduces dependence on external information and is better at capturing the internal correlation of data or features. Based on the user feature vector and the N semantic enhancement feature vectors, the prediction model predicts the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and the N semantic enhancement feature vectors.

[0012] The semantically enhanced feature vector is obtained by processing the N first fusion feature vectors corresponding to N word vectors through a self-attention mechanism model. Since the self-attention mechanism is better at capturing the internal correlation of data or features, the resulting semantically enhanced feature vector can reflect the correlation between the first fusion feature vectors, thus more accurately reflecting the impression feature information left by the image to the user. In this way, using the semantically enhanced feature vector to predict the degree of preference can improve the accuracy of the user's preference for the image.

[0013] As an achievable approach, based on user feature vectors and N semantically enhanced feature vectors, and using a prediction model to predict the target user's preference for each image, the following steps are taken: For each image, the N semantically enhanced feature vectors are fused using an additive attention mechanism to obtain a second fused feature vector; based on the user feature vector and the second fused feature vector, the target user's preference for each image is predicted using a prediction model, where the input to the prediction model is determined based on the user feature vector and the second fused feature vector.

[0014] The model, which employs an additive attention mechanism, fuses N semantically enhanced feature vectors and uses the fused second feature vector to predict the degree of preference, thereby improving the accuracy of users' image preference.

[0015] As one possible approach, the image feature data of each image includes a global feature vector, which is used to characterize the image; in this case, the image feature data can also be called global visual impression feature data; the method of obtaining the global feature vector can specifically include: inputting the image into an image characterizer to convert the image into a global feature vector through the image characterizer.

[0016] In this implementation, the global feature vector representing the image is used as the image feature data, which can better extract the global features of the image and improve the accuracy of predicting the user's preference for the image.

[0017] As an feasible approach, based on the target user's user feature data and image feature data, and using a prediction model to predict the target user's preference for each image, the following steps are taken: For each image, a content feature vector is obtained based on the candidate content in each image. The content feature vector is used to represent the candidate content. Since the title of news content can well reflect the main information of the news content, when the candidate content is news content, the title of the news content can be transformed into a title feature vector. Based on the content feature vector and the global feature vector, the weights of the content feature vector and the global feature vector are determined. Since users may have different sensitivities to visual impression information and text semantics, as an feasible approach, the weights of the content feature vector and the global feature vector can be adaptively controlled through a threshold addition network. Based on the weights of the content feature vector and the global feature vector, the content feature vector and the global feature vector are fused to obtain a third fused feature vector. Based on the user feature vector and the third fused feature vector, the prediction model predicts the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and the third fused vector, and the user feature vector is used to represent the target user's user feature data.

[0018] Based on content feature vectors and global feature vectors, the weights of the content feature vectors and global feature vectors are determined. Then, based on the weights of the content feature vectors and global feature vectors, a third fused feature vector is obtained by fusing the content feature vectors and global feature vectors. This third fused feature vector can represent the impression feature information left by the extracted image to the user from a global perspective. Therefore, using the third fused feature vector to predict the target user's preference for each image can improve the accuracy of the user's preference for the image.

[0019] As one feasible approach, selecting candidate content and / or candidate interface from candidate interfaces and candidate content contained in multiple images for recommendation based on preference includes: selecting a candidate content as a target candidate content from candidate content contained in multiple images based on preference; and selecting a candidate interface as a target candidate interface from candidate interfaces of an image containing the target candidate content based on preference, so as to recommend the target candidate content through the target candidate interface.

[0020] Based on preference, a candidate content is selected as the target candidate content from multiple images; based on preference, a candidate interface is selected as the target candidate interface from the candidate interfaces of the image containing the target candidate content, and the target candidate content is recommended through the target candidate interface. This realizes the recommendation of user-preferred candidate content to users through the candidate interface of user preference, thereby increasing the probability of users clicking on the recommended content.

[0021] As one possible approach, after selecting a candidate interface as the target candidate interface from candidate interfaces containing the target candidate content based on preference, the method further includes: sending metadata of the target candidate interface and the target candidate content to the terminal device, so that the terminal device displays the target candidate interface based on the metadata and recommends the target candidate content to the target user through the target candidate interface; wherein, the metadata contains various configuration data of the target candidate interface.

[0022] The target candidate interface metadata and target candidate content are sent to the terminal device, enabling the terminal device to display the target candidate interface based on the metadata and recommend the target candidate content to the target user through the target candidate interface, thereby increasing the probability of the user clicking on the recommended content.

[0023] Secondly, embodiments of this application provide a training method, comprising: acquiring multiple sample images, each sample image containing a sample candidate interface and sample candidate content presented through the sample candidate interface; acquiring image feature data for each sample image; predicting the sample user's preference for each sample image based on the sample user's user feature data and image feature data, wherein the input to the prediction model is determined based on the user feature data and image feature data; adjusting the prediction model based on the preference level and the sample user's historical click data on the sample candidate content, wherein the sample user's historical click data on the sample candidate content may include whether the sample user clicked on the sample candidate content and the number of times the sample user clicked on the sample candidate content; specifically, the weights of the prediction model can be adjusted, and the structure of the prediction model can also be adjusted.

[0024] Since the sample images contain both candidate content and candidate interfaces, the prediction model trained based on the image feature data of the sample images can accurately output the user's preference for the image while simultaneously considering the impact of candidate content and candidate interfaces on the user. This is beneficial for recommending content that the user is interested in through the interface that the user is interested in, thereby increasing the click-through rate of the recommended content.

[0025] As one possible approach, each sample image comprises multiple regions; the image feature data of each sample image includes multiple local feature vectors, each of which is used to characterize a region.

[0026] For the above-mentioned explanations and technical effects, please refer to the description of the first aspect of the embodiments of this application.

[0027] As an achievable approach, based on user feature data and image feature data of sample users, and through a prediction model, the degree of preference of sample users for each sample image is predicted as follows: For each sample image, N word vectors are obtained based on the sample candidate content in each sample image, where each word vector represents a word in the sample candidate content, and N is a positive integer; For each word vector, based on each word vector and multiple local feature vectors, attention weights of each local feature vector are calculated through an attention mechanism model, where the attention weights represent the degree to which the sample user pays attention to the region represented by the local feature vector when reading the word represented by each word vector; Based on the attention weights of each local feature vector, each word vector and multiple local feature vectors are fused to obtain a first fused feature vector, with each word vector corresponding to one first fused feature vector; Based on the user feature vector and the N first fused feature vectors corresponding to the N word vectors, the degree of preference of sample users for each sample image is predicted through a prediction model, where the input of the prediction model is determined based on the user feature vector and the N first fused feature vectors, and the user feature vector is used to represent the user feature data of the sample user.

[0028] For the above-mentioned explanations and technical effects, please refer to the description of the first aspect of the embodiments of this application.

[0029] As an achievable approach, based on the user feature vector and N first fusion feature vectors corresponding to N word vectors, and using a prediction model to predict the user's preference for each sample image, the following steps are taken: For each sample image, the N first fusion feature vectors corresponding to the N word vectors are processed using a self-attention mechanism model to obtain N semantic enhancement feature vectors, each first fusion feature vector corresponding to one semantic enhancement feature vector; based on the user feature vector and the N semantic enhancement feature vectors, the user's preference for each sample image is predicted using a prediction model, where the input to the prediction model is determined based on the user feature vector and the N semantic enhancement feature vectors.

[0030] For the above-mentioned explanations and technical effects, please refer to the description of the first aspect of the embodiments of this application.

[0031] As an achievable approach, based on user feature vectors and N semantically enhanced feature vectors, and using a prediction model to predict the user's preference for each sample image, the following steps are taken: for each sample image, the N semantically enhanced feature vectors are fused using an additive attention mechanism to obtain a second fused feature vector; based on the user feature vector and the second fused feature vector, the user's preference for each sample image is predicted using a prediction model, where the input to the prediction model is determined based on the user feature vector and the second fused feature vector.

[0032] For the above-mentioned explanations and technical effects, please refer to the description of the first aspect of the embodiments of this application.

[0033] As one possible approach, the image feature data for each sample image includes a global feature vector, which is used to characterize the sample image.

[0034] For the above-mentioned explanations and technical effects, please refer to the description of the first aspect of the embodiments of this application.

[0035] As one feasible approach, based on user feature data and image feature data of sample users, and using a prediction model to predict the preference level of sample users for each sample image, the following steps are taken: For each sample image, a content feature vector is obtained based on the candidate content in each sample image, and the content feature vector is used to represent the candidate content; based on the content feature vector and the global feature vector, the weights of the content feature vector and the global feature vector are determined; based on the weights of the content feature vector and the global feature vector, the content feature vector and the global feature vector are fused to obtain a third fused feature vector; based on the user feature vector and the third fused feature vector, the preference level of sample users for each sample image is predicted by a prediction model, and the input of the prediction model is determined based on the user feature vector and the third fused vector, whereby the user feature vector is used to represent the user feature data of the sample users.

[0036] For the above-mentioned explanations and technical effects, please refer to the description of the first aspect of the embodiments of this application.

[0037] Thirdly, embodiments of this application provide a recommendation apparatus, comprising: a first image acquisition unit, configured to acquire multiple images, each image containing a candidate interface and candidate content presented through the candidate interface; a first feature data acquisition unit, configured to acquire image feature data of each image; a first prediction unit, configured to predict the target user's preference level for each image based on the target user's user feature data and image feature data, and using a prediction model, wherein the input of the prediction model is determined based on the user feature data and image feature data; and a recommendation unit, configured to select candidate content and / or candidate interface from the candidate interfaces and candidate content contained in the multiple images based on the preference level, for recommendation.

[0038] As one possible approach, each image comprises multiple regions; the image feature data of each image includes multiple local feature vectors, each of which is used to characterize a region.

[0039] As one possible approach, the first prediction unit is used to obtain N word vectors for each image based on the candidate content in each image, where each word vector represents a word in the candidate content, and N is a positive integer;

[0040] For each word vector, based on each word vector and multiple local feature vectors, an attention weight is calculated for each local feature vector using an attention mechanism model. The attention weight represents the degree to which the target user pays attention to the region represented by the local feature vector when reading the word represented by each word vector. Based on the attention weights of the multiple local feature vectors, each word vector and the multiple local feature vectors are fused to obtain a first fused feature vector, with each word vector corresponding to one first fused feature vector. Based on the user feature vector and the N first fused feature vectors corresponding to the N word vectors, a prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and the N first fused feature vectors. The user feature vector is used to represent the user feature data of the target user.

[0041] As one possible approach, the first prediction unit is used to process the N first fusion feature vectors corresponding to the N word vectors for each image through a self-attention mechanism model to obtain N semantic enhancement feature vectors, each of which corresponds to a semantic enhancement feature vector. Based on the user feature vector and the N semantic enhancement feature vectors, the prediction model predicts the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and the N semantic enhancement feature vectors.

[0042] As one possible approach, the first prediction unit, used to predict the target user's preference for each image based on the user feature vector and N semantically enhanced feature vectors, includes: for each image, fusing the N semantically enhanced feature vectors through an additive attention mechanism to obtain a second fused feature vector; and predicting the target user's preference for each image based on the user feature vector and the second fused feature vector, wherein the input to the prediction model is determined based on the user feature vector and the second fused feature vector.

[0043] As one possible approach, the image feature data for each image includes a global feature vector, which is used to characterize the image.

[0044] As one possible approach, the first prediction unit is used to obtain a content feature vector for each image based on candidate content in each image, the content feature vector being used to characterize the candidate content; determine the weights of the content feature vector and the global feature vector based on the content feature vector and the global feature vector; fuse the content feature vector and the global feature vector based on the weights of the content feature vector and the global feature vector to obtain a third fused feature vector; and predict the target user's preference for each image based on the user feature vector and the third fused feature vector, the input of the prediction model being determined based on the user feature vector and the third fused vector, the user feature vector being used to characterize the target user's user feature data.

[0045] As one possible approach, the recommendation unit is used to select a candidate content as the target candidate content from the candidate content contained in multiple images based on the degree of preference; and to select a candidate interface as the target candidate interface from the candidate interfaces of the image containing the target candidate content based on the degree of preference, so as to recommend the target candidate content through the target candidate interface.

[0046] As one possible approach, the device also includes a sending unit for sending metadata of the target candidate interface and target candidate content to the terminal device, so that the terminal device can display the target candidate interface based on the metadata and recommend the target candidate content to the target user through the target candidate interface.

[0047] For details on the implementation, related explanations, and technical effects of each of the above units, please refer to the description of the first aspect of the embodiments of this application.

[0048] Fourthly, embodiments of this application provide a training apparatus, comprising: a second image acquisition unit for acquiring multiple sample images, each sample image containing a sample candidate interface and sample candidate content presented through the sample candidate interface; a second feature data acquisition unit for acquiring image feature data of each sample image; a second prediction unit for predicting the degree of preference of the sample user for each sample image based on the user feature data and image feature data of the sample user, and through a prediction model, wherein the input of the prediction model is determined based on the user feature data and image feature data; and an adjustment unit for adjusting the prediction model based on the degree of preference and the historical click data of the sample user on the sample candidate content.

[0049] As one possible approach, each sample image comprises multiple regions; the image feature data of each sample image includes multiple local feature vectors, each of which is used to characterize a region.

[0050] As one feasible approach, the second prediction unit is used to obtain N word vectors for each sample image based on the candidate content in each sample image, where each word vector represents a word in the candidate content, and N is a positive integer; for each word vector, based on each word vector and multiple local feature vectors, and through an attention mechanism model, calculate the attention weights of each of the multiple local feature vectors, where the attention weights represent the degree to which the sample user pays attention to the region represented by the local feature vector when reading the word represented by each word vector; based on the attention weights of each of the multiple local feature vectors, fuse each word vector and the multiple local feature vectors to obtain a first fused feature vector, where each word vector corresponds to one first fused feature vector; based on the user feature vector and the N first fused feature vectors corresponding to the N word vectors, predict the sample user's preference for each sample image through a prediction model, where the input of the prediction model is determined based on the user feature vector and the N first fused feature vectors, and the user feature vector is used to represent the user feature data of the sample user.

[0051] As one possible approach, the second prediction unit processes the N first fusion feature vectors corresponding to the N word vectors for each sample image using a self-attention mechanism model to obtain N semantic enhancement feature vectors, with each first fusion feature vector corresponding to a semantic enhancement feature vector. Based on the user feature vector and the N semantic enhancement feature vectors, the prediction model predicts the user's preference for each sample image. The input of the prediction model is determined based on the user feature vector and the N semantic enhancement feature vectors.

[0052] As one possible approach, the second prediction unit is used to fuse N semantically enhanced feature vectors for each sample image using an additive attention mechanism to obtain a second fused feature vector; based on the user feature vector and the second fused feature vector, a prediction model is used to predict the user's preference for each sample image. The input of the prediction model is determined based on the user feature vector and the second fused feature vector.

[0053] As one possible approach, the image feature data for each sample image includes a global feature vector, which is used to characterize the sample image.

[0054] As one possible approach, the second prediction unit is used to obtain a content feature vector for each sample image based on the candidate content in each sample image. The content feature vector is used to characterize the candidate content. Based on the content feature vector and the global feature vector, the weights of the content feature vector and the global feature vector are determined. Based on the weights of the content feature vector and the global feature vector, the content feature vector and the global feature vector are fused to obtain a third fused feature vector. Based on the user feature vector and the third fused feature vector, the prediction model predicts the degree of preference of the sample user for each sample image. The input of the prediction model is determined based on the user feature vector and the third fused vector. The user feature vector is used to characterize the user feature data of the sample user.

[0055] For details on the implementation, related explanations, and technical effects of each of the above units, please refer to the description of the second aspect of the embodiments of this application.

[0056] Fifthly, embodiments of this application provide a computer device, including: one or more processors and a memory; wherein the memory stores computer-readable instructions; the one or more processors read the computer-readable instructions to cause the vehicle-mounted device to implement the method as described in any implementation of the first aspect.

[0057] In a sixth aspect, embodiments of this application provide a training device, including: one or more processors and a memory; wherein the memory stores computer-readable instructions; the one or more processors read the computer-readable instructions to enable the vehicle-mounted device to implement the method as described in any implementation of the second aspect.

[0058] In a seventh aspect, embodiments of this application provide a computer-readable storage medium including computer-readable instructions that, when executed on a computer, cause the computer to perform a method as implemented in either the first or second aspect.

[0059] Eighthly, embodiments of this application provide a chip including one or more processors. Part or all of the processors are configured to read and execute a computer program stored in a memory to perform the methods in any possible implementation of the first or second aspect described above.

[0060] Optionally, the chip includes a memory, which is connected to the processor via a circuit or wire. Further optionally, the chip also includes a communication interface, to which the processor is connected. The communication interface is used to receive data and / or information that needs to be processed. The processor obtains the data and / or information from the communication interface, processes the data and / or information, and outputs the processing result through the communication interface. The communication interface can be an input / output interface.

[0061] In some implementations, some of the processors in one or more processors may implement some of the steps in the above methods through dedicated hardware. For example, the processing involving neural network models may be implemented by a dedicated neural network processor or graphics processor.

[0062] The method provided in this application embodiment can be implemented by a single chip or by multiple chips working together.

[0063] Ninthly, embodiments of this application provide a computer program product, which includes computer software instructions that can be loaded by a processor to implement the method of any one of the first or second aspects described above.

[0064] In a tenth aspect, embodiments of this application provide a recommendation system, including a terminal device and a server;

[0065] The server is used to execute methods such as any of the implementations in the first aspect;

[0066] The terminal device is used to receive metadata and target candidate content from the server's target candidate interface;

[0067] The target candidate interface is displayed based on metadata, and target candidate content is recommended to the target user through the target candidate interface. Attached Figure Description

[0068] Figure 1 A schematic diagram of the architecture of a news recommendation system provided in the embodiments of this application;

[0069] Figure 2 This is a schematic diagram of an example of news reporting.

[0070] Figure 3 A diagram illustrating the working process of a news recommendation system;

[0071] Figure 4 This application provides a schematic diagram of one embodiment of a training method.

[0072] Figure 5 This is a schematic diagram of the region of the sample image in the embodiments of this application;

[0073] Figure 6 This is a schematic diagram of a first embodiment of the present application for predicting the degree of preference of sample users for each sample image;

[0074] Figure 7 This is a schematic diagram of a second embodiment of the present application for predicting the degree of preference of sample users for each sample image;

[0075] Figure 8 This is a schematic diagram of the process of fusing the second feature vector in an embodiment of this application;

[0076] Figure 9 This is a schematic diagram of a third embodiment of the present application that predicts the degree of preference of sample users for each sample image;

[0077] Figure 10 This is a schematic diagram illustrating the process of obtaining the third fused feature vector in an embodiment of this application;

[0078] Figure 11 This application provides a schematic diagram of one embodiment of a recommended method;

[0079] Figure 12 This is a schematic diagram of a first embodiment of the present application for predicting the target user's preference for each image;

[0080] Figure 13 This is a schematic diagram of a second embodiment of the present application for predicting the target user's preference for each image;

[0081] Figure 14 This is a schematic diagram of a third embodiment of the present application, which predicts the target user's preference for each image.

[0082] Figure 15 This is a schematic diagram illustrating an embodiment of predicting a user's preference for news in this application.

[0083] Figure 16 This is a schematic diagram illustrating an embodiment of obtaining the optimal user interface configuration in this application.

[0084] Figure 17 This application provides a schematic diagram of one embodiment of a training device;

[0085] Figure 18 This application provides a schematic diagram of one embodiment of a recommended device;

[0086] Figure 19 A schematic diagram illustrating an embodiment of the computer device provided in this application. Detailed Implementation

[0087] The embodiments of this application are described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. As those skilled in the art will recognize, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0088] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps appearing in this application does not imply that the steps in the method flow must be performed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical purpose, as long as the same or similar technical effect is achieved.

[0089] Furthermore, in the description of this invention, unless otherwise stated, "multiple" means two or more. The term "and / or" or the character " / " in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B, or A / B, can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0090] The embodiments of this application can be applied to Figure 1 In the news recommendation system shown, such as Figure 1 As shown, the news recommendation system includes terminal devices and a server, and the terminal devices are connected to the server.

[0091] This application does not specifically limit the type of terminal device. For example, the terminal device may include any device that can deploy news applications, such as mobile phones, tablets, desktop computers, and in-vehicle devices; the terminal device will be referred to as a terminal below.

[0092] The server can be a regular server or a cloud server.

[0093] like Figure 1 As shown, a news application is deployed on the terminal, and a recommendation service is deployed on the server.

[0094] When a user accesses a news application on their device, the device sends a request to the server to request the recommendation service. Upon receiving the request, the server activates the recommendation service and selects news content that the user is interested in from a large amount of news content as recommended news content. The server then sends the recommended news content to the device, which then displays the recommended news content to the user.

[0095] This application does not specifically limit the news content; for example, such as Figure 2 As shown, news content can include the headline, author, and category of the news. In addition, news content can also include the body of the news article.

[0096] However, for a news article, not only does the content affect click-through rates, but the interface on which the content is presented also influences them. Specifically, the layout of text and images on the news interface (including the position of the title and the relative position of the title and image), whether an image is included and its size, color, clarity, font, and font size all leave different visual impressions on users, affecting their browsing experience and thus influencing their click-through behavior.

[0097] In this application embodiment, the information that leaves a visual impression on the user in the news interface is referred to as visual impression information. This visual impression information can be understood as the multimodal information of the news displayed in the news interface from the user's perspective. Specifically, it may include the aforementioned information such as the layout of the text and images, whether or not there are pictures and the size of the pictures, the color of the pictures, the clarity of the pictures, the font, and the size of the font.

[0098] by Figure 2 For example, Figure 2 The first news item in the article differs from the second in terms of text and image layout and image size; users who prefer larger images are more likely to click on the first news item.

[0099] Based on this, in order to improve the click-through rate of news, this application provides a recommendation method. This method involves acquiring multiple images, each containing a candidate interface and a candidate content. Then, based on the target user's user feature data and the image feature data of the images, a prediction model is used to predict the target user's preference for each image. Finally, candidate content and / or candidate interface are selected from the multiple images based on the preference level for recommendation. In the above recommendation method, the candidate interface can be a news interface, and the candidate content can be news content. Thus, this recommendation method can achieve news recommendation. Furthermore, in the process of recommending news using this method, not only the influence of the news content on the target user is considered, but also the influence of the news interface on the target user. Therefore, news of interest (including news content and news interface) can be recommended to the target user, further improving the click-through rate of news.

[0100] It should be noted that the candidate content can be not only news content, but also short videos, product information, and other content; correspondingly, the candidate interface can be not only a news interface, but also an interface for presenting short videos or an interface for presenting product information. The following description uses news content and a news interface as examples to illustrate the method provided in this application's embodiments.

[0101] Therefore, in Figure 1 In the news recommendation system shown, the server can also select news pages that users are interested in, then send the metadata of the news pages to the terminal, and the terminal displays the news pages based on the metadata, and then displays the recommended news content to the user through the news pages.

[0102] As explained above, a prediction model is needed in the news recommendation process, so the prediction model needs to be trained in advance.

[0103] Specifically, Figure 1 The working process of the news recommendation system shown can be as follows: Figure 3 As shown.

[0104] The server extracts news-related data from the user's behavior logs (specifically, data on browsing or clicking on news), uses this data to build training data, and then performs offline training based on the training data to obtain a prediction model. After receiving a request for a recommendation service, the server uses the prediction model to make online predictions to obtain the user's preference for multiple news images, and then selects news content and news interface based on this preference. Finally, the terminal displays the news content to the user through the news interface.

[0105] The offline training process of the prediction model will be introduced below.

[0106] like Figure 4 As shown, this application provides an embodiment of a training method, which is typically applied to a server. Specifically, this embodiment includes:

[0107] Step 101: Obtain multiple sample images, each sample image containing a sample candidate interface and a sample candidate content presented through the sample candidate interface.

[0108] A sample image can be understood as an image that presents sample candidate content through a sample candidate interface. For a better understanding of the sample candidate interface and sample candidate content, please refer to the relevant explanations of the candidate interface and sample candidate content in the previous text.

[0109] The situation involving multiple sample images can include the following three types.

[0110] The first scenario is: multiple sample images include one sample candidate interface and multiple sample candidate contents, that is, the sample candidate interface is the same in all sample images.

[0111] The second scenario is that multiple sample images include multiple candidate interfaces and one candidate content, meaning that the candidate content in all sample images is the same.

[0112] The third scenario involves multiple sample images containing multiple candidate interfaces and multiple candidate contents. In this case, all sample images containing the same candidate content can contain multiple candidate contents. For example, if there are 10,000 sample images, each containing 100 candidate contents, all sample images containing the same candidate content can contain 100 candidate interfaces. In other words, each candidate content can be presented through 100 candidate interfaces.

[0113] Step 102: Obtain image feature data for each sample image.

[0114] There are various methods for obtaining image feature data of sample images, and this application does not specifically limit them.

[0115] Based on different acquisition methods, the embodiments of this application roughly divide image feature data into two categories: global visual impression feature data and local visual impression feature data. It should be noted that the image feature data may include only global visual impression feature data, only local visual impression feature data, or both global visual impression feature data and local visual impression feature data.

[0116] As one possible approach, each sample image includes multiple regions, and correspondingly, the image feature data of each sample image includes multiple local feature vectors, each of which is used to characterize a region; in this case, the image feature data can also be called local visual impression feature data.

[0117] It should be noted that sample images can be divided into multiple regions using various methods. For example, as explained above, a news article may include the headline, author, and category, as well as accompanying images. Therefore, the coordinates of each of these parts can be obtained according to the news layout, and then the sample image can be divided into multiple regions based on these coordinates.

[0118] For example, with Figure 5 For example, the above method can be used to... Figure 5 The sample images in the data are divided into news headlines, news categories, and news accessories. Figure 3 Each region.

[0119] The method for obtaining local feature vectors can specifically include: inputting images of multiple regions into an image representation device to transform multiple regions into multiple local feature vectors; where the image representation device can be understood as a model obtained through pre-training, and there can be many types of models, for example, the model can be ResNet101.

[0120] As one possible approach, the image feature data for each sample image includes a global feature vector, which is used to characterize the sample image; in this case, the image feature data can also be called global visual impression feature data.

[0121] Methods for obtaining global feature vectors may include: inputting the sample image into an image representation device to convert the sample image into a global feature vector; since the image representation device has been explained above, it will not be described in detail here.

[0122] Step 103: Obtain user characteristic data of sample users.

[0123] This application does not specifically limit the types of feature data of sample users. For example, the feature data of sample users includes the age information of sample users, the city where sample users live, and the historical data of sample users related to news. Specifically, the historical data of sample users related to news may include the types of news that sample users browse, the types of news that sample users click, the time when sample users clicked the news, and the location when sample users clicked the news.

[0124] Historical data related to news from sample users can be obtained from their behavior logs.

[0125] Step 104: Based on the user feature data and image feature data of the sample users, a prediction model is used to predict the degree of preference of the sample users for each sample image. The input of the prediction model is determined based on the user feature data and image feature data.

[0126] It should be noted that user feature data and image feature data can also be combined with specific environmental information (such as time, date, whether it is a weekend, whether it is a holiday, etc.) and a prediction model can be used to predict the degree of preference of sample users for each sample image.

[0127] For a sample image, the sample user feature data and image feature data can be directly input into the prediction model to obtain the sample user's preference for the sample image output by the prediction model; alternatively, the image feature data can be processed first to obtain intermediate feature data, and then the sample user feature data and the intermediate feature data can be input into the prediction model to obtain the sample user's preference for the sample image output by the prediction model.

[0128] The following section will explain in detail the process of predicting the degree of preference of sample users for each sample image using a predictive model.

[0129] Step 105: Adjust the prediction model based on the degree of preference and the historical click data of sample users on the sample candidate content.

[0130] The historical click data of sample users on sample candidate content may include whether the sample user clicked on the sample candidate content and the number of times the sample user clicked on the sample candidate content.

[0131] Specifically, sample labels can be set based on the historical click data of sample users on sample candidate content. For example, for a sample image, if a sample user has clicked on the sample candidate content in the sample image, the sample label of preference level can be set to 1; if a sample user has not clicked on the sample candidate content in the sample image, the sample label of preference level can be set to 0.

[0132] For example, for a sample image, if the number of times a sample user clicks on the candidate content in the sample image is greater than or equal to the first threshold, the sample label of preference level can be set to 1; if the number of times a sample user clicks on the candidate content in the sample image is less than the first threshold but greater than or equal to the second threshold, the sample label of preference level can be set to 0.5; if the number of times a sample user clicks on the candidate content in the sample image is less than the second threshold, or if the sample user has not clicked on the candidate content in the sample image, the sample label of preference level can be set to 0.

[0133] Based on this, a loss function can be calculated using the sample user's preference for the sample image output by the prediction model and the sample label. The weights of the prediction model can be updated through backpropagation of the loss function, or the structure of the prediction model can be adjusted so that the preference output by the prediction model is close to the sample label.

[0134] In this embodiment, since the sample image contains both candidate content and candidate interface, the prediction model trained based on the image feature data of the sample image can accurately output the user's preference for the image while simultaneously considering the impact of candidate content and candidate interface on the user. This is beneficial for recommending content that the user is interested in through the interface that the user is interested in, thereby increasing the click-through rate of the recommended content.

[0135] The following describes the process of predicting the degree of preference of sample users for each sample image using a predictive model.

[0136] First, we will introduce the case where image feature data includes local visual impression feature data.

[0137] As a feasible approach, such as Figure 6 As shown, step 104 includes:

[0138] Step 201: For each sample image, obtain N word vectors based on the sample candidate content in each sample image. Each word vector represents a word in the sample candidate content, where N is a positive integer.

[0139] The sample candidate content includes N words. For each word, a word vector can be generated using a text representation. Similar to the image representation, the text representation can also be understood as a model obtained through pre-training. There can be various types of models, such as the BERT model.

[0140] Understandably, when the candidate sample content is news content, the title of the news content can usually reflect the main information of the news content well. Therefore, when the candidate sample content is news content, the title of the news content can be segmented to obtain N words, and then N word vectors representing the N words can be obtained through a text representation tool.

[0141] Step 202: For each word vector, based on each word vector and multiple local feature vectors, calculate the attention weights of the multiple local feature vectors using an attention mechanism model. The attention weights represent the degree to which the sample user pays attention to the region represented by the local feature vector when reading the words represented by each word vector.

[0142] The attention mechanism is a mechanism that dynamically controls the attention given to each part or a certain part of the neural network model by calculating the attention weights of each part in the neural network model and merging them into an attention vector.

[0143] Attention mechanisms are diverse, but generally fall into two categories: top-down conscious attention, known as focused attention, which refers to attention that is purposeful, task-dependent, and actively and consciously focused on a particular object; and bottom-up unconscious attention, known as saliency-based attention.

[0144] In addition, attention mechanisms include the following variants: multi-head attention, hard attention, key-value pair attention, and structured attention.

[0145] Among them, the multi-head attention mechanism uses multiple queries to compute multiple pieces of information from the input information in parallel, with each attention focusing on a different part of the input information.

[0146] It should be noted that the above-described attention mechanism and its variants are applicable in this embodiment.

[0147] The above process will be illustrated with specific examples below.

[0148] For example, using o j Let w represent the j-th local eigenvector. i Let represent the vector of the i-th word; based on this, the formula can be used. Calculate the word vector w i The attention weights of multiple local feature vectors, where, q represents the attention weight.m (·) and k m (·) denotes a linear transformation with a bias term, and k1 represents the number of local eigenvectors (i.e., the K1th one).

[0149] by Figure 5 For example, Figure 5 The sample images were categorized into news headlines, news categories, and news accessories. Figure 3 For each region, a local feature vector representing the three regions can be obtained. Taking the word "states" as an example, for the word vector representing the word "states", the attention weights of the local feature vectors of the three regions respectively represent the degree to which the sample user pays attention to the three regions when paying attention to the word "states".

[0150] Step 203: Based on the attention weights of each of the multiple local feature vectors, fuse each word vector with the multiple local feature vectors to obtain a first fused feature vector. Each word vector corresponds to a first fused feature vector.

[0151] Specifically, multiple local feature vectors can be weighted by their respective attention weights, and then the weighted result can be added to the word vectors to obtain the first fused feature vector.

[0152] The above process can be achieved through formulas Implementation, where v m (·) denotes a linear transformation with a bias term. This represents the first fused feature vector.

[0153] Step 204: Based on the user feature vector and the N first fusion feature vectors corresponding to the N word vectors, the prediction model is used to predict the preference of the sample user for each sample image. The input of the prediction model is determined based on the user feature vector and the N first fusion feature vectors. The user feature vector is used to characterize the user feature data of the sample user.

[0154] For example, for a sample image, the user feature vector and N first fused feature vectors can be directly input into the prediction model to obtain the user's preference for the sample image output by the prediction model; alternatively, the N first fused feature vectors can be processed first to obtain intermediate feature data, and then the user feature vector and the intermediate feature data can be input into the prediction model to obtain the user's preference for the sample image output by the prediction model.

[0155] As a feasible approach, such as Figure 7 As shown, step 204 includes:

[0156] Step 301: For each sample image, the N first fusion feature vectors corresponding to the N word vectors are processed by the self-attention mechanism model to obtain N semantic enhancement feature vectors, and each first fusion feature vector corresponds to a semantic enhancement feature vector.

[0157] The self-attention mechanism is an improvement on the attention mechanism. It reduces the reliance on external information and is better at capturing the internal correlations of data or features.

[0158] For example, in this embodiment, since the N first fusion feature vectors are obtained from N word vectors, and the words represented by the N word vectors come from the same candidate content, the self-attention mechanism can better analyze the correlation between the N first fusion feature vectors. Correspondingly, the attention mechanism is used to capture the correlation outside the data. As can be seen from the above description, the attention mechanism is used to process word vectors and multiple local feature vectors. Compared with the words represented by the word vectors, the image regions represented by the local feature vectors are external. Therefore, this embodiment captures the correlation between the words represented by the word vectors and the image regions represented by the local feature vectors through the attention mechanism.

[0159] Among them, self-attention mechanisms include single-head self-attention mechanisms and multi-head self-attention mechanisms.

[0160] It is understandable that since the N first fusion feature vectors are obtained from the N word vectors, and there are semantic relationships between the N word vectors, there are also semantic relationships between the N first fusion feature vectors. Therefore, in this embodiment, the N first fusion feature vectors are semantically enhanced by a self-attention mechanism model.

[0161] Specifically, the process of processing the N first fused feature vectors using a self-attention mechanism model can include: using the formula and The N first fused feature vectors are processed, where q(·) and k(·) represent linear transformations. Represents the j-th first fused feature vector For the i-th first fused feature vector The degree of semantic enhancement, where k2 represents the number of local feature vectors (i.e., the K2th one).

[0162] Step 302: Based on the user feature vector and N semantically enhanced feature vectors, a prediction model is used to predict the user's preference for each sample image. The input of the prediction model is determined based on the user feature vector and N semantically enhanced feature vectors.

[0163] For example, for a sample image, the user feature vector and N semantic enhancement feature vectors can be directly input into the prediction model to obtain the user's preference for the sample image output by the prediction model; alternatively, the N semantic enhancement feature vectors can be processed first to obtain intermediate feature data, and then the user feature vector and the intermediate feature data can be input into the prediction model to obtain the user's preference for the sample image output by the prediction model.

[0164] As one possible approach, step 302 includes:

[0165] For each sample image, the N semantically enhanced feature vectors are fused using an additive attention mechanism to obtain a second fused feature vector;

[0166] Based on user feature vectors and a second fused feature vector, a prediction model is used to predict the preference of sample users for each sample image. The input of the prediction model is determined based on user feature vectors and the second fused feature vector.

[0167] The model using an additive attention mechanism fuses N semantically enhanced feature vectors, including by employing the formula... and The N semantically enhanced feature vectors are fused and processed, k a Used to Transform into a latent space vector, q a Used to calculate attention weights during the fusion process. e1 represents the attention weight of the i-th semantic enhancement feature vector, e1 represents the second fused feature vector, and k3 represents the number of local feature vectors (i.e., the K3-th one).

[0168] Based on the above explanation, as one possible approach, such as Figure 8 As shown, the process of obtaining the second fused feature vector can be summarized as follows: taking word vectors and local feature vectors as input, and using attention mechanism, self-attention mechanism and additive attention mechanism in sequence, the second fused feature vector is output.

[0169] The above describes the case where image feature data includes local visual impression feature data. The following describes the case where image feature data includes global visual impression feature data.

[0170] As a feasible approach, such as Figure 9 As shown, step 104 includes:

[0171] Step 401: For each sample image, obtain a content feature vector based on the sample candidate content in each sample image. The content feature vector is used to characterize the sample candidate content.

[0172] Similar to the process of obtaining word vectors, text representation can also be used to transform sample candidate content into content feature vectors.

[0173] It is understandable that when the candidate sample content is news content, the title of the news content can usually reflect the main information of the news content well. Therefore, when the candidate sample content is news content, the title of the news content can be transformed into a title feature vector, and this title feature vector can be used as the content feature vector representing the candidate sample content.

[0174] Step 402: Determine the weights of the content feature vector and the global feature vector based on the content feature vector and the global feature vector.

[0175] It should be noted that there are multiple methods for determining the weights of the content feature vector and the global feature vector, and this application does not specifically limit these methods.

[0176] Since users may have different sensitivities to visual impressions and textual semantics, as a feasible approach, a threshold addition network can be used to adaptively control the weights of the content feature vector and the global feature vector.

[0177] Specifically, the process of controlling the weights of the content feature vector and the global feature vector using a threshold addition network includes: using the formula a = σ(g(o * The weights of the content feature vectors are calculated, and the weights of the global feature vectors are (1-a), where g(·) represents the linear transformation, σ represents the sigmoid function, e2 represents the content feature vector, and o * This represents the global feature vector; as can be seen from the formula, 'a' is composed of e² and o. * It is jointly determined and can be adjusted adaptively.

[0178] Step 403: Based on the weights of the content feature vector and the global feature vector, the content feature vector and the global feature vector are fused to obtain a third fused feature vector.

[0179] The above process can be described using formula e. * = a*e2+(1-a)*o * accomplish.

[0180] Based on the above explanation, it can be seen that, Figure 10 As shown, the process of obtaining the third fusion feature vector can be summarized as follows: taking the content feature vector and the global feature vector as input, and using a threshold addition network, the third fusion feature vector is output.

[0181] Step 404: Based on the user feature vector and the third fusion feature vector, the prediction model is used to predict the preference of the sample user for each sample image. The input of the prediction model is determined based on the user feature vector and the third fusion vector. The user feature vector is used to characterize the user feature data of the sample user.

[0182] In this embodiment, the user feature vector and the third fused feature vector can be directly input into the prediction model to predict the degree of preference for each sample image.

[0183] The recommended methods provided in the embodiments of this application are described below.

[0184] like Figure 11 As shown, this application provides an embodiment of a recommended method, which can be applied to a server or a terminal. Specifically, this embodiment includes:

[0185] Step 501: Obtain multiple images, each image containing a candidate interface and a candidate content presented through the candidate interface.

[0186] Step 502: Obtain image feature data for each image.

[0187] As one possible approach, each image comprises multiple regions, and correspondingly, the image feature data of each image comprises multiple local feature vectors, each of which is used to characterize a region.

[0188] As one possible approach, the image feature data for each image includes a global feature vector, which is used to characterize the image.

[0189] Step 503: Obtain user characteristic data of the target user.

[0190] Step 504: Based on the user feature data and image feature data of the target user, a prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature data and image feature data.

[0191] As a feasible approach, such as Figure 12 As shown, when the image feature data of each image includes multiple local feature vectors, step 504 includes:

[0192] Step 601: For each image, obtain N word vectors based on the candidate content in each image. Each word vector represents a word in the candidate content, where N is a positive integer.

[0193] Step 602: For each word vector, based on each word vector and multiple local feature vectors, and through an attention mechanism model, calculate the attention weights of the multiple local feature vectors respectively. The attention weights represent the degree to which the target user pays attention to the region represented by the local feature vector when reading the words represented by each word vector.

[0194] Step 603: Based on the attention weights of each of the multiple local feature vectors, fuse each word vector with the multiple local feature vectors to obtain a first fused feature vector. Each word vector corresponds to a first fused feature vector.

[0195] Step 604: Based on the user feature vector and the N first fusion feature vectors corresponding to the N word vectors, the prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and the N first fusion feature vectors. The user feature vector is used to represent the user feature data of the target user.

[0196] As a feasible approach, such as Figure 13 As shown, step 604 includes:

[0197] Step 701: For each image, the N first fusion feature vectors corresponding to the N word vectors are processed by the self-attention mechanism model to obtain N semantic enhancement feature vectors, and each first fusion feature vector corresponds to a semantic enhancement feature vector.

[0198] Step 702: Based on the user feature vector and N semantically enhanced feature vectors, a prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and N semantically enhanced feature vectors.

[0199] As one possible approach, step 702 includes:

[0200] For each image, the N semantically enhanced feature vectors are fused using an additive attention mechanism to obtain a second fused feature vector;

[0201] Based on user feature vectors and a second fused feature vector, a prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature vectors and the second fused feature vector.

[0202] As a feasible approach, such as Figure 14 As shown, when the image feature data of each image includes a global feature vector, step 504 includes:

[0203] Step 801: For each image, obtain a content feature vector based on the candidate content in each image. The content feature vector is used to represent the candidate content.

[0204] Step 802: Determine the weights of the content feature vector and the global feature vector based on the content feature vector and the global feature vector.

[0205] Step 803: Based on the weights of the content feature vector and the global feature vector, the content feature vector and the global feature vector are fused to obtain a third fused feature vector;

[0206] Step 804: Based on the user feature vector and the third fusion feature vector, the prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and the third fusion vector. The user feature vector is used to characterize the user feature data of the target user.

[0207] As one possible approach, step 804 includes:

[0208] Based on preference, select one candidate content from multiple images as the target candidate content;

[0209] Based on preference, a candidate interface is selected as the target candidate interface from the candidate interfaces of the image containing the target candidate content, so as to recommend the target candidate content through the target candidate interface.

[0210] It should be noted that steps 501 to 504 are similar to steps 101 to 104. For details, please refer to the relevant explanations of steps 101 and 103 in the previous text for understanding.

[0211] Step 505: Based on preference levels, select candidate content and / or candidate interface from the candidate interfaces and candidate content contained in multiple images for recommendation.

[0212] It should be noted that the preference-based program can select only candidate content from multiple images for recommendation, or only candidate interfaces from multiple images for recommendation, or it can select both candidate content and candidate interfaces from multiple images for recommendation simultaneously. This will be explained in detail below.

[0213] For example, such as Figure 15 As shown, user click history is obtained using user logs, and visual impressions of news are obtained using news materials and news interfaces. Then, through data preprocessing, local impression, global impression, and model prediction modules, the user's preference program for news is obtained. This preference degree specifically refers to the user's preference program for news content (i.e., candidate content) in the image. Finally, multiple images are sorted from high to low according to the preference degree, and the news content of the top M images is selected and recommended to the target user.

[0214] The data preprocessing module is used to execute steps 502 and 503, the local impression module is used to execute the fusion operations in steps 603, 701 and 702 to obtain the second fused feature vector, the global impression module is used to execute steps 802 and 803, and the model prediction module is used to execute the prediction operations in step 702 and step 804.

[0215] For example, such as Figure 16 As shown, the system obtains the current user's user-side features (i.e., user feature data), uses news materials and news interfaces to obtain multiple candidate news interface combinations (i.e., the multiple images mentioned above), and then processes them through a data preprocessing module, a local impression module, a global impression module, a model prediction module, and an interface generation module to obtain the user's preference program for news. This preference degree specifically refers to the user's preference program for the user interface in the image (i.e., the candidate interface). Finally, the multiple images are sorted from high to low according to the preference degree, and the user interface in the image with the highest preference degree (i.e., the best user interface) is selected, and then the best user interface configuration is generated. After that, the best user interface configuration can be displayed according to the best user interface configuration, and various content can be recommended to the current user through the best user interface.

[0216] The data preprocessing module is used to execute steps 502 and 503; the local impression module is used to execute the fusion operations in steps 603, 701 and 702 to obtain the second fused feature vector; the global impression module is used to execute steps 802 and 803; the model prediction module is used to execute the prediction operations in step 702 and step 804; and the interface generation module is used to generate the best user interface based on the prediction results of the model prediction module.

[0217] In addition, as one possible implementation, step 505 includes:

[0218] Based on preference, select one candidate content from multiple images as the target candidate content;

[0219] Based on preference, a candidate interface is selected as the target candidate interface from the candidate interfaces of the image containing the target candidate content, so as to recommend the target candidate content through the target candidate interface.

[0220] It should be noted that multiple candidate contents can be selected to recommend to the target user based on the degree of preference, and the target candidate content is one of the multiple candidate contents selected.

[0221] The above process will be illustrated with specific examples below.

[0222] For example, there are 4 images. The first image contains candidate content A and candidate interface A. The second image contains candidate content A and candidate interface B. The third image contains candidate content B and candidate interface A. The fourth image contains candidate content B and candidate interface B. The target user's preference for these 4 images from high to low is as follows: the first image, the second image, the fourth image, and the third image.

[0223] If the target candidate content is candidate content A, since the first image and the second image contain candidate content A, the target candidate interface is selected from the candidate interfaces of the first image and the second image. Since the target user has a higher preference for the first image than for the second image, candidate interface A in the first image is selected as the target candidate interface, and then candidate content A is recommended to the target user through candidate interface A.

[0224] Similarly, if the target candidate content is candidate content B, since the third and fourth images contain candidate content B, the target candidate interface is selected from the candidate interfaces of the fourth and third images; and since the target user has a higher preference for the fourth image than for the third image, the candidate interface B in the fourth image is selected as the target candidate interface, and then candidate content B is recommended to the target user through candidate interface B.

[0225] Therefore, it can be seen that the target candidate interface may be different for different target candidate content.

[0226] Step 506: Send the metadata of the target candidate interface and the target candidate content to the terminal device, so that the terminal device can display the target candidate interface based on the metadata and recommend the target candidate content to the target user through the target candidate interface.

[0227] Understandably, when the above method is executed by the server, the server will send the metadata of the target candidate interface and the target candidate content to the terminal device; accordingly, the terminal device will receive the metadata of the target candidate interface and the target candidate content, then display the target candidate interface based on the metadata, and recommend the target candidate content to the target user through the target candidate interface.

[0228] Please see Figure 17This application provides an embodiment of a recommendation device, comprising: a first image acquisition unit 601, configured to acquire multiple images, each image containing a candidate interface and candidate content presented through the candidate interface; a first feature data acquisition unit 602, configured to acquire image feature data for each image; a first prediction unit 603, configured to predict the target user's preference for each image based on the target user's user feature data and image feature data, and using a prediction model, wherein the input to the prediction model is determined based on the user feature data and image feature data; and a recommendation unit 604, configured to select candidate content and / or candidate interface from the candidate interfaces and candidate content contained in the multiple images based on the preference level, for recommendation.

[0229] As one possible approach, each image comprises multiple regions; the image feature data of each image includes multiple local feature vectors, each of which is used to characterize a region.

[0230] As one possible approach, the first prediction unit 603 is used to obtain N word vectors for each image based on the candidate content in each image, where each word vector represents a word in the candidate content, and N is a positive integer; for each word vector, based on each word vector and multiple local feature vectors, and through an attention mechanism model, calculate the attention weights of the multiple local feature vectors, where the attention weights represent the degree to which the target user pays attention to the region represented by the local feature vector when reading the word represented by each word vector; based on the attention weights of the multiple local feature vectors, fuse each word vector and the multiple local feature vectors to obtain a first fused feature vector, where each word vector corresponds to one first fused feature vector; based on the user feature vector and the N first fused feature vectors corresponding to the N word vectors, predict the target user's preference for each image through a prediction model, where the input of the prediction model is determined based on the user feature vector and the N first fused feature vectors, and the user feature vector is used to represent the user feature data of the target user.

[0231] As one possible approach, the first prediction unit 603 is used to process the N first fusion feature vectors corresponding to the N word vectors for each image through a self-attention mechanism model to obtain N semantic enhancement feature vectors, each of which corresponds to a semantic enhancement feature vector; based on the user feature vector and the N semantic enhancement feature vectors, the prediction model predicts the target user's preference for each image, and the input of the prediction model is determined based on the user feature vector and the N semantic enhancement feature vectors.

[0232] As one possible approach, the first prediction unit 603 is used to predict the target user's preference for each image based on the user feature vector and N semantically enhanced feature vectors through a prediction model. This includes: for each image, fusing the N semantically enhanced feature vectors through an additive attention mechanism to obtain a second fused feature vector; and predicting the target user's preference for each image based on the user feature vector and the second fused feature vector through a prediction model. The input of the prediction model is determined based on the user feature vector and the second fused feature vector.

[0233] As one possible approach, the image feature data for each image includes a global feature vector, which is used to characterize the image.

[0234] As one possible approach, the first prediction unit 603 is used to: obtain a content feature vector for each image based on candidate content in each image, the content feature vector being used to characterize the candidate content; determine the weights of the content feature vector and the global feature vector based on the content feature vector and the global feature vector; fuse the content feature vector and the global feature vector based on the weights of the content feature vector and the global feature vector to obtain a third fused feature vector; and predict the target user's preference for each image based on the user feature vector and the third fused feature vector, the input of the prediction model being determined based on the user feature vector and the third fused vector, the user feature vector being used to characterize the user feature data of the target user.

[0235] As one possible approach, the recommendation unit 604 is used to select a candidate content as the target candidate content from the candidate content contained in multiple images based on the degree of preference; and to select a candidate interface as the target candidate interface from the candidate interfaces of the image containing the target candidate content based on the degree of preference, so as to recommend the target candidate content through the target candidate interface.

[0236] As one possible approach, the apparatus also includes a sending unit 605, which sends metadata of the target candidate interface and target candidate content to the terminal device, so that the terminal device displays the target candidate interface based on the metadata and recommends the target candidate content to the target user through the target candidate interface.

[0237] For details on the implementation, related explanations, and technical effects of each of the above units, please refer to the description in the method section of the embodiments of this application.

[0238] Please see Figure 18This application provides an embodiment of a training device, comprising: a second image acquisition unit 701, configured to acquire multiple sample images, each sample image including a sample candidate interface and sample candidate content presented through the sample candidate interface; a second feature data acquisition unit 702, configured to acquire image feature data for each sample image; a second prediction unit 703, configured to predict the degree of preference of the sample user for each sample image based on the user feature data and image feature data of the sample user, and through a prediction model, wherein the input of the prediction model is determined based on the user feature data and image feature data; and an adjustment unit 704, configured to adjust the prediction model based on the degree of preference and the sample user's historical click data on the sample candidate content.

[0239] As one possible approach, each sample image comprises multiple regions; the image feature data of each sample image includes multiple local feature vectors, each of which is used to characterize a region.

[0240] As one possible approach, the second prediction unit 703 is used to obtain N word vectors for each sample image based on the sample candidate content in each sample image, where each word vector represents a word in the sample candidate content, and N is a positive integer; for each word vector, based on each word vector and multiple local feature vectors, and through an attention mechanism model, calculate the attention weights of the multiple local feature vectors, where the attention weights represent the degree to which the sample user pays attention to the region represented by the local feature vector when reading the word represented by each word vector; based on the attention weights of the multiple local feature vectors, fuse each word vector and the multiple local feature vectors to obtain a first fused feature vector, where each word vector corresponds to one first fused feature vector; based on the user feature vector and the N first fused feature vectors corresponding to the N word vectors, predict the sample user's preference for each sample image through a prediction model, where the input of the prediction model is determined based on the user feature vector and the N first fused feature vectors, and the user feature vector is used to represent the user feature data of the sample user.

[0241] As one possible approach, the second prediction unit 703 is used to process the N first fusion feature vectors corresponding to the N word vectors through a self-attention mechanism model for each sample image to obtain N semantic enhancement feature vectors, each of which corresponds to a semantic enhancement feature vector; based on the user feature vector and the N semantic enhancement feature vectors, the prediction model predicts the user's preference for each sample image, and the input of the prediction model is determined based on the user feature vector and the N semantic enhancement feature vectors.

[0242] As one possible approach, the second prediction unit 703 is used to fuse N semantically enhanced feature vectors for each sample image through an additive attention mechanism model to obtain a second fused feature vector; based on the user feature vector and the second fused feature vector, the prediction model predicts the user's preference for each sample image, and the input of the prediction model is determined based on the user feature vector and the second fused feature vector.

[0243] As one possible approach, the image feature data for each sample image includes a global feature vector, which is used to characterize the sample image.

[0244] As one possible approach, the second prediction unit 703 is used to: obtain a content feature vector for each sample image based on the candidate content in each sample image, the content feature vector being used to characterize the candidate content; determine the weights of the content feature vector and the global feature vector based on the content feature vector and the global feature vector; fuse the content feature vector and the global feature vector based on the weights of the content feature vector and the global feature vector to obtain a third fused feature vector; and predict the preference level of the sample user for each sample image based on the user feature vector and the third fused feature vector, the input of the prediction model being determined based on the user feature vector and the third fused vector, the user feature vector being used to characterize the user feature data of the sample user.

[0245] For details on the implementation, related explanations, and technical effects of each of the above units, please refer to the description in the method section of the embodiments of this application.

[0246] This application also provides an embodiment of a computer device, which can be a terminal or a server. When the computer device is a server, it can be used as a training device.

[0247] Please see Figure 19 , Figure 19 This is a schematic diagram of a computer device provided in an embodiment of this application, used to implement... Figure 17 The function of the recommended device in the corresponding embodiment or Figure 18In accordance with the function of the training device in the corresponding embodiment, specifically, the computer device 1800 is implemented by one or more servers. The computer device 1800 can vary considerably due to differences in configuration or performance, and may include one or more central processing units (CPUs) 1822 (e.g., one or more processors) and memory 1832, and one or more storage media 1830 (e.g., one or more mass storage devices) for storing application programs 1842 or data 1844. The memory 1832 and storage media 1830 can be temporary or persistent storage. The program stored in the storage media 1830 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the computer device. Furthermore, the CPU 1822 may be configured to communicate with the storage media 1830 and execute the series of instruction operations in the storage media 1830 on the computer device 1800.

[0248] The computer device 1800 may also include one or more power supplies 1826, one or more wired or wireless network interfaces 1850, one or more input / output interfaces 1858, and / or one or more operating systems 1841, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0249] In this embodiment of the application, the central processing unit 1822 can be used to execute... Figure 17 The recommended retrieval method executed by the corresponding embodiment. Specifically, the central processing unit 1822 can be used for:

[0250] Acquire multiple images, each containing a candidate interface and a candidate content presented through the candidate interface;

[0251] Obtain image feature data for each image;

[0252] Based on user feature data and image feature data of the target user, a prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on user feature data and image feature data.

[0253] Based on preference levels, candidate content and / or candidate interfaces are selected from candidate interfaces and candidate content contained in multiple images for recommendation.

[0254] In this embodiment of the application, the central processing unit 1822 can be used to execute... Figure 18 The corresponding embodiment describes the model training method executed by the training device. Specifically, the central processing unit 1822 can be used for:

[0255] Acquire multiple sample images, each sample image containing a sample candidate interface and a sample candidate content presented through the sample candidate interface;

[0256] Obtain the image feature data for each sample image;

[0257] Based on user feature data and image feature data of sample users, a prediction model is used to predict the degree of preference of sample users for each sample image. The input of the prediction model is determined based on user feature data and image feature data.

[0258] The prediction model is adjusted based on preference levels and historical click data of sample users on sample candidate content.

[0259] This application also provides a chip including one or more processors. Part or all of the processors are used to read and execute computer programs stored in a memory to perform the methods of the foregoing embodiments.

[0260] Optionally, the chip includes a memory, which is connected to the processor via a circuit or wire. Further optionally, the chip also includes a communication interface, to which the processor is connected. The communication interface is used to receive data and / or information that needs to be processed. The processor obtains the data and / or information from the communication interface, processes the data and / or information, and outputs the processing result through the communication interface. The communication interface can be an input / output interface.

[0261] In some implementations, some of the processors may implement some steps of the above method through dedicated hardware. For example, the processing of neural network models may be implemented by a dedicated neural network processor or graphics processor.

[0262] The method provided in this application embodiment can be implemented by a single chip or by multiple chips working together.

[0263] This application also provides a computer storage medium for storing computer software instructions for use with the aforementioned computer device, including instructions for executing programs designed for the computer device.

[0264] The computer device can be as described above. Figure 17 Recommended device or in corresponding embodiments Figure 18 The training device in the corresponding embodiment.

[0265] This application also provides a computer program product, which includes computer software instructions that can be loaded by a processor to implement the processes in the methods shown in the foregoing embodiments.

[0266] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0267] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some feature data may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0268] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0269] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0270] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A recommendation method, characterized in that, include: Acquire multiple images, each image containing a candidate interface and candidate content presented through the candidate interface. The candidate interface includes at least one of the following: image and text layout, whether an image is included and the size of the image, the color of the image, the clarity of the image, the font, and the font size. Image feature data of each of the multiple images is extracted, and the image feature data is related to the candidate interface and candidate content; each image includes multiple regions, and the image feature data of each image includes multiple local feature vectors and / or global feature vectors, each local feature vector is used to characterize a region, and the global feature vector is used to characterize the image; Based on the user feature data of the target user and the image feature data, a prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature data and the image feature data. Based on the stated preference level, candidate content and / or candidate interface are selected from the candidate interfaces and candidate content contained in the multiple images for recommendation. The step of selecting candidate content and / or candidate interface from the candidate interfaces and candidate content contained in the multiple images based on the preference level for recommendation includes: Based on the degree of preference, one candidate content is selected as the target candidate content from the candidate content contained in the multiple images; Based on the preference level, a candidate interface is selected as the target candidate interface from the candidate interfaces of the image containing the target candidate content, so as to recommend the target candidate content through the target candidate interface.

2. The method according to claim 1, characterized in that, The step of predicting the target user's preference for each image based on the target user's user feature data and the image feature data, and using a prediction model, includes: For each image, N word vectors are obtained based on the candidate content in each image, where each word vector represents a word in the candidate content, and N is a positive integer; For each of the word vectors, based on each of the word vectors and the plurality of local feature vectors, and through an attention mechanism model, the attention weights of the plurality of local feature vectors are calculated respectively. The attention weights represent the degree to which the target user pays attention to the region represented by the local feature vector when reading the words represented by each word vector. Based on the attention weights of the multiple local feature vectors, each word vector and the multiple local feature vectors are fused to obtain a first fused feature vector, and each word vector corresponds to one first fused feature vector; Based on the user feature vector and the N first fused feature vectors corresponding to the N word vectors, a prediction model is used to predict the target user's preference for each of the images. The input of the prediction model is determined based on the user feature vector and the N first fused feature vectors. The user feature vector is used to characterize the user feature data of the target user.

3. The method according to claim 2, characterized in that, The step of predicting the target user's preference for each image based on the user feature vector and the N first fused feature vectors corresponding to the N word vectors, and using a prediction model, includes: For each image, the N first fusion feature vectors corresponding to the N word vectors are processed by a self-attention mechanism model to obtain N semantic enhancement feature vectors, and each first fusion feature vector corresponds to a semantic enhancement feature vector. Based on the user feature vector and the N semantic enhancement feature vectors, a prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and the N semantic enhancement feature vectors.

4. The method according to claim 3, characterized in that, The step of predicting the target user's preference for each image based on the user feature vector and the N semantically enhanced feature vectors, using a prediction model, includes: For each image, the N semantic enhancement feature vectors are fused using an additive attention mechanism to obtain a second fused feature vector; Based on the user feature vector and the second fused feature vector, a prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and the second fused feature vector.

5. The method according to claim 1, characterized in that, The step of predicting the target user's preference for each image based on the target user's user feature data and the image feature data, and using a prediction model, includes: For each image, a content feature vector is obtained based on the candidate content in each image, and the content feature vector is used to characterize the candidate content; Based on the content feature vector and the global feature vector, determine the weights of the content feature vector and the global feature vector; Based on the weights of the content feature vector and the weights of the global feature vector, the content feature vector and the global feature vector are fused to obtain a third fused feature vector; Based on the user feature vector and the third fused feature vector, a prediction model is used to predict the target user's preference for each image. The input of the prediction model is determined based on the user feature vector and the third fused vector. The user feature vector is used to characterize the user feature data of the target user.

6. The method according to claim 1, characterized in that, After selecting a candidate interface as the target candidate interface from the candidate interfaces containing the image containing the target candidate content based on the preference level, the method further includes: The target candidate interface and the target candidate content are sent to the terminal device so that the terminal device can display the target candidate interface based on the metadata and recommend the target candidate content to the target user through the target candidate interface.

7. A training method, characterized in that, include: Multiple sample images are acquired, each of which includes a sample candidate interface and a sample candidate content presented through the sample candidate interface. The sample candidate interface includes at least one of the following: image and text layout, whether an image is included and the size of the image, the color of the image, the clarity of the image, the font, and the font size. Image feature data of each of the multiple sample images is extracted, and the image feature data is related to the sample candidate interface and sample candidate content; each sample image includes multiple regions, and the image feature data of each sample image includes multiple local feature vectors and / or global feature vectors, each local feature vector is used to characterize a region, and the global feature vector is used to characterize the sample image; Based on the user feature data of the sample users and the image feature data, a prediction model is used to predict the degree of preference of the sample users for each of the sample images. The input of the prediction model is determined based on the user feature data and the image feature data. Based on the preference level and the historical click data of the sample users on the sample candidate content, the prediction model is adjusted so that the adjusted prediction model can be used to obtain the user's preference level for multiple images. The preference level of multiple images is used to select one candidate content as the target candidate content from the candidate content contained in the multiple images. From the candidate interfaces of the image containing the target candidate content, a candidate interface is selected as the target candidate interface to recommend the target candidate content through the target candidate interface.

8. The method according to claim 7, characterized in that, The process of predicting the user's preference for each sample image based on the user feature data and the image feature data of the sample users, and using a prediction model, includes: For each of the sample images, N word vectors are obtained based on the sample candidate content in each sample image, and each word vector represents a word in the sample candidate content, where N is a positive integer; For each of the word vectors, based on each of the word vectors and the plurality of local feature vectors, and through an attention mechanism model, the attention weights of the plurality of local feature vectors are calculated respectively. The attention weights represent the degree to which the sample user pays attention to the region represented by the local feature vector when reading the words represented by each word vector. Based on the attention weights of the multiple local feature vectors, each word vector and the multiple local feature vectors are fused to obtain a first fused feature vector, and each word vector corresponds to one first fused feature vector; Based on the user feature vector and the N first fused feature vectors corresponding to the N word vectors, a prediction model is used to predict the preference of the sample user for each sample image. The input of the prediction model is determined based on the user feature vector and the N first fused feature vectors. The user feature vector is used to characterize the user feature data of the sample user.

9. The method according to claim 8, characterized in that, The step of predicting the user's preference for each sample image based on the user feature vector and the N first fused feature vectors corresponding to the N word vectors, and using a prediction model, includes: For each of the sample images, the N first fusion feature vectors corresponding to the N word vectors are processed by a self-attention mechanism model to obtain N semantic enhancement feature vectors, and each of the first fusion feature vectors corresponds to a semantic enhancement feature vector. Based on the user feature vector and the N semantic enhancement feature vectors, a prediction model is used to predict the preference of the sample user for each sample image. The input of the prediction model is determined based on the user feature vector and the N semantic enhancement feature vectors.

10. The method according to claim 9, characterized in that, The step of predicting the user's preference for each sample image based on the user feature vector and the N semantically enhanced feature vectors, using a prediction model, includes: For each of the sample images, the N semantic enhancement feature vectors are fused using an additive attention mechanism model to obtain a second fused feature vector; Based on the user feature vector and the second fused feature vector, a prediction model is used to predict the preference of the sample user for each sample image. The input of the prediction model is determined based on the user feature vector and the second fused feature vector.

11. The method according to claim 7, characterized in that, The process of predicting the user's preference for each sample image based on the user feature data and the image feature data of the sample users, and using a prediction model, includes: For each of the sample images, a content feature vector is obtained based on the sample candidate content in each of the sample images, and the content feature vector is used to characterize the sample candidate content; Based on the content feature vector and the global feature vector, determine the weights of the content feature vector and the global feature vector; Based on the weights of the content feature vector and the weights of the global feature vector, the content feature vector and the global feature vector are fused to obtain a third fused feature vector; Based on the user feature vector and the third fused feature vector, a prediction model is used to predict the preference of the sample user for each sample image. The input of the prediction model is determined based on the user feature vector and the third fused vector. The user feature vector is used to characterize the user feature data of the sample user.

12. A recommendation device, characterized in that, include: The first image acquisition unit is used to acquire multiple images, each of which includes a candidate interface and candidate content presented through the candidate interface. The candidate interface includes at least one of the following: image and text layout, whether an image is included and the size of the image, the color of the image, the clarity of the image, the font, and the font size. The first feature data acquisition unit is used to extract image feature data of each of the multiple images, wherein the image feature data is related to the candidate interface and candidate content; each image includes multiple regions, and the image feature data of each image includes multiple local feature vectors and / or global feature vectors, wherein each local feature vector is used to characterize one of the regions, and the global feature vector is used to characterize the image; The first prediction unit is used to predict the target user's preference for each image based on the target user's user feature data and the image feature data, and through a prediction model. The input of the prediction model is determined based on the user feature data and the image feature data. The recommendation unit is used to select candidate content and / or candidate interface from the candidate interfaces and candidate content contained in the multiple images based on the preference level, so as to make recommendations; The recommendation unit is specifically used to select one candidate content as the target candidate content from the candidate content contained in the multiple images based on the preference degree. Based on the preference level, a candidate interface is selected as the target candidate interface from the candidate interfaces of the image containing the target candidate content, so as to recommend the target candidate content through the target candidate interface.

13. A training device, characterized in that, include: The second image acquisition unit is used to acquire multiple sample images. Each sample image includes a sample candidate interface and a sample candidate content presented through the sample candidate interface. The sample candidate interface includes at least one of the following: graphic layout, whether an image is included and the size of the image, the color of the image, the clarity of the image, the font, and the font size. The second feature data acquisition unit is used to extract image feature data of each of the multiple sample images. The image feature data is related to the sample candidate interface and sample candidate content. Each sample image includes multiple regions. The image feature data of each sample image includes multiple local feature vectors and / or global feature vectors. Each local feature vector is used to characterize one of the regions, and the global feature vector is used to characterize the sample image. The second prediction unit is used to predict the degree of preference of the sample user for each sample image based on the user feature data of the sample user and the image feature data, and through a prediction model. The input of the prediction model is determined based on the user feature data and the image feature data. An adjustment unit is used to adjust the prediction model based on the preference level and the historical click data of the sample user on the sample candidate content, so as to use the adjusted prediction model to obtain the user's preference level for multiple images. The preference level of the multiple images is used to select one candidate content as the target candidate content from the candidate content contained in the multiple images. From the candidate interfaces of the image containing the target candidate content, a candidate interface is selected as the target candidate interface to recommend the target candidate content through the target candidate interface.

14. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory is used to store computer-readable instructions; and the processor is used to read the computer-readable instructions and implement the method as described in any one of claims 1-6.

15. A training device, characterized in that, The method includes a memory and a processor, wherein the memory is used to store computer-readable instructions; and the processor is used to read the computer-readable instructions and implement the method as described in any one of claims 7-11.

16. A computer storage medium, characterized in that, The device stores computer-readable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-11.

17. A computer program product, characterized in that, The computer program product includes computer-readable instructions that, when executed by a processor, implement the method as described in any one of claims 1-11.

18. A recommendation system, characterized in that, Including terminal devices and servers; The server is used to perform the method as described in any one of claims 1-6; The terminal device is used to receive metadata and target candidate content from the target candidate interface of the server; The target candidate interface is displayed based on the metadata, and the target candidate content is recommended to the target user through the target candidate interface.

Images