Content recommendation methods, devices, storage media and electronic devices

Abstract

This disclosure relates to the field of computer technology, specifically to a content recommendation method, a content recommendation device, a storage medium, and an electronic device. The content recommendation method includes: inputting content to be recommended into a pre-trained cross-modal model to obtain first modal feature vectors for each modality in the content to be recommended; inputting the first modal feature vectors and the tags of the content to be recommended into a pre-trained content representation model to obtain a first content feature vector for the content to be recommended; and if it is determined that the first content feature vector meets preset conditions, then recommending the content to be recommended to the user. This disclosure can improve the accuracy of recommendation results.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a content recommendation method, a content recommendation device, a storage medium, and an electronic device. Background Technology

[0002] With the development of internet technology and applications, the number of internet users is growing rapidly, generating a large amount of new content on various platforms every day, and the content format is becoming increasingly multimodal. How to utilize recommendation systems to distribute this new content more accurately and quickly to potential user groups is particularly important for the user experience and ecosystem development of content distribution platforms; this is precisely the problem that content cold start methods aim to solve.

[0003] In existing technologies, a common cold start solution for content is the Bandit algorithm. The core idea of ​​the Bandit algorithm is to try different strategies to recommend new content entering the recommendation system to different users, and then use user feedback to determine potentially interested user groups. Common Bandit algorithms include epsilon-greedy and LinUCB.

[0004] The epsilon-greedy algorithm recommends the content with the highest average return in the current recommendation pool with an epsilon probability, and randomly selects one content with a 1-epsilon probability. However, the epsilon-greedy algorithm does not consider content features when making recommendations. The LinUCB algorithm, on the other hand, acquires content features, constructs feature vectors, and makes recommendations based on these vectors. However, the LinUCB algorithm does not integrate multimodal features when acquiring content feature vectors. Therefore, the feature vectors acquired by the LinUCB algorithm do not fully represent the content features, leading to less accurate recommendation results.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a content recommendation method, apparatus, storage medium, and electronic device, thereby overcoming, to at least some extent, the problem of inaccurate recommendation results caused by limitations and defects in related technologies.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to a first aspect of this disclosure, a content recommendation method is provided, comprising:

[0009] The content to be recommended is input into a pre-trained cross-modal model to obtain the first modal feature vector of each modality in the content to be recommended;

[0010] The first modality feature vector and the label of the content to be recommended are input into a pre-trained content representation model to obtain the first content feature vector of the content to be recommended.

[0011] If it is determined that the first content feature vector meets the preset conditions, then the content to be recommended is recommended to the user.

[0012] In one exemplary embodiment of this disclosure, determining that the first content feature vector satisfies a preset condition includes:

[0013] When the similarity between the first content feature vector and the first user feature vector is greater than or equal to a preset similarity threshold, the first content feature vector is determined to meet the preset condition.

[0014] In one exemplary embodiment of this disclosure, before recommending the content to be recommended to the user, the method further includes:

[0015] Obtain user identification information and user profile data;

[0016] The user identification information and user profile data are input into the content representation model to obtain the first user feature vector.

[0017] In one exemplary embodiment of this disclosure, determining that the first content feature vector satisfies a preset condition includes:

[0018] When the similarity between the first content feature vector and the second content feature vector of the target content browsed in the user's history is greater than or equal to a preset similarity threshold, the content feature vector is determined to meet the preset condition.

[0019] In one exemplary embodiment of this disclosure, before recommending the content to be recommended to the user, the method further includes:

[0020] Obtain the target content that the user has browsed in the past;

[0021] The target content is input into a pre-trained cross-modal model to obtain the second modal feature vectors of each modality in the target content;

[0022] The second modality feature vector and the label of the target content are input into the pre-trained content representation model to obtain the second content feature vector of the target content.

[0023] In one exemplary embodiment of this disclosure, before inputting the first modality feature vector and the label of the content to be recommended into a pre-trained content representation model, the method further includes:

[0024] Obtain positive and negative samples;

[0025] Obtain the third content feature vector corresponding to the positive sample;

[0026] Obtain the fourth content feature vector corresponding to the negative sample;

[0027] The second user feature vector is obtained based on the positive and negative samples;

[0028] The loss function of the content representation model is constructed based on the second user feature vector, the third content feature vector, and the fourth content feature vector;

[0029] The content representation model is trained using the loss function, and the training of the content representation model is completed when the value of the loss function reaches its minimum.

[0030] In one exemplary embodiment of this disclosure, the content representation model includes a target convolutional neural network and a convolutional layer connected to the target convolutional neural network. The step of inputting the first modality feature vector and the label of the content to be recommended into the pre-trained content representation model to obtain the first content feature vector of the content to be recommended includes:

[0031] The first modality feature vector and the label of the content to be recommended are input into the global average pooling layer in the target convolutional neural network to obtain the compressed feature vector.

[0032] The compressed feature vector is input into the fully connected layer of the target convolutional neural network to obtain the first weight of the first modality feature vector and the second weight of the tag of the content to be recommended.

[0033] The weighted value of the first modality feature vector and the tag of the content to be recommended is calculated based on the first weight and the second weight.

[0034] The weighted value is input into the convolutional layer to extract features from the weighted value and obtain the first content feature vector of the content to be recommended.

[0035] According to a second aspect of this disclosure, a content recommendation device is provided, comprising:

[0036] The first modality feature vector acquisition module is used to input the content to be recommended into a pre-trained cross-modal model to obtain the first modality feature vector of each modality in the content to be recommended.

[0037] The first content feature vector acquisition module is used to input the first modality feature vector and the label of the content to be recommended into a pre-trained content representation model to obtain the first content feature vector of the content to be recommended.

[0038] The content recommendation module is used to recommend the content to the user if it is determined that the first content feature vector meets the preset conditions.

[0039] In one exemplary embodiment of this disclosure, the content recommendation module includes:

[0040] The first preset condition determination unit is used to determine that the first content feature vector satisfies the preset condition when the similarity between the first content feature vector and the first user feature vector is greater than or equal to a preset similarity threshold.

[0041] In one exemplary embodiment of this disclosure, the apparatus further includes:

[0042] The first user feature vector acquisition module is used for:

[0043] Obtain user identification information and user profile data;

[0044] The user identification information and user profile data are input into the content representation model to obtain the first user feature vector.

[0045] In one exemplary embodiment of this disclosure, the content recommendation module includes:

[0046] The first preset condition determination unit is used to determine that the content feature vector satisfies the preset condition when the similarity between the first content feature vector and the second content feature vector of the target content browsed in the user's history is greater than or equal to a preset similarity threshold.

[0047] In one exemplary embodiment of this disclosure, the apparatus further includes:

[0048] The second content feature vector acquisition module is used for:

[0049] Obtain the target content that the user has browsed in the past;

[0050] The target content is input into a pre-trained cross-modal model to obtain the second modal feature vectors of each modality in the target content;

[0051] The second modality feature vector and the label of the target content are input into the pre-trained content representation model to obtain the second content feature vector of the target content.

[0052] In one exemplary embodiment of this disclosure, the apparatus further includes:

[0053] The content representation model training module is used for:

[0054] Obtain positive and negative samples;

[0055] Obtain the third content feature vector corresponding to the positive sample;

[0056] Obtain the fourth content feature vector corresponding to the negative sample;

[0057] A second user feature vector is obtained based on the positive and negative samples;

[0058] The loss function of the content representation model is constructed based on the second user feature vector, the third content feature vector, and the fourth content feature vector;

[0059] The content representation model is trained using the loss function, and the training of the content representation model is completed when the value of the loss function reaches its minimum.

[0060] In one exemplary embodiment of this disclosure, the content representation model includes a target convolutional neural network and a convolutional layer connected to the target convolutional neural network, and the first content feature vector acquisition module includes:

[0061] The compressed feature vector acquisition unit is used to input the first modality feature vector and the label of the content to be recommended into the global average pooling layer in the target convolutional neural network to obtain the compressed feature vector.

[0062] The weight acquisition unit is used to input the compressed feature vector into the fully connected layer in the target convolutional neural network to obtain the first weight of the first modality feature vector and the second weight of the tag of the content to be recommended.

[0063] The weighted value acquisition unit is used to calculate the weighted value of the first modal feature vector and the tag of the content to be recommended based on the first weight and the second weight;

[0064] The first content feature vector acquisition unit is used to input the weighted value into the convolutional layer to extract features from the weighted value and obtain the first content feature vector of the content to be recommended.

[0065] According to a third aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any of the first aspects.

[0066] According to a fourth aspect of this disclosure, an electronic device is provided, comprising:

[0067] Processor; and

[0068] Memory for storing the executable instructions of the processor;

[0069] The processor is configured to perform the steps of any of the methods in the first aspect by executing the executable instructions.

[0070] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0071] In summary, the method provided in this disclosure obtains the first modal feature vectors of each modality in the content to be recommended by inputting the content to be recommended into a pre-trained cross-modal model; the first modal feature vectors and the labels of the content to be recommended are input into a pre-trained content representation model to obtain the first content feature vectors of the content to be recommended; if it is determined that the first content feature vectors meet preset conditions, the content to be recommended is recommended to the user. This method can obtain the content feature vectors of the content to be recommended based on the modal features of the content, so that the content feature vectors can fully reflect all the features of the content. Therefore, when the content feature vectors meet the preset conditions, the content to be recommended is recommended to the user, which greatly improves the accuracy of the recommendation results.

[0072] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0073] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0074] Figure 1 This is a flowchart of a content recommendation method provided according to an exemplary embodiment;

[0075] Figure 2 This is a block diagram of a content recommendation system provided according to an exemplary embodiment;

[0076] Figure 3 This is a schematic diagram of a CLIP model provided according to an exemplary embodiment;

[0077] Figure 4 This is a schematic diagram of a first DSSM sub-model provided according to an exemplary embodiment;

[0078] Figure 5 This is a block diagram of a content recommendation apparatus provided according to an exemplary embodiment;

[0079] Figure 6 This is a schematic diagram of a storage medium provided according to an exemplary embodiment;

[0080] Figure 7 This is a block diagram of an electronic device provided according to an exemplary embodiment.

[0081] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0082] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0083] Those skilled in the art will recognize that embodiments of the present invention can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0084] The data disclosed herein may be data authorized by the user or fully authorized by all parties. The collection, dissemination, and use of the data shall comply with the requirements of relevant national laws and regulations. The implementation methods / executives of this disclosure may be combined with each other.

[0085] To address the shortcomings of existing technologies, this example implementation first provides a content recommendation method. (Reference) Figure 1 As shown, the content recommendation method described above may include:

[0086] S11. Input the content to be recommended into a pre-trained cross-modal model to obtain the first modal feature vector of each modality in the content to be recommended;

[0087] S12. Input the first modality feature vector and the label of the content to be recommended into a pre-trained content representation model to obtain the first content feature vector of the content to be recommended.

[0088] S13. If it is determined that the first content feature vector meets the preset conditions, then the content to be recommended is recommended to the user.

[0089] In summary, the method provided in this disclosure can obtain a first content feature vector of the content to be recommended based on the modal features of the content to be recommended, so that the first content feature vector can fully reflect all the features of the content to be recommended. Then, when the first content feature vector meets the preset conditions, the content to be recommended is recommended to the user, which can greatly improve the accuracy of the content recommendation results.

[0090] The following will describe in more detail each step of the content recommendation method in this example embodiment, with reference to the accompanying drawings and embodiments.

[0091] In step S11, the content to be recommended is input into a pre-trained cross-modal model to obtain the first modal feature vector of each modality in the content to be recommended.

[0092] In one exemplary embodiment of this disclosure, reference is made to Figure 2 The system architecture shown may include: a user-side mobile terminal device 201, a user-side smart terminal device 204, and a server 203. Data transmission between the user-side mobile terminal device 201, the user-side smart terminal device 204, and the server 203 can all occur via network 202. The network may include various connection types, such as wired communication links, wireless communication links, etc. The content recommendation method described above can be executed on the server side, on the user-side terminal device, or by the user-side terminal device and the server in collaboration. Taking the method executed by the user-side terminal device and the server in collaboration as an example, after the user-side terminal device obtains the content to be recommended, it sends it to the server. After receiving the content to be recommended, the server can input the content to be recommended into a pre-trained cross-modal model to obtain the first modality feature vector of each modality in the content to be recommended; input the first modality feature vector and the label of the content to be recommended into a pre-trained content representation model to obtain the first content feature vector of the content to be recommended; if it is determined that the first content feature vector meets preset conditions, the content to be recommended is sent to the user-side terminal device, which then recommends it to the user.

[0093] In one exemplary embodiment of this disclosure, the content to be recommended is multimodal content, that is, content that simultaneously contains multiple modalities such as images, text, and sound. Previously, when acquiring content features, a corresponding unimodal model was trained to obtain the content features. Since the training data for the unimodal model is the training data for a single modality, the unimodal model trained on the unimodal training data can only acquire the unimodal features of the content to be recommended, and cannot comprehensively represent the information represented by the multiple modal features in the content to be recommended. To overcome the problem that the unimodal model trained on the unimodal training data can only acquire the unimodal features of the content to be recommended, in one exemplary embodiment of this disclosure, a pre-trained cross-modal model can be used to obtain the first modal feature vectors of each modality in the content to be recommended.

[0094] In one exemplary embodiment of this disclosure, the cross-modal model may employ a contrastive language-image pre-training (CLIP) model, which may be obtained by jointly training images and text.

[0095] In one exemplary embodiment of this disclosure, the CLIP model has the following structure: Figure 3 As shown, it includes a text encoder and an image encoder. The core of the CLIP model is the learning of a visual model supervised by natural language processing, with image-text pairs as input data for training. Specifically, the text encoder, after natural language processing, calculates the text feature vector embedding T. i ∈R d The image is processed by a computer vision image encoder to obtain the image feature vector embedding I. i ∈R d The CLIP model is trained by calculating the cosine similarity between image feature vector embeddings and text feature vector embeddings within a batch of size N, resulting in an N×N image-text similarity matrix. The CLIP model is considered complete when the similarity between image and text feature vectors belonging to a true pair within the same batch is maximized, and the similarity between image and text feature vectors not belonging to the same pair is minimized.

[0096] Furthermore, after the CLIP model is trained, the content to be recommended is input into the pre-trained cross-modal model to obtain the first text feature vector in the text of the content to be recommended and the first image feature vector in the recommended content.

[0097] In step S12, the first modality feature vector and the label of the content to be recommended are input into a pre-trained content representation model to obtain the first content feature vector of the content to be recommended.

[0098] In one exemplary embodiment of this disclosure, the content representation model is a Deep Structured Semantic Model (DSSM). The DSSM includes a first DSSM sub-model and a second DSSM sub-model, and the first and second DSSM sub-models have similar structures. The first DSSM sub-model is used to process the features of the content to be processed. To enable the model to acquire new content features, unlike conventional DSSM models, in this disclosure, the content identifier ID feature is removed from the input of the first DSSM sub-model, and only content information is used as input. The second DSSM sub-model is used to process user-related features.

[0099] In one exemplary embodiment of this disclosure, the first text feature vector in the text of the content to be recommended, the first image feature vector in the recommended content, and the tag of the content to be recommended are input into the first DSSM sub-model to obtain the first content feature vector of the content to be recommended.

[0100] In step S13, if it is determined that the first content feature vector meets the preset conditions, then the content to be recommended is recommended to the user.

[0101] In an exemplary embodiment of this disclosure, determining that the first content feature vector satisfies a preset condition includes:

[0102] When the similarity between the first content feature vector and the first user feature vector is greater than or equal to a preset similarity threshold, the first content feature vector is determined to meet the preset condition.

[0103] Based on the above, in an exemplary embodiment of this disclosure, before recommending the content to be recommended to the user, the method further includes:

[0104] S151. Obtain user identification information and user profile data;

[0105] S152. Input the user identification information and user profile data into the content representation model to obtain the first user feature vector.

[0106] In one exemplary embodiment of this disclosure, the user identification information and user profile data can be input into a second DSSM sub-model to obtain a first user feature vector.

[0107] In one exemplary embodiment of this disclosure, the user identification information may be username information. The user profile data may include data such as the user's age, gender, occupation, and preferences.

[0108] Based on the above, in an exemplary embodiment of this disclosure, determining that the first content feature vector satisfies a preset condition includes:

[0109] When the similarity between the first content feature vector and the second content feature vector of the target content browsed in the user's history is greater than or equal to a preset similarity threshold, the content feature vector is determined to meet the preset condition.

[0110] Based on the above, in an exemplary embodiment of this disclosure, before recommending the content to be recommended to the user, the method further includes:

[0111] S161. Obtain the target content that the user has browsed in the past;

[0112] S162. Input the target content into a pre-trained cross-modal model to obtain the second modal feature vector of each modality in the target content;

[0113] S163. Input the second modality feature vector and the label of the target content into the pre-trained content representation model to obtain the second content feature vector of the target content.

[0114] In one exemplary embodiment of this disclosure, target content viewed in the user's history is obtained, and then the target content is input into a pre-trained CLIP model to obtain a second text feature vector and a second image feature vector of the target content. Further, the second text feature vector, the second image feature vector, and the label of the target content are input into a first DSSM sub-model in a pre-trained DSSM model to obtain a second content feature vector of the target content.

[0115] Based on the above, in an exemplary embodiment of this disclosure, before inputting the first modality feature vector and the label of the content to be recommended into a pre-trained content representation model, the method further includes:

[0116] S171. Obtain positive and negative samples.

[0117] In one exemplary embodiment of this disclosure, the training data for the DSSM model is real user behavior data. Each user u who clicked on content i in the real user behavior data constitutes a positive sample (u, i) in the positive sample set P. + ), (u,i + )∈P. Negative samples (u,i) in the set N of negative samples. -It consists of other content that user u has clicked, (u, i - )∈N.

[0118] S172. Obtain the third content feature vector corresponding to the positive sample;

[0119] S173. Obtain the fourth content feature vector corresponding to the negative sample;

[0120] Furthermore, after obtaining positive and negative samples, a third content feature vector corresponding to the positive sample is obtained. Specifically, features of content i can be constructed based on the positive sample. in, in, Let i be the image feature vector of content i. Let i be the text feature vector of content i. Tags that can represent content i. Constructing the features of content i. Then, the characteristics of content i The third content feature vector corresponding to the positive sample is obtained by inputting the second DSSM sub-model into the DSSM model. in, Similarly, obtain the fourth content feature vector corresponding to the negative sample. Fourth content feature vector The acquisition process and the third content feature vector Similarly, this embodiment will not be described in detail here.

[0121] S174. Obtain the second user feature vector based on the positive samples and the negative samples;

[0122] In one exemplary embodiment of this disclosure, user features corresponding to user u can be constructed based on positive and negative samples. Among them, U id User identification information for user u, U profile User profile data for user u. User feature data. Input the second DSSM sub-model into the DSSM model to obtain the second user feature vector. in,

[0123] S175. Construct the loss function of the content representation model based on the second user feature vector, the third content feature vector, and the fourth content feature vector;

[0124] Furthermore, the third content feature vector is obtained. Fourth content feature vector and the second user feature vector Then, the loss function of the DSSM model is constructed using the following formula:

[0125]

[0126] Where L represents the loss function, This indicates that the third content feature vector is used to represent the feature vector. Fourth content feature vector The concatenated content feature vector, where K is the number of negative samples, w represents the weight matrix of the DSSM model, and b represents the bias vector of the DSSM model.

[0127] S176. The content representation model is trained using the loss function, and the training of the content representation model is completed when the value of the loss function reaches its minimum.

[0128] Furthermore, after obtaining the loss function L, the DSSM model is trained using the loss function L. Specifically, when training the DSSM model using the loss function L, the parameters of the DSSM model are adjusted, such as the weight matrix w and the bias vector b, so that the value of the loss function L reaches its minimum. The training of the DSSM model is completed when the value of the loss function L reaches its minimum.

[0129] In one exemplary embodiment of this disclosure, the content representation model includes a target convolutional neural network and a convolutional layer connected to the target convolutional neural network. The step of inputting the first modality feature vector and the label of the content to be recommended into the pre-trained content representation model to obtain the first content feature vector of the content to be recommended includes:

[0130] S111. Input the first modality feature vector and the label of the content to be recommended into the global average pooling layer in the target convolutional neural network to obtain the compressed feature vector.

[0131] S112. Input the compressed feature vector into the fully connected layer of the target convolutional neural network to obtain the first weight of the first modality feature vector and the second weight of the tag of the content to be recommended.

[0132] S113. Calculate the weighted value of the first modal feature vector and the tag of the content to be recommended based on the first weight and the second weight;

[0133] S114. Input the weighted value into the convolutional layer to extract features from the weighted value and obtain the first content feature vector of the content to be recommended.

[0134] The following section, using the structure of the first DSSM sub-model, explains how to obtain the first content feature vector of the content to be recommended. In an exemplary embodiment of this disclosure, the structure of the first DSSM sub-model is as follows: Figure 4 As shown, the first DSSM sub-model 40 includes a target convolutional neural network 41 and a convolutional layer 42 connected to the target convolutional neural network. The target convolutional neural network 41 can be a Squeeze-and-Excitation Network (SENet), including a global average pooling layer 411 and a fully connected layer 412 connected to the global average pooling layer 411.

[0135] In one exemplary embodiment of this disclosure, the first image feature vector, the first text feature vector, and the tags of the content to be recommended are first compressed using a global average pooling layer to obtain compressed feature vectors. Then, the compressed feature vectors are passed through a fully connected layer 412 for excitation to obtain the first weights corresponding to the first image feature vector and the first text feature vector, respectively, and the second weights of the tags of the content to be recommended.

[0136] Further, after obtaining the first weight and the second weight, the first weight is used as the coefficient of the first image feature vector and the first text feature vector, and the second weight is used as the coefficient of the tag of the content to be recommended. A weighted operation is performed on the first image feature vector, the first text feature vector, and the tag of the content to be recommended to obtain a weighted value. After obtaining the weighted value, the weighted value is input into multiple cascaded convolutional layers 42 to extract features from the weighted value, thereby obtaining the first content feature vector of the content to be recommended.

[0137] In summary, the method provided in this disclosure can obtain the content feature vector of the content to be recommended based on its various modal features. This ensures that the content feature vector fully reflects all the features of the content. When the content feature vector meets preset conditions, the content to be recommended is recommended to the user, significantly improving the accuracy of the recommendation results. Furthermore, when the similarity between the first content feature vector and the first user feature vector is greater than or equal to a preset similarity threshold, or when the similarity between the first content feature vector and the second content feature vector of the user's historically viewed target content is greater than or equal to a preset similarity threshold, it is determined that the content feature vector meets the preset conditions, and the content to be recommended is then recommended to the user. This method can push content to users based on user behavior data, making the pushed content more in line with user needs and improving the user experience.

[0138] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0139] Further reference Figure 5 As shown, this example embodiment also provides a content recommendation device, which includes: a first modality feature vector acquisition module 501, a first content feature vector acquisition module, and a content recommendation module 503. Wherein,

[0140] The first modality feature vector acquisition module 501 is used to input the content to be recommended into a pre-trained cross-modal model to obtain the first modality feature vector of each modality in the content to be recommended.

[0141] The first content feature vector acquisition module 502 is used to input the first modality feature vector and the label of the content to be recommended into a pre-trained content representation model to obtain the first content feature vector of the content to be recommended.

[0142] The content recommendation module 503 is used to recommend the content to the user if it is determined that the first content feature vector meets the preset conditions.

[0143] In one exemplary embodiment of this disclosure, the content recommendation module includes:

[0144] The first preset condition determination unit is used to determine that the first content feature vector satisfies the preset condition when the similarity between the first content feature vector and the first user feature vector is greater than or equal to a preset similarity threshold.

[0145] In one exemplary embodiment of this disclosure, the apparatus further includes:

[0146] The first user feature vector acquisition module is used for:

[0147] Obtain user identification information and user profile data;

[0148] The user identification information and user profile data are input into the content representation model to obtain the first user feature vector.

[0149] In one exemplary embodiment of this disclosure, the content recommendation module includes:

[0150] The first preset condition determination unit is used to determine that the content feature vector satisfies the preset condition when the similarity between the first content feature vector and the second content feature vector of the target content browsed in the user's history is greater than or equal to a preset similarity threshold.

[0151] In one exemplary embodiment of this disclosure, the apparatus further includes:

[0152] The second content feature vector acquisition module is used for:

[0153] Obtain the target content that the user has browsed in the past;

[0154] The target content is input into a pre-trained cross-modal model to obtain the second modal feature vectors of each modality in the target content;

[0155] The second modality feature vector and the label of the target content are input into the pre-trained content representation model to obtain the second content feature vector of the target content.

[0156] In one exemplary embodiment of this disclosure, the apparatus further includes:

[0157] The content representation model training module is used for:

[0158] Obtain positive and negative samples;

[0159] Obtain the first user feature vector and the third content feature vector corresponding to the positive sample;

[0160] Obtain the fourth content feature vector corresponding to the negative sample;

[0161] The loss function of the content representation model is constructed based on the first user feature vector, the third content feature vector, and the fourth content feature vector corresponding to the positive sample.

[0162] The content representation model is trained using the loss function, and the training of the content representation model is completed when the value of the loss function reaches its minimum.

[0163] In one exemplary embodiment of this disclosure, the content representation model includes a target convolutional neural network and a convolutional layer connected to the target convolutional neural network, and the first content feature vector acquisition module includes:

[0164] The compressed feature vector acquisition unit is used to input the first modality feature vector and the label of the content to be recommended into the global average pooling layer in the target convolutional neural network to obtain the compressed feature vector.

[0165] The weight acquisition unit is used to input the compressed feature vector into the fully connected layer in the target convolutional neural network to obtain the first weight of the first modality feature vector and the second weight of the tag of the content to be recommended.

[0166] The weighted value acquisition unit is used to calculate the weighted value of the first modal feature vector and the tag of the content to be recommended based on the first weight and the second weight;

[0167] The first content feature vector acquisition unit is used to input the weighted value into the convolutional layer to extract features from the weighted value and obtain the first content feature vector of the content to be recommended.

[0168] The specific details of each module in the above-mentioned content recommendation device have been described in detail in the corresponding content recommendation methods, so they will not be repeated here.

[0169] After introducing the content recommendation method and content recommendation apparatus according to exemplary embodiments of the present invention, the following will refer to... Figure 6 The storage medium of an exemplary embodiment of the present invention will be described.

[0170] refer to Figure 6 As shown, a program product 600 for implementing the above-described method according to an embodiment of the present invention is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a device such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0171] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0172] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0173] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0174] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0175] After introducing the storage medium of exemplary embodiments of the present invention, the following references are made. Figure 7 An electronic device according to an exemplary embodiment of the present invention will be described.

[0176] Figure 7 The electronic device 70 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0177] like Figure 7 As shown, the electronic device 70 is presented in the form of a general-purpose computing device. The components of the electronic device 70 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, a bus 730 connecting different system components (including storage unit 720 and processing unit 710), and a display unit 740.

[0178] The storage unit stores program code that can be executed by the processing unit 710, causing the processing unit 710 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 710 can perform actions such as... Figure 1Steps S11 to S13 are shown in the diagram.

[0179] Storage unit 720 may include volatile storage units, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include read-only memory (ROM) 7203. Storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0180] Bus 730 may include a data bus, an address bus, and a control bus.

[0181] Electronic device 70 can also communicate with one or more external devices 80 (e.g., keyboard, pointing device, Bluetooth device, etc.) via input / output (I / O) interface 750. Electronic device 70 also includes a display unit 740 connected to input / output (I / O) interface 750 for display purposes. Furthermore, electronic device 70 can communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 760. As shown, network adapter 1060 communicates with other modules of electronic device 70 via bus 730. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 70, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0182] It should be noted that although several modules or sub-modules of the content recommendation device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0183] Furthermore, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0184] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A content recommendation method characterized by, The method comprises the following steps: inputting the to-be-recommended content into a pre-trained cross-modal model to obtain a first modal feature vector of each modal in the to-be-recommended content; inputting the first modal feature vector and a label of the to-be-recommended content into a pre-trained content representation model to obtain a first content feature vector of the to-be-recommended content; wherein the content representation model is a deep semantic matching model (DSSM), the DSSM comprises a first DSSM sub-model and a second DSSM sub-model, the first DSSM sub-model comprises a target convolutional neural network and a convolutional layer connected to the target convolutional neural network; the inputting the first modal feature vector and the label of the to-be-recommended content into the pre-trained content representation model to obtain the first content feature vector of the to-be-recommended content comprises: inputting the first modal feature vector and the label of the to-be-recommended content into a global average pooling layer in the target convolutional neural network to obtain a compressed feature vector; inputting the compressed feature vector into a fully connected layer in the target convolutional neural network to obtain a first weight of the first modal feature vector and a second weight of the label of the to-be-recommended content; calculating a weighted value of the first modal feature vector and the label of the to-be-recommended content according to the first weight and the second weight; inputting the weighted value into the convolutional layer to extract features of the weighted value and obtain the first content feature vector of the to-be-recommended content; wherein the second DSSM sub-model is used to process features related to a user; if it is determined that the first content feature vector meets a preset condition, recommending the to-be-recommended content to the user.

2. The method of claim 1, wherein, The determination that the first content feature vector meets the preset condition comprises: when a similarity between the first content feature vector and a first user feature vector is greater than or equal to a preset similarity threshold, determining that the first content feature vector meets the preset condition.

3. The method of claim 2, wherein, Before the to-be-recommended content is recommended to the user, the method further comprises: obtaining user identification information and user portrait data; inputting the user identification information and the user portrait data into the content representation model to obtain the first user feature vector.

4. The method of claim 1, wherein, The determination that the first content feature vector meets the preset condition comprises: when a similarity between the first content feature vector and a second content feature vector of a target content browsed by the user in the past is greater than or equal to a preset similarity threshold, determining that the content feature vector meets the preset condition.

5. The method of claim 4, wherein, Before the to-be-recommended content is recommended to the user, the method further comprises: obtaining a target content browsed by the user in the past; inputting the target content into a pre-trained cross-modal model to obtain a second modal feature vector of each modal in the target content; inputting the second modal feature vector and a label of the target content into the pre-trained content representation model to obtain a second content feature vector of the target content.

6. The method of claim 1, wherein, Before the first modal feature vector and the label of the to-be-recommended content are inputted into the pre-trained content representation model, the method further comprises: obtaining positive samples and negative samples; obtaining a third content feature vector corresponding to the positive sample; obtaining a fourth content feature vector corresponding to the negative sample; obtaining a second user feature vector according to the positive sample and the negative sample; constructing a loss function of the content representation model according to the second user feature vector, the third content feature vector and the fourth content feature vector; training the content representation model by using the loss function, and completing the training of the content representation model when a value of the loss function reaches a minimum.

7. A content recommendation apparatus characterized by comprising: comprise: a first modal feature vector obtaining module, configured to input a to-be-recommended content into a pre-trained cross-modal model to obtain a first modal feature vector of each modal in the to-be-recommended content; a first content feature vector obtaining module, configured to input the first modal feature vector and a label of the to-be-recommended content into a pre-trained content representation model to obtain a first content feature vector of the to-be-recommended content; wherein the content representation model is a deep semantic matching model (DSSM), the DSSM comprises a first DSSM sub-model and a second DSSM sub-model, the first DSSM sub-model comprises a target convolutional neural network and a convolutional layer connected to the target convolutional neural network; the inputting of the first modal feature vector and the label of the to-be-recommended content into the pre-trained content representation model to obtain the first content feature vector of the to-be-recommended content comprises: inputting the first modal feature vector and the label of the to-be-recommended content into a global average pooling layer in the target convolutional neural network to obtain a compressed feature vector; inputting the compressed feature vector into a fully connected layer in the target convolutional neural network to obtain a first weight of the first modal feature vector and a second weight of the label of the to-be-recommended content; calculating a weighted value of the first modal feature vector and the label of the to-be-recommended content according to the first weight and the second weight; inputting the weighted value into the convolutional layer to extract features of the weighted value and obtain the first content feature vector of the to-be-recommended content; wherein the second DSSM sub-model is used to process features related to a user; a to-be-recommended content recommendation module, configured to recommend the to-be-recommended content to a user if it is determined that the first content feature vector meets a preset condition.

8. The apparatus of claim 7, wherein, The to-be-recommended content recommendation module comprises: a first preset condition determining unit, configured to determine that the first content feature vector meets a preset condition when a similarity between the first content feature vector and a first user feature vector is greater than or equal to a preset similarity threshold.

9. The apparatus of claim 8, wherein, The apparatus further comprises: a first user feature vector obtaining module, configured to: obtain user identification information and user portrait data; input the user identification information and the user portrait data into the content representation model to obtain the first user feature vector.

10. The apparatus of claim 7, wherein, The to-be-recommended content recommendation module comprises: The first preset condition determining unit is configured to determine that the content feature vector satisfies a preset condition when a similarity between the first content feature vector and a second content feature vector of a target content browsed by a user in history is greater than or equal to a preset similarity threshold.

11. The apparatus of claim 10, wherein, The device further comprises: The second content feature vector obtaining module is configured to: obtain a target content browsed by a user in history; input the target content into a pre-trained cross-modal model to obtain a second modal feature vector of each modal in the target content; input the second modal feature vector and a label of the target content into the pre-trained content representation model to obtain a second content feature vector of the target content.

12. The apparatus of claim 7, wherein, The device further comprises: The content representation model training module is configured to: obtain positive samples and negative samples; obtain a third content feature vector corresponding to the positive samples; obtain a fourth content feature vector corresponding to the negative samples; obtain a second user feature vector according to the positive samples and the negative samples; construct a loss function of the content representation model according to the second user feature vector, the third content feature vector, and the fourth content feature vector; train the content representation model by using the loss function, and complete the training of the content representation model when a value of the loss function reaches a minimum.

13. A computer readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the steps of the method of any one of claims 1 to 6.

14. An electronic device, comprising: comprises: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to implement the steps of the method of any one of claims 1 to 6 by executing the executable instructions.

Images