Method and device for training recall model, computer device and storage medium

Abstract

Embodiments of the present application disclose a method and device for training a recall model, a computer device and a storage medium. The method comprises obtaining user information and note information published by a user in a target platform, extracting a user feature vector of the user information, and extracting a note feature vector of the note information; obtaining a multi-modal vector corresponding to multi-modal information in the note information, and calculating a loss parameter based on the user feature vector, the note feature vector and the multi-modal vector; and adjusting a parameter of a preset recall model based on the loss parameter to obtain a target recall model. In this way, the content recall accuracy of the recall model can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a method, apparatus, computer device, and storage medium for training a recall model. Background Technology

[0002] Recommendation systems are systems used to recommend content that users are interested in, and are widely used in recommendation scenarios such as note-taking, audio and video recommendation, and shopping recommendation. The content recommendation process typically includes recall and ranking. Recall refers to filtering candidate content from the content database that matches the user's interests, while ranking is the process of finely sorting and ranking the candidate content.

[0003] In related technologies, deep learning is typically used to train content recall models for content retrieval. However, the content-related features used in these models are coarse-grained features such as categories and topics, which to some extent lose other modal information in the notes content and affect the recall accuracy of the model. Summary of the Invention

[0004] This application provides a training method, apparatus, computer device, and storage medium for a recall model, which can improve the content recall accuracy of the recall model.

[0005] This application provides a method for training a recall model, including:

[0006] Obtain user and media information from the target platform;

[0007] The user information is vectorized to obtain a first vector corresponding to the user information;

[0008] The media information is vectorized to obtain a second vector corresponding to the media information;

[0009] Obtain the multimodal vector corresponding to the multimodal information in the media information, and calculate the loss parameters of the preset recall model based on the first vector, the second vector and the multimodal vector;

[0010] The parameters of the preset recall model are adjusted based on the loss parameters to obtain the target recall model.

[0011] Accordingly, embodiments of this application also provide a training apparatus for a recall model, comprising:

[0012] The first acquisition unit is used to acquire user information and media information of the target platform;

[0013] The first processing unit is used to perform vectorization processing on the user information to obtain a first vector corresponding to the user information;

[0014] The second processing unit is used to perform vectorization processing on the media information to obtain a second vector corresponding to the media information;

[0015] The calculation unit is used to obtain the multimodal vector corresponding to the multimodal information in the media information, and calculate the loss parameters of the preset recall model based on the first vector, the second vector and the multimodal vector;

[0016] The adjustment unit is used to adjust the parameters of the preset recall model based on the loss parameters to obtain the target recall model.

[0017] In some embodiments, the first processing unit includes:

[0018] The first extraction subunit is used to extract features from the user information to obtain a user feature vector;

[0019] The first processing subunit is used to perform dimensionality reduction processing on the user feature vector to obtain the first vector.

[0020] In some embodiments, the second processing unit includes:

[0021] The second extraction subunit is used to extract features from the media information to obtain a media feature vector;

[0022] The second processing subunit is used to perform dimensionality reduction processing on the media feature vector to obtain the second vector.

[0023] In some embodiments, the computing unit includes:

[0024] The acquisition subunit is used to acquire text modal information and image modal information from the media information;

[0025] The third extraction subunit is used to extract features of the text modal information to obtain a text feature vector, and to extract features of the image modal information to obtain an image feature vector;

[0026] The third processing subunit is used to perform dimensionality reduction processing on the text feature vector and the image feature vector respectively to obtain the first modality vector corresponding to the text modality information and the second modality vector corresponding to the image modality information.

[0027] In some embodiments, the computing unit includes:

[0028] The first calculation subunit is used to calculate a first loss function based on the first vector and the second vector;

[0029] The second calculation subunit is used to calculate the second loss function based on the second vector, the first mode vector, and the second mode vector;

[0030] The third calculation subunit is used to calculate the third loss function based on the first mode vector and the second mode vector;

[0031] The fourth calculation subunit is used to calculate the loss parameters based on the first loss function, the second loss function, and the third loss function.

[0032] In some embodiments, the first computing subunit is specifically used for:

[0033] Perform a linear transformation on the first vector to obtain the predicted vector and the random deactivation vector corresponding to the first vector;

[0034] Perform a linear transformation on the second vector to obtain the predicted vector and the random deactivation vector corresponding to the second vector;

[0035] The first loss function is obtained by fitting the random deactivation vector corresponding to the second vector to the predicted vector corresponding to the first vector, and by fitting the random deactivation vector corresponding to the first vector to the predicted vector corresponding to the second vector.

[0036] In some embodiments, the second computing subunit is specifically used for:

[0037] A linear transformation is performed on the first mode vector to obtain the predicted vector and random deactivation vector corresponding to the first mode vector;

[0038] A linear transformation is performed on the second mode vector to obtain the predicted vector and random deactivation vector corresponding to the second mode vector;

[0039] The second loss function is obtained by fitting the random deactivation vector corresponding to the first mode vector and the random deactivation vector corresponding to the second mode vector with the predicted vector corresponding to the second vector.

[0040] In some embodiments, the third computing subunit is specifically used for:

[0041] The third loss function is obtained by fitting the random deactivation vector corresponding to the second mode vector to the predicted vector corresponding to the first mode vector, and by fitting the random deactivation vector corresponding to the first mode vector to the predicted vector corresponding to the second mode vector.

[0042] In some embodiments, the device further includes:

[0043] The third acquisition unit is used to acquire target user information and extract features from the target user information to obtain target user features;

[0044] The recall unit is used to input the target user features into the target recall model, and recall target media information that matches the target user features from the media information of the target platform based on the target recall model.

[0045] Accordingly, embodiments of this application also provide a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes a training method for any of the recall models provided in embodiments of this application.

[0046] Accordingly, embodiments of this application also provide a storage medium storing multiple instructions adapted for loading by a processor to execute the above-described training method for the recall model.

[0047] This application embodiment obtains user information and user-posted note information from the target platform, then extracts user feature vectors from the user information and note feature vectors from the note information; obtains multimodal vectors corresponding to multimodal information in the note information, and calculates loss parameters based on the user feature vectors, note feature vectors, and multimodal vectors. Then, based on the loss parameters, the parameters of a preset recall model are adjusted to obtain the target recall model. This improves the content recall accuracy of the recall model. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating a training method for a recall model provided in an embodiment of this application.

[0050] Figure 2 This is a schematic diagram illustrating an application scenario of a training method for a recall model provided in an embodiment of this application.

[0051] Figure 3 This is a structural block diagram of a training device for a recall model provided in an embodiment of this application.

[0052] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] This application provides a method, apparatus, storage medium, and computer device for training a recall model. Specifically, the training method for the recall model in this application can be executed by a computer device, which can be a terminal or a server. The terminal can be a smartphone, tablet, laptop, touchscreen, personal computer (PC), personal digital assistant (PDA), or other terminal device. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0055] For example, the computer device can be a server that can acquire user information and media information of the target platform; perform vectorization processing on the user information to obtain a first vector corresponding to the user information; perform vectorization processing on the media information to obtain a second vector corresponding to the media information; acquire the multimodal vector corresponding to the multimodal information in the media information, and calculate the loss parameters of the preset recall model based on the first vector, the second vector, and the multimodal vector; adjust the parameters of the preset recall model based on the loss parameters to obtain the target recall model.

[0056] Based on the above problems, this application provides a first method, apparatus, computer equipment, and storage medium for training a recall model, which can improve the content recall accuracy of the recall model.

[0057] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0058] This application provides a method for training a recall model, which can be executed by a terminal or a server. This application uses the example of a recall model training method executed by a terminal for illustration.

[0059] Please see Figure 1 , Figure 1This is a flowchart illustrating a training method for a recall model provided in an embodiment of this application. The specific process of training this recall model can be as follows:

[0060] 101. Obtain user information and media information from the target platform.

[0061] In this application embodiment, the target platform, namely the social networking platform, can be used by users for various social interactions. Social networking platforms can include various types, such as communication-based social platforms and content-sharing social platforms. Communication-based social platforms refer to social platforms primarily for user communication, while content-sharing social platforms refer to social platforms primarily for user-published content dissemination. In this application solution, the target platform can be a content-sharing social platform.

[0062] User information refers to information generated by users who register on the target platform during their use of the platform. For example, user information includes personal information and user behavior information. User behavior information can be the user's actions on the target platform, such as clicking on media information to view, liking media information, or saving media information.

[0063] Media information refers to content published by users through the target platform. This media information is presented in a multimedia format and may include text content and image content.

[0064] For example, media information can be content published through content-sharing social platforms. Users can share media information with other users by uploading it to such platforms.

[0065] 102. Perform vectorization on the user information to obtain the first vector corresponding to the user information.

[0066] In this embodiment of the application, the acquired user information needs to be transformed into data that can be processed by a machine learning model, that is, the user information is vectorized, wherein the first vector is a feature vector representing the user information.

[0067] In some embodiments, to improve model training efficiency, the step "vectorizing user information to obtain a first vector corresponding to the user information" may include the following operations:

[0068] Feature extraction is performed on user information to obtain user feature vectors;

[0069] The user feature vector is reduced in dimensionality to obtain the first vector.

[0070] Specifically, user information can be input into a user feature extraction model, which will then extract features from the user information and output the corresponding user feature vector.

[0071] Dimensionality reduction of user feature vectors can include feature fusion and feature compression.

[0072] In this embodiment of the application, user feature vectors can be input into the user tower model, and the user feature vectors can be fused and compressed by the user tower model to output a lower-dimensional (e.g., 64-dimensional) vector, namely the first vector, which can comprehensively represent the attributes and characteristics of a user.

[0073] 103. Perform vectorization on the media information to obtain the second vector corresponding to the media information.

[0074] In this embodiment of the application, the acquired media information needs to be transformed into data that can be processed by a machine learning model, that is, the media information is vectorized, wherein the second vector is a feature vector representing the media information.

[0075] In some embodiments, to improve model training efficiency, the step "vectorizing the media information to obtain a second vector corresponding to the media information" may include the following operations:

[0076] Feature extraction is performed on media information to obtain media feature vectors;

[0077] The media feature vector is reduced in dimensionality to obtain the second vector.

[0078] Specifically, media information can be input into a media feature extraction model, which will then extract features from the media information and output the corresponding media feature vector.

[0079] Dimensionality reduction of media feature vectors can include feature fusion and feature compression.

[0080] In this embodiment of the application, the media feature vector can be input into the media tower model, and the media feature vector can be fused and compressed by the media tower model to output a lower-dimensional (e.g., 64-dimensional) vector, that is, the second vector, which can comprehensively represent the attributes and characteristics of a piece of media information.

[0081] 104. Obtain the multimodal vector corresponding to the multimodal information in the media information, and calculate the loss parameters of the preset recall model based on the first vector, the second vector and the multimodal vector.

[0082] In this embodiment, media information may include multiple types of information, each type representing a modality. For example, media information may include image information and text information, thus encompassing two modalities. A multimodal vector refers to the feature vector corresponding to each modality of information within the media information.

[0083] In some embodiments, in order to improve the utilization rate of multimodal information in media information, the step of "obtaining the multimodal vector corresponding to the multimodal information in the media information" may include the following operations:

[0084] To acquire text modal information and image modal information from media information;

[0085] Features of text modal information are extracted to obtain text feature vectors, and features of image modal information are extracted to obtain image feature vectors;

[0086] The text feature vector and the image feature vector are dimensionality reduced respectively to obtain the first modality vector corresponding to the text modality information and the second modality vector corresponding to the image modality information.

[0087] Text modal information refers to the text content included in the media information, while image modal information refers to the image content included in the media information.

[0088] Furthermore, text feature extraction is performed on the acquired text modality information, and image feature extraction is performed on the acquired image modality information, thereby obtaining the text feature vector corresponding to the text modality information and the image feature vector corresponding to the image modality information.

[0089] In some embodiments, in order to associate text feature vectors and image feature vectors corresponding to the same media information, the text feature vectors and image feature vectors can be aligned and learned. Based on the learned text feature vectors and image feature vectors, the semantic understanding and alignment of text modal information and image modal information in the media information can be achieved.

[0090] Alternatively, in some embodiments, a trained cross-modal network model can be used to extract text features for text modality information and image features for image modality information, respectively. The cross-modal network model can be an image-text cross-modal network model, which can be used to extract feature vectors for images and feature vectors for text. Specifically, the text features and image features extracted by the image-text cross-modal network model have the same feature dimension.

[0091] For example, inputting image A into the image-text cross-modal network model allows the model to extract features from image A and output the image features of image A; inputting text B into the image-text cross-modal network model allows the model to extract features from text B and output the text features of text B.

[0092] In this embodiment of the application, training a preset cross-modal network model based on sample pairs can be achieved by performing multimodal alignment learning on the sample pairs to align image features with text features, thereby making the image features and text features output by the cross-modal network model have the same dimension.

[0093] Specifically, image-text alignment methods can infer the potential alignment between sentence fragments and image regions by learning the relationships between different modal features of the same instance, thereby achieving image-text matching. Image-text alignment methods focus more on local region fragment information of images and text.

[0094] Dimensionality reduction of text feature vectors can include feature fusion and feature compression.

[0095] In this embodiment, text feature vectors can be input into a natural language model, which then fuses and compresses them to output a lower-dimensional (e.g., 64-dimensional) vector, known as the first modality vector. This first modality vector can comprehensively represent the attributes and characteristics of the text modal information in a piece of media information.

[0096] Dimensionality reduction of image feature vectors can include feature fusion and feature compression.

[0097] In this embodiment, image feature vectors can be input into a visual model, which then fuses and compresses them to output a lower-dimensional (e.g., 64-dimensional) vector, known as the second modality vector. This second modality vector can comprehensively represent the attributes and characteristics of image modal information in a piece of media information.

[0098] In this embodiment, the recall effect of the recall model is improved by introducing the second modality vector obtained by the visual model and the first modality vector obtained by the natural language model into the recall model as input.

[0099] In some embodiments, to further improve the performance of the recall model, the step of "calculating the loss parameters of the preset recall model based on the first vector, the second vector, and the multimodal vector" may include the following operations:

[0100] Calculate the first loss function based on the first vector and the second vector;

[0101] The second loss function is calculated based on the second vector, the first mode vector, and the second mode vector;

[0102] The third loss function is calculated based on the first and second mode vectors;

[0103] The loss parameters are calculated based on the first loss function, the second loss function, and the third loss function.

[0104] The first loss function is calculated based on the first vector corresponding to user information and the second vector corresponding to media information. The purpose of this first loss function is to allow the recall model to learn to fit the user's preferences for notes.

[0105] In some embodiments, the step "calculate the first loss function based on the first vector and the second vector" may include the following operations:

[0106] Perform a linear transformation on the first vector to obtain the predicted vector and the random deactivation vector corresponding to the first vector;

[0107] Perform a linear transformation on the second vector to obtain the predicted vector and the random deactivation vector corresponding to the second vector;

[0108] The first loss function is obtained by fitting the random deactivation vector corresponding to the second vector to the predicted vector corresponding to the first vector, and by fitting the random deactivation vector corresponding to the first vector to the predicted vector corresponding to the second vector.

[0109] In this application embodiment, a contrastive learning builder is provided, which can be used to perform linear transformation processing on feature vectors to output the predicted vector and random deactivation vector corresponding to the feature vector.

[0110] The estimated vector can be calculated by multiplying the input vector (dimension d) with a preset parameter matrix (dimension d*d) to obtain another new d-dimensional vector, which is the estimated vector corresponding to the input vector.

[0111] The random deactivation vector can be calculated by setting each dimension of the input vector to zero with a small probability (e.g., 25%) (while keeping it unchanged with a large probability, i.e., 75%), resulting in a new d-dimensional vector, which is the random deactivation vector corresponding to the input vector.

[0112] Specifically, performing linear transformation on the first vector can include: inputting the first vector into a contrastive learning builder, multiplying the first vector with a preset parameter matrix through the contrastive learning builder to obtain the predicted vector corresponding to the first vector; and setting each dimension of the first vector to zero with a small probability through the contrastive learning builder to obtain a random deactivated vector of the first vector.

[0113] Specifically, linear transformation of the second vector can include: inputting the second vector into a contrastive learning builder, multiplying the second vector with a preset parameter matrix to obtain the predicted vector corresponding to the second vector; and setting each dimension of the second vector to zero with a small probability through the contrastive learning builder to obtain a random deactivated vector of the second vector.

[0114] The process of fitting the random deactivation vector corresponding to the second vector based on the predicted vector corresponding to the first vector may include matching the predicted vector of the first vector with the random deactivation vector of the second vector and using it to train the model parameters of the user tower model, so that the predicted vector of the first vector can approximate the random deactivation vector of the second vector.

[0115] Specifically, fitting the predicted vector corresponding to the first vector to the estimated vector corresponding to the second vector can include matching the predicted vector of the second vector with the random deactivation vector of the first vector, and using this matching to train the model parameters of the media tower model, so that the predicted vector of the second vector can approximate the random deactivation vector of the first vector. Thus, based on the fitting of the predicted vector of the first vector and the random deactivation vector with the random deactivation vector, a first loss function is obtained.

[0116] The second loss function is calculated based on the second vector corresponding to the media information, the first modal vector corresponding to the text modal information in the media information, and the second modal vector corresponding to the image modal information. The purpose of the second loss function is to make the second vector of the media information move closer to the first modal vector and the second modal vector corresponding to the media information.

[0117] In some embodiments, the step "calculate the second loss function based on the second vector, the first mode vector, and the second mode vector" may include the following operations:

[0118] A linear transformation is performed on the first mode vector to obtain the predicted vector and the random deactivation vector corresponding to the first mode vector;

[0119] A linear transformation is performed on the second mode vector to obtain the predicted vector and random deactivation vector corresponding to the second mode vector;

[0120] The second loss function is obtained by fitting the random deactivation vector corresponding to the first mode vector and the random deactivation vector corresponding to the second mode vector with the predicted vector corresponding to the second vector.

[0121] Specifically, linear transformation of the first modality vector may include: inputting the first modality vector into a contrastive learning builder, multiplying the first modality vector with a preset parameter matrix through the contrastive learning builder to obtain the predicted vector corresponding to the first modality vector; and setting each dimension of the first modality vector to zero with a small probability through the contrastive learning builder to obtain a random deactivation vector of the first modality vector.

[0122] Specifically, linear transformation of the second modality vector can include: inputting the second modality vector into a contrastive learning builder, multiplying the second modality vector with a preset parameter matrix through the contrastive learning builder to obtain the predicted vector corresponding to the second modality vector; and setting each dimension of the second modality vector to zero with a small probability through the contrastive learning builder to obtain a random deactivation vector of the second modality vector.

[0123] The fitting of the predicted vectors corresponding to the first and second modal vectors with the predicted vectors corresponding to the second vector can include: matching the predicted vector of the second vector with the random inactivation vector of the first modal vector and using this matching to train the model parameters of the media tower model, so that the predicted vector of the second vector can approximate the random inactivation vector of the first modal vector; and matching the predicted vector of the second vector with the random inactivation vector of the second modal vector and using this matching to train the model parameters of the media tower model, so that the predicted vector of the second vector can approximate the random inactivation vector of the second modal vector. Thus, based on the fitting of the second vector with the predicted vectors and random inactivation vectors of the first and second modal vectors, a second loss function is obtained.

[0124] The third loss function is calculated based on the first modality vector corresponding to the text modality information and the second modality vector corresponding to the image modality information. The purpose of this third loss function is to enable the recall model to learn to fit the image modality information and the text modality information.

[0125] In some embodiments, the step "calculate the third loss function based on the first mode vector and the second mode vector" may include the following operations:

[0126] The third loss function is obtained by fitting the random deactivation vector corresponding to the second mode vector to the predicted vector corresponding to the first mode vector, and by fitting the random deactivation vector corresponding to the first mode vector to the predicted vector corresponding to the second mode vector.

[0127] Fitting the random deactivation vector corresponding to the second modality vector based on the predicted vector corresponding to the first modality vector can include matching the predicted vector of the first modality vector with the random deactivation vector of the second modality vector and using it to train the model parameters of the natural language model, so that the predicted vector of the first modality vector can approximate the random deactivation vector of the second modality vector.

[0128] The process of fitting the predicted vector corresponding to the first modality vector to the predicted vector corresponding to the second modality vector can include matching the predicted vector of the second modality vector with the random inactivation vector of the first modality vector, and using this matching to train the model parameters of the visual model, so that the predicted vector of the second modality vector can approximate the random inactivation vector of the first modality vector. Thus, based on the fitting of the predicted vector of the first modality vector and the random inactivation vector with the random inactivation vector, a third loss function is obtained.

[0129] Based on the calculation of the first loss function, the second loss function, and the third loss function, the loss parameters can be obtained by summing the first loss function, the second loss function, and the third loss function.

[0130] 105. Adjust the parameters of the preset recall model based on the loss parameters to obtain the target recall model.

[0131] Specifically, the model parameters of the preset recall model are adjusted based on the loss parameters obtained in the above steps to obtain the trained recall model, which is the target model.

[0132] In this embodiment of the application, after the model parameters are calculated, the gradient information can be determined based on the model parameters, that is, the adjustment direction of the parameters of each layer in the model can be determined. The gradient information is gradually rotated back to update the model parameters of each layer in order to complete the adjustment of the model parameters.

[0133] In some embodiments, this scheme designs a model training step for the recall model; please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram illustrating an application scenario of a training method for a recall model provided in an embodiment of this application. For example... Figure 2 As shown, the training steps for the recall model include:

[0134] User characteristics e u The input user tower is processed to obtain the user vector h. u , will the user vector h u Input a contrastive learning builder, and then use the contrastive learning builder to process the user vector h. u Perform a linear transformation to obtain the user vector h. u Corresponding prediction vector and random deactivation vector

[0135] The note features (notes can be media information including images and text) e i The input to the note-taking tower is processed to obtain the note vector h. i , transfer the note vector h i Input a contrastive learning builder, and then process the note vector h using the contrastive learning builder. i Perform a linear transformation to obtain the note vector h. i Corresponding prediction vector and random deactivation vector

[0136] Visual features of notes v The input visual model is processed to obtain the visual vector h. v , the visual vector h v Input a contrastive learning builder, and then process the visual vector h through the contrastive learning builder. v A linear transformation is performed to obtain the visual vector h. v Corresponding prediction vector and random deactivation vector

[0137] The note text features e t The input is processed by a natural language model to obtain the text vector h. t , the text vector h t Input a contrastive learning builder, and use the contrastive learning builder to process the text vector h. t Perform a linear transformation to obtain the text vector h. t Corresponding prediction vector and random deactivation vector

[0138] For user side vector pairs Behavioral vector pairs with note-taking side Calculate the cross-entropy loss (Loss1) by fitting the random inactivation vector on the other side with the predicted vectors on both sides respectively;

[0139] Content vector pairs for the note-taking side and multimodal vector pairs Calculate the cross-entropy loss (Loss2) and fit the predicted vectors from the note side to the random deactivation vectors from the visual and text sides.

[0140] For two multimodal vector pairs of vision and text and Calculate the cross-entropy loss (Loss3) by fitting the random inactivation vector on the other side with the predicted vectors on both sides respectively;

[0141] Calculate the loss parameter Loss = Loss1 + Loss2 + Loss3, and calculate the gradient backpropagation based on Loss to update the model parameters of the preset recall model.

[0142] In some embodiments, to improve the recall accuracy of the recall model, the following steps may also be included:

[0143] Obtain target user information and extract features from it to obtain target user features;

[0144] The target user characteristics are input into the target recall model, and the target media information that matches the target user characteristics is recalled from the media information of the target platform based on the target recall model.

[0145] Among them, target user information refers to the information of a specific user on the target platform who needs to be recommended media content.

[0146] Specifically, feature extraction is performed on target user information to obtain target user features. These features are then input into a target recall model, which calculates media features that match the target user features. Finally, the media information corresponding to these media features is retrieved from all media information on the target platform and used as candidate media information, or target information, to recommend to the target user. This improves the accuracy of information recommendations to users.

[0147] This application discloses a training method for a recall model. The method includes: acquiring user information and media information from a target platform; vectorizing the user information to obtain a first vector corresponding to the user information; vectorizing the media information to obtain a second vector corresponding to the media information; acquiring the multimodal vector corresponding to the multimodal information in the media information; and calculating loss parameters of a preset recall model based on the first vector, the second vector, and the multimodal vector; and adjusting the parameters of the preset recall model based on the loss parameters to obtain a target recall model. This can improve the content recall accuracy of the recall model.

[0148] To facilitate better implementation of the recall model training method provided in this application embodiment, this application embodiment also provides a recall model training device based on the above-described recall model training method. The meanings of the terms used are the same as in the above-described recall model training method, and specific implementation details can be found in the description of the method embodiment.

[0149] Please see Figure 3 , Figure 3 This application provides a structural block diagram of a training device for a recall model, which includes:

[0150] The first acquisition unit 301 is used to acquire user information and media information of the target platform;

[0151] The first processing unit 302 is used to perform vectorization processing on the user information to obtain a first vector corresponding to the user information;

[0152] The second processing unit 303 is used to perform vectorization processing on the media information to obtain a second vector corresponding to the media information;

[0153] The calculation unit 304 is used to obtain the multimodal vector corresponding to the multimodal information in the media information, and calculate the loss parameters of the preset recall model based on the first vector, the second vector and the multimodal vector;

[0154] The adjustment unit 305 is used to adjust the parameters of the preset recall model based on the loss parameters to obtain the target recall model.

[0155] In some embodiments, the first processing unit 302 may include:

[0156] The first extraction subunit is used to extract features from the user information to obtain a user feature vector;

[0157] The first processing subunit is used to perform dimensionality reduction processing on the user feature vector to obtain the first vector.

[0158] In some embodiments, the second processing unit 303 may include:

[0159] The second extraction subunit is used to extract features from the media information to obtain a media feature vector;

[0160] The second processing subunit is used to perform dimensionality reduction processing on the media feature vector to obtain the second vector.

[0161] In some embodiments, the computing unit 304 may include:

[0162] The acquisition subunit is used to acquire text modal information and image modal information from the media information;

[0163] The third extraction subunit is used to extract features of the text modal information to obtain a text feature vector, and to extract features of the image modal information to obtain an image feature vector;

[0164] The third processing subunit is used to perform dimensionality reduction processing on the text feature vector and the image feature vector respectively to obtain the first modality vector corresponding to the text modality information and the second modality vector corresponding to the image modality information.

[0165] In some embodiments, the computing unit 304 may include:

[0166] The first calculation subunit is used to calculate a first loss function based on the first vector and the second vector;

[0167] The second calculation subunit is used to calculate the second loss function based on the second vector, the first mode vector, and the second mode vector;

[0168] The third calculation subunit is used to calculate the third loss function based on the first mode vector and the second mode vector;

[0169] The fourth calculation subunit is used to calculate the loss parameters based on the first loss function, the second loss function, and the third loss function.

[0170] In some embodiments, the first computing subunit may specifically be used for:

[0171] Perform a linear transformation on the first vector to obtain the predicted vector and the random deactivation vector corresponding to the first vector;

[0172] Perform a linear transformation on the second vector to obtain the predicted vector and the random deactivation vector corresponding to the second vector;

[0173] The first loss function is obtained by fitting the random deactivation vector corresponding to the second vector to the predicted vector corresponding to the first vector, and by fitting the random deactivation vector corresponding to the first vector to the predicted vector corresponding to the second vector.

[0174] In some embodiments, the second computing subunit may specifically be used for:

[0175] A linear transformation is performed on the first mode vector to obtain the predicted vector and random deactivation vector corresponding to the first mode vector;

[0176] A linear transformation is performed on the second mode vector to obtain the predicted vector and random deactivation vector corresponding to the second mode vector;

[0177] The second loss function is obtained by fitting the random deactivation vector corresponding to the first mode vector and the random deactivation vector corresponding to the second mode vector with the predicted vector corresponding to the second vector.

[0178] In some embodiments, the third computing subunit may specifically be used for:

[0179] The third loss function is obtained by fitting the random deactivation vector corresponding to the second mode vector to the predicted vector corresponding to the first mode vector, and by fitting the random deactivation vector corresponding to the first mode vector to the predicted vector corresponding to the second mode vector.

[0180] In some embodiments, the device may further include:

[0181] The third acquisition unit is used to acquire target user information and extract features from the target user information to obtain target user features;

[0182] The recall unit is used to input the target user features into the target recall model, and recall target media information that matches the target user features from the media information of the target platform based on the target recall model.

[0183] This application discloses a training device for a recall model. A first acquisition unit 301 acquires user information and media information of a target platform; a first processing unit 302 vectorizes the user information to obtain a first vector corresponding to the user information; a second processing unit 303 vectorizes the media information to obtain a second vector corresponding to the media information; a calculation unit 304 acquires the multimodal vector corresponding to the multimodal information in the media information and calculates the loss parameters of a preset recall model based on the first vector, the second vector, and the multimodal vector; an adjustment unit 305 adjusts the parameters of the preset recall model based on the loss parameters to obtain a target recall model. This improves the content recall accuracy of the recall model.

[0184] Accordingly, embodiments of this application also provide a computer device, which can be a server. For example... Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device 500 includes a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, and a computer program stored on the memory 502 and executable on the processor. The processor 501 and the memory 502 are electrically connected. Those skilled in the art will understand that the computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0185] The processor 501 is the control center of the computer device 500. It connects various parts of the computer device 500 through various interfaces and lines. By running or loading software programs and / or modules stored in the memory 502, and calling data stored in the memory 502, it performs various functions of the computer device 500 and processes data, thereby monitoring the computer device 500 as a whole.

[0186] In this embodiment, the processor 501 in the computer device 500 loads the instructions corresponding to the processes of one or more applications into the memory 502 according to the following steps, and the processor 501 runs the applications stored in the memory 502 to achieve various functions:

[0187] Obtain user and media information from the target platform;

[0188] The user information is vectorized to obtain the first vector corresponding to the user information;

[0189] The media information is vectorized to obtain the second vector corresponding to the media information;

[0190] Obtain the multimodal vector corresponding to the multimodal information in the media information, and calculate the loss parameters of the preset recall model based on the first vector, the second vector and the multimodal vector;

[0191] The target recall model is obtained by adjusting the parameters of the preset recall model based on the loss parameters.

[0192] In some embodiments, the user information is vectorized to obtain a first vector corresponding to the user information, including:

[0193] Feature extraction is performed on user information to obtain user feature vectors;

[0194] The user feature vector is reduced in dimensionality to obtain the first vector.

[0195] In some embodiments, the media information is vectorized to obtain a second vector corresponding to the media information, including:

[0196] Feature extraction is performed on media information to obtain media feature vectors;

[0197] The media feature vector is reduced in dimensionality to obtain the second vector.

[0198] In some embodiments, obtaining the multimodal vector corresponding to the multimodal information in the media information includes:

[0199] To acquire text modal information and image modal information from media information;

[0200] Features of text modal information are extracted to obtain text feature vectors, and features of image modal information are extracted to obtain image feature vectors;

[0201] The text feature vector and the image feature vector are dimensionality reduced respectively to obtain the first modality vector corresponding to the text modality information and the second modality vector corresponding to the image modality information.

[0202] In some embodiments, calculating the loss parameters of a preset recall model based on the first vector, the second vector, and the multimodal vector includes:

[0203] Calculate the first loss function based on the first vector and the second vector;

[0204] The second loss function is calculated based on the second vector, the first mode vector, and the second mode vector;

[0205] The third loss function is calculated based on the first and second mode vectors;

[0206] The loss parameters are calculated based on the first loss function, the second loss function, and the third loss function.

[0207] In some embodiments, calculating a first loss function based on a first vector and a second vector includes:

[0208] Perform a linear transformation on the first vector to obtain the predicted vector and the random deactivation vector corresponding to the first vector;

[0209] Perform a linear transformation on the second vector to obtain the predicted vector and the random deactivation vector corresponding to the second vector;

[0210] The first loss function is obtained by fitting the random deactivation vector corresponding to the second vector to the predicted vector corresponding to the first vector, and by fitting the random deactivation vector corresponding to the first vector to the predicted vector corresponding to the second vector.

[0211] In some embodiments, calculating a second loss function based on a second vector, a first mode vector, and a second mode vector includes:

[0212] A linear transformation is performed on the first mode vector to obtain the predicted vector and the random deactivation vector corresponding to the first mode vector;

[0213] A linear transformation is performed on the second mode vector to obtain the predicted vector and random deactivation vector corresponding to the second mode vector;

[0214] The second loss function is obtained by fitting the random deactivation vector corresponding to the first mode vector and the random deactivation vector corresponding to the second mode vector with the predicted vector corresponding to the second vector.

[0215] In some embodiments, calculating a third loss function based on a first mode vector and a second mode vector includes:

[0216] The third loss function is obtained by fitting the random deactivation vector corresponding to the second mode vector to the predicted vector corresponding to the first mode vector, and by fitting the random deactivation vector corresponding to the first mode vector to the predicted vector corresponding to the second mode vector.

[0217] In some embodiments, it also includes:

[0218] Obtain target user information and extract features from it to obtain target user features;

[0219] The target user characteristics are input into the target recall model, and the target media information that matches the target user characteristics is recalled from the media information of the target platform based on the target recall model.

[0220] This application embodiment obtains user information and user-posted note information from the target platform, then extracts user feature vectors from the user information and note feature vectors from the note information; obtains multimodal vectors corresponding to multimodal information in the note information, and calculates loss parameters based on the user feature vectors, note feature vectors, and multimodal vectors. Then, based on the loss parameters, the parameters of a preset recall model are adjusted to obtain the target recall model. This improves the content recall accuracy of the recall model.

[0221] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0222] Optional, such as Figure 4 As shown, the computer device 500 also includes: a touch screen display 503, a radio frequency circuit 504, an audio circuit 505, an input unit 506, and a power supply 507. The processor 501 is electrically connected to the touch screen display 503, the radio frequency circuit 504, the audio circuit 505, the input unit 506, and the power supply 507. Those skilled in the art will understand that... Figure 4 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0223] The touch display screen 503 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 503 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the computer device. These graphical user interfaces can be composed of graphics, guidance information, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar devices. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 501. It can also receive and execute commands from the processor 501. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 501 to determine the type of touch event. Subsequently, the processor 501 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and display panel can be integrated into the touch display screen 503 to achieve input and output functions. However, in some embodiments, the touch panel and display panel can be implemented as two independent components to achieve input and output functions. That is, the touch display screen 503 can also be used as part of the input unit 506 to achieve input functions.

[0224] The radio frequency circuit 504 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other computer devices, and to transmit and receive signals with network devices or other computer devices.

[0225] Audio circuitry 505 can be used to provide an audio interface between a user and a computer device via a speaker and a microphone. Audio circuitry 505 converts received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 505, converted back into audio data, and then processed by processor 501 before being transmitted via radio frequency circuitry 504 to, for example, another computer device, or output to memory 502 for further processing. Audio circuitry 505 may also include an earphone jack to facilitate communication between peripheral headphones and computer devices.

[0226] The input unit 506 can be used to receive input numbers, characters, or user characteristics (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0227] Power supply 507 is used to supply power to various components of computer device 500. Optionally, power supply 507 can be logically connected to processor 501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 507 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0228] although Figure 4 As not shown in the diagram, the computer device 500 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0229] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0230] As can be seen from the above, the computer device provided in this embodiment can obtain user information and media information of the target platform; perform vectorization processing on the user information to obtain a first vector corresponding to the user information; perform vectorization processing on the media information to obtain a second vector corresponding to the media information; obtain the multimodal vector corresponding to the multimodal information in the media information, and calculate the loss parameters based on the first vector, the second vector and the multimodal vector; adjust the parameters of the preset recall model based on the loss parameters to obtain the target recall model.

[0231] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0232] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of computer programs that can be loaded by a processor to execute steps in any of the recall model training methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0233] Obtain user and media information from the target platform;

[0234] The user information is vectorized to obtain the first vector corresponding to the user information;

[0235] The media information is vectorized to obtain the second vector corresponding to the media information;

[0236] Obtain the multimodal vector corresponding to the multimodal information in the media information, and calculate the loss parameters of the preset recall model based on the first vector, the second vector and the multimodal vector;

[0237] The target recall model is obtained by adjusting the parameters of the preset recall model based on the loss parameters.

[0238] In some embodiments, the user information is vectorized to obtain a first vector corresponding to the user information, including:

[0239] Feature extraction is performed on user information to obtain user feature vectors;

[0240] The user feature vector is reduced in dimensionality to obtain the first vector.

[0241] In some embodiments, the media information is vectorized to obtain a second vector corresponding to the media information, including:

[0242] Feature extraction is performed on media information to obtain media feature vectors;

[0243] The media feature vector is reduced in dimensionality to obtain the second vector.

[0244] In some embodiments, obtaining the multimodal vector corresponding to the multimodal information in the media information includes:

[0245] To acquire text modal information and image modal information from media information;

[0246] Features of text modal information are extracted to obtain text feature vectors, and features of image modal information are extracted to obtain image feature vectors;

[0247] The text feature vector and the image feature vector are dimensionality reduced respectively to obtain the first modality vector corresponding to the text modality information and the second modality vector corresponding to the image modality information.

[0248] In some embodiments, calculating the loss parameters of a preset recall model based on the first vector, the second vector, and the multimodal vector includes:

[0249] Calculate the first loss function based on the first vector and the second vector;

[0250] The second loss function is calculated based on the second vector, the first mode vector, and the second mode vector;

[0251] The third loss function is calculated based on the first and second mode vectors;

[0252] The loss parameters are calculated based on the first loss function, the second loss function, and the third loss function.

[0253] In some embodiments, calculating a first loss function based on a first vector and a second vector includes:

[0254] Perform a linear transformation on the first vector to obtain the predicted vector and the random deactivation vector corresponding to the first vector;

[0255] Perform a linear transformation on the second vector to obtain the predicted vector and the random deactivation vector corresponding to the second vector;

[0256] The first loss function is obtained by fitting the random deactivation vector corresponding to the second vector to the predicted vector corresponding to the first vector, and by fitting the random deactivation vector corresponding to the first vector to the predicted vector corresponding to the second vector.

[0257] In some embodiments, calculating a second loss function based on a second vector, a first mode vector, and a second mode vector includes:

[0258] A linear transformation is performed on the first mode vector to obtain the predicted vector and the random deactivation vector corresponding to the first mode vector;

[0259] A linear transformation is performed on the second mode vector to obtain the predicted vector and random deactivation vector corresponding to the second mode vector;

[0260] The second loss function is obtained by fitting the random deactivation vector corresponding to the first mode vector and the random deactivation vector corresponding to the second mode vector with the predicted vector corresponding to the second vector.

[0261] In some embodiments, calculating a third loss function based on a first mode vector and a second mode vector includes:

[0262] The third loss function is obtained by fitting the random deactivation vector corresponding to the second mode vector to the predicted vector corresponding to the first mode vector, and by fitting the random deactivation vector corresponding to the first mode vector to the predicted vector corresponding to the second mode vector.

[0263] In some embodiments, it also includes:

[0264] Obtain target user information and extract features from it to obtain target user features;

[0265] The target user characteristics are input into the target recall model, and the target media information that matches the target user characteristics is recalled from the media information of the target platform based on the target recall model.

[0266] This application embodiment obtains user information and user-posted note information from the target platform, then extracts user feature vectors from the user information and note feature vectors from the note information; obtains multimodal vectors corresponding to multimodal information in the note information, and calculates loss parameters based on the user feature vectors, note feature vectors, and multimodal vectors. Then, based on the loss parameters, the parameters of a preset recall model are adjusted to obtain the target recall model. This improves the content recall accuracy of the recall model.

[0267] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0268] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0269] Since the computer program stored in the storage medium can execute the steps in any of the recall model training methods provided in the embodiments of this application, the beneficial effects that any of the recall model training methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0270] According to one aspect of this application, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0271] The training method, apparatus, storage medium, and computer device for a recall model provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A training method for a recall model, characterized in that, The method includes: Obtain user and media information from the target platform; The user information is vectorized to obtain a first vector corresponding to the user information; The media information is vectorized to obtain a second vector corresponding to the media information; Obtain the multimodal vector corresponding to the multimodal information in the media information, wherein the multimodal information includes text modal information and image modal information in the media information, and the multimodal vector includes a first modal vector corresponding to the text modal information and a second modal vector corresponding to the image modal information; Calculate the first loss function based on the first vector and the second vector; Calculate the second loss function based on the second vector, the first mode vector, and the second mode vector; Calculate the third loss function based on the first mode vector and the second mode vector; The loss parameters of the preset recall model are calculated based on the first loss function, the second loss function, and the third loss function. The parameters of the preset recall model are adjusted based on the loss parameters to obtain the target recall model.

2. The method according to claim 1, characterized in that, The step of vectorizing the user information to obtain the first vector corresponding to the user information includes: Feature extraction is performed on the user information to obtain a user feature vector; The user feature vector is reduced in dimensionality to obtain the first vector.

3. The method according to claim 1, characterized in that, The step of vectorizing the media information to obtain the second vector corresponding to the media information includes: Feature extraction is performed on the media information to obtain a media feature vector; The media feature vector is reduced in dimensionality to obtain the second vector.

4. The method according to claim 1, characterized in that, The step of obtaining the multimodal vector corresponding to the multimodal information in the media information includes: Obtain the text modal information and image modal information from the media information; The features of the text modal information are extracted to obtain a text feature vector, and the features of the image modal information are extracted to obtain an image feature vector; The text feature vector and the image feature vector are respectively subjected to dimensionality reduction processing to obtain the first modality vector corresponding to the text modality information and the second modality vector corresponding to the image modality information.

5. The method according to claim 1, characterized in that, The step of calculating the first loss function based on the first vector and the second vector includes: Perform a linear transformation on the first vector to obtain the predicted vector and the random deactivation vector corresponding to the first vector; Perform a linear transformation on the second vector to obtain the predicted vector and the random deactivation vector corresponding to the second vector; The first loss function is obtained by fitting the random deactivation vector corresponding to the second vector to the predicted vector corresponding to the first vector, and by fitting the random deactivation vector corresponding to the first vector to the predicted vector corresponding to the second vector.

6. The method according to claim 5, characterized in that, The step of calculating the second loss function based on the second vector, the first mode vector, and the second mode vector includes: A linear transformation is performed on the first mode vector to obtain the predicted vector and random deactivation vector corresponding to the first mode vector; A linear transformation is performed on the second mode vector to obtain the predicted vector and random deactivation vector corresponding to the second mode vector; The second loss function is obtained by fitting the random deactivation vector corresponding to the first mode vector and the random deactivation vector corresponding to the second mode vector with the predicted vector corresponding to the second vector.

7. The method according to claim 6, characterized in that, The step of calculating the third loss function based on the first mode vector and the second mode vector includes: The third loss function is obtained by fitting the random deactivation vector corresponding to the second mode vector to the predicted vector corresponding to the first mode vector, and by fitting the random deactivation vector corresponding to the first mode vector to the predicted vector corresponding to the second mode vector.

8. The method according to claim 1, characterized in that, Also includes: Obtain target user information and extract features from the target user information to obtain target user features; The target user characteristics are input into the target recall model, and the target media information that matches the target user characteristics is recalled from the media information of the target platform based on the target recall model.

9. A training device for a recall model, characterized in that, The device includes: The first acquisition unit is used to acquire user information and media information of the target platform; The first processing unit is used to perform vectorization processing on the user information to obtain a first vector corresponding to the user information; The second processing unit is used to perform vectorization processing on the media information to obtain a second vector corresponding to the media information; A calculation unit is configured to acquire multimodal vectors corresponding to multimodal information in the media information, wherein the multimodal information includes text modal information and image modal information in the media information, and the multimodal vectors include a first modal vector corresponding to the text modal information and a second modal vector corresponding to the image modal information; calculate a first loss function based on the first vector and the second vector; calculate a second loss function based on the second vector, the first modal vector, and the second modal vector; calculate a third loss function based on the first modal vector and the second modal vector; and calculate loss parameters of a preset recall model based on the first loss function, the second loss function, and the third loss function. The adjustment unit is used to adjust the parameters of the preset recall model based on the loss parameters to obtain the target recall model.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein, When the processor executes the program, it implements the training method for the recall model as described in any one of claims 1 to 8.

11. A storage medium, characterized in that, The storage medium stores a plurality of instructions adapted for loading by a processor to execute the training method of the recall model according to any one of claims 1 to 8.

Images