Resource recommendations, model training methods, devices, equipment, storage media and programs

Abstract

The present disclosure relates to a resource recommendation, model training method, apparatus, device, storage medium and program, the method comprising: obtaining content features of sample multimedia resources; determining labels corresponding to multiple sample accounts according to operation behavior information of multiple sample accounts on the sample multimedia resources; determining at least one target account among the multiple sample accounts, and determining an operation behavior sequence according to the operation behavior information of the at least one target account on the sample multimedia resources; determining a multimedia resource sequence according to multiple target multimedia resources respectively operated by each target account; inputting account features corresponding to each of the multiple sample accounts, content features of the sample multimedia resources, operation behavior sequences and multimedia resource sequences into a preset neural network model for prediction processing to obtain prediction results corresponding to each sample account; and training the preset neural network model according to the difference between the labels corresponding to each sample account and the prediction results.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technology, and in particular to a resource recommendation, model training method, device, equipment, storage medium and program. Background Art

[0002] With the development of science and technology, people's entertainment life is becoming more and more abundant. One of the entertainment life is to browse the videos recommended by the entertainment platform. Generally speaking, the entertainment platform will select some videos that the current user may be interested in for recommendation based on the video recommendation model. In order to enable the video recommendation model to normally perceive which videos the current user may be interested in, the video recommendation model needs to be trained, and a large number of training samples are required in the training process. These training samples include whether users who have watched a sample video have liked, forwarded, commented on, shared, and other behaviors on the sample video. For old videos that have been released for a long time, many users have watched the old videos, so the behaviors of these users after watching the old videos can be obtained, and the video recommendation model can be better trained based on these behaviors. For new videos that have just been released, not enough users have watched the new videos yet, so it is difficult to obtain the behaviors generated by enough different users after watching the new videos, and thus it is impossible to improve the performance of the video recommendation model for recommending new videos, which ultimately makes it difficult for new videos to be accurately recommended. Summary of the invention

[0003] The present disclosure provides a resource recommendation, model training method, device, equipment, storage medium and program to at least solve the problem in the related art that the performance of the video recommendation model for recommending new videos cannot be improved, resulting in the difficulty of accurately recommending new videos. The technical solution of the present disclosure is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, a multimedia resource recommendation model training method is provided, comprising:

[0005] Acquire content features of sample multimedia resources, where the sample multimedia resources refer to multimedia resources whose duration between release time and current time is less than a preset threshold;

[0006] Determining operation behavior information of multiple sample accounts that have operated the sample multimedia resource on the sample multimedia resource;

[0007] Determine, according to the operation behavior information of the multiple sample accounts on the sample multimedia resource, labels corresponding to the multiple sample accounts respectively, wherein the labels are used to indicate whether the corresponding sample accounts generate target operation behaviors on the sample multimedia resource;

[0008] Determine at least one target account from the multiple sample accounts, and determine an operation behavior sequence according to the operation behavior information of the at least one target account on the sample multimedia resource, wherein the target account refers to an account whose operation behavior on the sample multimedia resource meets a first preset condition;

[0009] Determine a multimedia resource sequence according to a plurality of target multimedia resources respectively operated by each target account, wherein the target multimedia resource refers to a multimedia resource that satisfies a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period;

[0010] Inputting the account features corresponding to each of the multiple sample accounts, the content features of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence into a preset neural network model for prediction processing, and obtaining a prediction result corresponding to each of the sample accounts, wherein the prediction result is used to indicate the probability that the corresponding sample account will generate the target operation behavior on the sample multimedia resource;

[0011] According to the difference between the labels corresponding to each of the sample accounts and the prediction results, the preset neural network model is trained until a preset training end condition is met to obtain a multimedia resource recommendation model.

[0012] Optionally, obtaining content features of sample multimedia resources includes:

[0013] Extracting features of an intermediate network layer output in a preset multimedia resource processing model, wherein the multimedia resource processing model takes the sample multimedia resource as input;

[0014] The extracted features are determined as content features of the sample multimedia resources.

[0015] Optionally, the inputting the account features corresponding to each of the plurality of sample accounts, the content features of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence into a preset neural network model for prediction processing to obtain the prediction results corresponding to each of the sample accounts includes:

[0016] Aggregating the operation behavior sequence by presetting a plurality of first network layers in a neural network model to obtain operation behavior features;

[0017] Aggregating the multimedia resource sequence through the plurality of second network layers in the preset neural network model to obtain multimedia resource features;

[0018] The account characteristics corresponding to each of the multiple sample accounts, the content characteristics of the sample multimedia resources, the operation behavior characteristics and the multimedia resource characteristics are input into multiple third network layers in the preset neural network model to obtain prediction results corresponding to each of the sample accounts.

[0019] Optionally, the multiple first network layers adopt a multi-head attention mechanism to aggregate the operation behavior sequence;

[0020] The multiple second network layers adopt a multi-head attention mechanism to perform aggregation processing on the multimedia resource sequence.

[0021] Optionally, the operation behavior sequence is aggregated by using a plurality of first network layers in a preset neural network model to obtain the operation behavior features, including:

[0022] Determine the importance of each target account by presetting multiple first network layers in the neural network model;

[0023] The importance of each target account is used as a weight, and the operation behavior information of each target account in the operation behavior sequence is weighted and summed to obtain the operation behavior feature.

[0024] Optionally, determining the importance of each target account by presetting a plurality of first network layers in a neural network model includes:

[0025] Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource;

[0026] The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources and the operation behavior sequence are input into multiple first network layers in a preset neural network model to obtain the importance of each target account.

[0027] Optionally, the aggregating the multimedia resource sequence through the plurality of second network layers in the preset neural network model to obtain multimedia resource features includes:

[0028] Determining the importance of each target multimedia resource through a plurality of second network layers in the preset neural network model;

[0029] The importance of each target multimedia resource is used as a weight, and weighted sum is performed on each target multimedia resource in the multimedia resource sequence to obtain a multimedia resource feature.

[0030] Optionally, determining the importance of each target multimedia resource through a plurality of second network layers in the preset neural network model includes:

[0031] Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource;

[0032] The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence are input into multiple second network layers in the preset neural network model to obtain the importance of each target multimedia resource.

[0033] Optionally, after obtaining the multimedia resource recommendation model, the method further includes:

[0034] The model parameters in the multimedia resource recommendation model are sent to the model application device, so that the model application device updates the online multimedia resource recommendation model set in the model application device based on the received model parameters.

[0035] According to a second aspect of an embodiment of the present disclosure, a multimedia resource recommendation method is provided, including:

[0036] Acquire content features of a multimedia resource to be recommended, wherein the multimedia resource to be recommended refers to a multimedia resource whose duration between a release time and a current time is less than a preset threshold;

[0037] Determine at least one target account from multiple reference accounts that have operated the multimedia resource to be recommended, and determine an operation behavior sequence according to the operation behavior information of the at least one target account on the multimedia resource to be recommended, wherein the target account refers to an account whose operation behavior on the multimedia resource to be recommended meets a first preset condition;

[0038] Acquiring multimedia resource features, wherein the multimedia resource features are determined based on a plurality of target multimedia resources respectively operated by each target account, and the target multimedia resources refer to multimedia resources that meet a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period;

[0039] Inputting the account characteristics corresponding to the account to be recommended, the content characteristics of the multimedia resource to be recommended, the operation behavior sequence and the multimedia resource characteristics into the multimedia resource recommendation model for prediction processing, and obtaining the prediction result corresponding to the account to be recommended, wherein the prediction result is used to indicate the probability that the account to be recommended will generate the target operation behavior on the multimedia resource to be recommended;

[0040] The multimedia resource to be recommended is recommended based on the prediction result corresponding to the account to be recommended.

[0041] Optionally, the target operation behavior includes multiple operation behaviors, and the recommending the multimedia resource to be recommended based on the prediction result corresponding to the account to be recommended includes:

[0042] Performing weighted summation on the probabilities of generating various operation behaviors to obtain a recommendation index of the multimedia resource to be recommended;

[0043] The multimedia resource to be recommended is recommended based on the recommendation index of the multimedia resource to be recommended.

[0044] Optionally, the recommending the multimedia resource to be recommended based on the recommendation index of the multimedia resource to be recommended includes:

[0045] Acquire recommendation indicators corresponding to each of a plurality of candidate multimedia resources, wherein the candidate multimedia resources include the multimedia resource to be recommended;

[0046] The plurality of candidate multimedia resources are sorted in descending order according to the recommendation index, and a multimedia resource ranked first by a preset number of places is selected for recommendation.

[0047] Optionally, the method further comprises:

[0048] Receiving model parameters sent by the model training device according to a preset period;

[0049] The locally set multimedia resource recommendation model is updated based on the received model parameters.

[0050] Optionally, the acquiring multimedia resource features includes:

[0051] Determining a multimedia resource sequence based on the multiple target multimedia resources respectively operated by the target accounts;

[0052] Aggregation processing is performed on the multimedia resource sequence to obtain multimedia resource features.

[0053] Optionally, the aggregating the multimedia resource sequence to obtain multimedia resource features includes:

[0054] A multi-head attention mechanism is adopted to aggregate the multimedia resource sequence to obtain multimedia resource features.

[0055] Optionally, the multi-head attention mechanism is used to aggregate the multimedia resource sequence to obtain multimedia resource features, including:

[0056] Determining the importance of each target multimedia resource in the multimedia resource sequence;

[0057] The importance of each target multimedia resource is used as a weight, and weighted sum is performed on the target multimedia resources to obtain multimedia resource features.

[0058] Optionally, determining the importance of each target multimedia resource in the multimedia resource sequence includes:

[0059] Acquire attribute information of the multimedia resource to be recommended and playback environment information when each target account plays the multimedia resource to be recommended;

[0060] The importance of each target multimedia resource is determined based on the account characteristics corresponding to the account to be recommended, the attribute information, the playback environment information, the content characteristics of the multimedia resource to be recommended, the operation behavior sequence and the multimedia resource sequence.

[0061] According to a third aspect of an embodiment of the present disclosure, a multimedia resource recommendation model training device is provided, characterized in that it includes:

[0062] An acquisition unit is configured to acquire content features of sample multimedia resources, where the sample multimedia resources refer to multimedia resources whose duration between release time and current time is less than a preset threshold;

[0063] A determining unit, configured to determine operation behavior information of a plurality of sample accounts that have operated the sample multimedia resource on the sample multimedia resource;

[0064] The determining unit is configured to determine, according to the operation behavior information of the multiple sample accounts on the sample multimedia resources, a label corresponding to each of the multiple sample accounts, wherein the label is used to indicate whether the corresponding sample account generates a target operation behavior on the sample multimedia resources;

[0065] The determining unit is configured to determine at least one target account from the multiple sample accounts, and determine an operation behavior sequence according to the operation behavior information of the at least one target account on the sample multimedia resource, wherein the target account refers to an account whose operation behavior on the sample multimedia resource meets a first preset condition;

[0066] The determining unit is configured to determine a multimedia resource sequence according to a plurality of target multimedia resources respectively operated by each target account, wherein the target multimedia resource refers to a multimedia resource that meets a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period;

[0067] A prediction unit is configured to perform prediction processing by inputting account features corresponding to each of the multiple sample accounts, content features of the sample multimedia resources, the operation behavior sequence, and the multimedia resource sequence into a preset neural network model to obtain a prediction result corresponding to each of the sample accounts, wherein the prediction result is used to indicate the probability that the corresponding sample account generates the target operation behavior on the sample multimedia resource;

[0068] The training unit is configured to execute training of the preset neural network model according to the difference between the labels corresponding to each of the sample accounts and the prediction results until a preset training end condition is met to obtain a multimedia resource recommendation model.

[0069] Optionally, the acquisition unit is configured to execute:

[0070] Extracting features of an intermediate network layer output in a preset multimedia resource processing model, wherein the multimedia resource processing model takes the sample multimedia resource as input;

[0071] The extracted features are determined as content features of the sample multimedia resources.

[0072] Optionally, the prediction unit is configured to execute:

[0073] Aggregating the operation behavior sequence by presetting a plurality of first network layers in a neural network model to obtain operation behavior features;

[0074] Aggregating the multimedia resource sequence through the plurality of second network layers in the preset neural network model to obtain multimedia resource features;

[0075] The account characteristics corresponding to each of the multiple sample accounts, the content characteristics of the sample multimedia resources, the operation behavior characteristics and the multimedia resource characteristics are input into multiple third network layers in the preset neural network model to obtain prediction results corresponding to each of the sample accounts.

[0076] Optionally, the multiple first network layers adopt a multi-head attention mechanism to aggregate the operation behavior sequence;

[0077] The multiple second network layers adopt a multi-head attention mechanism to perform aggregation processing on the multimedia resource sequence.

[0078] Optionally, the prediction unit is configured to execute:

[0079] Determine the importance of each target account by presetting multiple first network layers in the neural network model;

[0080] The importance of each target account is used as a weight, and the operation behavior information of each target account in the operation behavior sequence is weighted and summed to obtain the operation behavior feature.

[0081] Optionally, the prediction unit is configured to execute:

[0082] Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource;

[0083] The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources and the operation behavior sequence are input into multiple first network layers in a preset neural network model to obtain the importance of each target account.

[0084] Optionally, the prediction unit is configured to execute:

[0085] Determining the importance of each target multimedia resource through a plurality of second network layers in the preset neural network model;

[0086] The importance of each target multimedia resource is used as a weight, and weighted sum is performed on each target multimedia resource in the multimedia resource sequence to obtain a multimedia resource feature.

[0087] Optionally, the prediction unit is configured to execute:

[0088] Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource;

[0089] The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence are input into multiple second network layers in the preset neural network model to obtain the importance of each target multimedia resource.

[0090] Optionally, the device further comprises:

[0091] The updating unit is configured to send the model parameters in the multimedia resource recommendation model to the model application device, so that the model application device updates the online multimedia resource recommendation model set in the model application device based on the received model parameters.

[0092] According to a fourth aspect of an embodiment of the present disclosure, a multimedia resource recommendation device is provided, including:

[0093] An acquisition unit is configured to acquire content features of a multimedia resource to be recommended, wherein the multimedia resource to be recommended refers to a multimedia resource whose duration between a release time and a current time is less than a preset threshold;

[0094] a determining unit configured to determine at least one target account from a plurality of reference accounts that have operated the multimedia resource to be recommended, and determine an operation behavior sequence according to operation behavior information of the at least one target account on the multimedia resource to be recommended, wherein the target account refers to an account whose operation behavior on the multimedia resource to be recommended meets a first preset condition;

[0095] The acquisition unit is configured to execute acquisition of multimedia resource features, wherein the multimedia resource features are determined based on a plurality of target multimedia resources respectively operated by each target account, and the target multimedia resources refer to multimedia resources that meet a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period;

[0096] A prediction unit is configured to perform prediction processing by inputting account features corresponding to the account to be recommended, content features of the multimedia resource to be recommended, the operation behavior sequence, and the multimedia resource features into a multimedia resource recommendation model to obtain a prediction result corresponding to the account to be recommended, wherein the prediction result is used to indicate the probability that the account to be recommended will generate a target operation behavior on the multimedia resource to be recommended;

[0097] The recommendation unit is configured to perform a recommendation on the multimedia resource to be recommended based on the prediction result corresponding to the account to be recommended.

[0098] Optionally, the target operation behavior includes multiple operation behaviors, and the recommendation unit is configured to execute:

[0099] Performing weighted summation on the probabilities of generating various operation behaviors to obtain a recommendation index of the multimedia resource to be recommended;

[0100] The multimedia resource to be recommended is recommended based on the recommendation index of the multimedia resource to be recommended.

[0101] Optionally, the recommendation unit is configured to execute:

[0102] Acquire recommendation indicators corresponding to each of a plurality of candidate multimedia resources, wherein the candidate multimedia resources include the multimedia resource to be recommended;

[0103] The plurality of candidate multimedia resources are sorted in descending order according to the recommendation index, and the multimedia resource ranked first by a preset number of places is selected for recommendation.

[0104] Optionally, the apparatus further includes an updating unit, wherein the updating unit is configured to execute:

[0105] Receiving model parameters sent by the model training device according to a preset period;

[0106] The locally set multimedia resource recommendation model is updated based on the received model parameters.

[0107] Optionally, the acquiring unit is configured to execute:

[0108] Determining a multimedia resource sequence based on the multiple target multimedia resources respectively operated by the target accounts;

[0109] Aggregation processing is performed on the multimedia resource sequence to obtain multimedia resource features.

[0110] Optionally, the acquiring unit is configured to execute:

[0111] A multi-head attention mechanism is adopted to aggregate the multimedia resource sequence to obtain multimedia resource features.

[0112] Optionally, the acquiring unit is configured to execute:

[0113] Determining the importance of each target multimedia resource in the multimedia resource sequence;

[0114] The importance of each target multimedia resource is used as a weight, and weighted sum is performed on the target multimedia resources to obtain multimedia resource features.

[0115] Optionally, the acquiring unit is configured to execute:

[0116] Acquire attribute information of the multimedia resource to be recommended and playback environment information when each target account plays the multimedia resource to be recommended;

[0117] The importance of each target multimedia resource is determined based on the account characteristics corresponding to the account to be recommended, the attribute information, the playback environment information, the content characteristics of the multimedia resource to be recommended, the operation behavior sequence and the multimedia resource sequence.

[0118] According to a fifth aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0119] processor;

[0120] a memory for storing instructions executable by the processor;

[0121] Among them, the processor is configured to execute the instructions to implement the multimedia resource recommendation model training method provided in the first aspect of the present disclosure or the multimedia resource recommendation method provided in the second aspect of the present disclosure.

[0122] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the multimedia resource recommendation model training method provided in the first aspect of the present disclosure or the multimedia resource recommendation method provided in the second aspect of the present disclosure.

[0123] According to the seventh aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the multimedia resource recommendation model training method provided in the first aspect of the present disclosure or the multimedia resource recommendation method provided in the second aspect of the present disclosure.

[0124] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0125] By adopting the present disclosure, for newly released sample multimedia resources, even if the sample multimedia resources lack a large number of behaviors generated by a large number of accounts as training samples, it is still possible to collect content features, operation behavior sequences, and multimedia resource sequences obtained by understanding multimodal information such as image information and audio information in the sample multimedia resources. Among them, the operation behavior sequence can indicate what kind of account is more interested in such sample multimedia resources, so the operation behavior sequence can describe the sample multimedia resources from the side. Based on account features, content features of sample multimedia resources, operation behavior features, and multimedia resource features, etc., the preset neural network model is trained, so that the multimedia resource recommendation model obtained after training can accurately recommend newly released multimedia resources.

[0126] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0127] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0128] Figure 1 is a flowchart of a multimedia resource recommendation model training method according to an exemplary embodiment;

[0129] Figure 2 is a structural schematic diagram of a multimedia resource recommendation model according to an exemplary embodiment;

[0130] Figure 3 is a schematic diagram showing a principle of multiple first network layers according to an exemplary embodiment;

[0131] Figure 4 is a schematic diagram showing a principle of multiple second network layers according to an exemplary embodiment;

[0132] Figure 5 is a flowchart of a multimedia resource recommendation method according to an exemplary embodiment;

[0133] Figure 6 is a block diagram of a multimedia resource recommendation model training device according to an exemplary embodiment;

[0134] Figure 7 is a block diagram of a multimedia resource recommendation device according to an exemplary embodiment;

[0135] Figure 8 According to an exemplary embodiment, a method for Figure 6 The device shown is a block diagram of an electronic device;

[0136] Fig. 9 According to an exemplary embodiment, a method for Figure 7 The apparatus shown is a block diagram of an electronic device. DETAILED DESCRIPTION

[0137] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0138] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0139] The disclosed embodiment provides a multimedia resource recommendation model training method, which can be performed by an electronic device, which can be a terminal device such as a PC (Personal Computer), a laptop, a smart phone, or a server. The server can be a physical server including an independent host, or a virtual server, or a cloud server or a server cluster.

[0140] Figure 1 is a flowchart of a multimedia resource recommendation model training method according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps.

[0141] In step S11, content features of sample multimedia resources are obtained. Sample multimedia resources refer to multimedia resources whose duration between release time and current time is less than a preset threshold.

[0142] In step S12, operation behavior information of a plurality of sample accounts that have operated the sample multimedia resources on the sample multimedia resources is determined.

[0143] In step S13, based on the operation behavior information of the multiple sample accounts on the sample multimedia resources, the labels corresponding to the multiple sample accounts are determined, and the labels are used to indicate whether the corresponding sample accounts have generated the target operation behavior on the sample multimedia resources.

[0144] In step S14, at least one target account is determined from multiple sample accounts, and an operation behavior sequence is determined based on the operation behavior information of the at least one target account on the sample multimedia resources. The target account refers to an account whose operation behavior on the sample multimedia resources meets the first preset condition.

[0145] In step S15, a multimedia resource sequence is determined according to a plurality of target multimedia resources respectively operated by each target account, wherein the target multimedia resource refers to a multimedia resource that meets a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period.

[0146] In step S16, the account characteristics corresponding to each of the multiple sample accounts, the content characteristics of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence are input into the preset neural network model for prediction processing to obtain the prediction results corresponding to each sample account. The prediction results are used to indicate the probability of the corresponding sample account generating the target operation behavior on the sample multimedia resources.

[0147] In step S17, according to the difference between the label and the prediction result corresponding to each sample account, the preset neural network model is trained until the preset training end condition is met to obtain a multimedia resource recommendation model.

[0148] The sample multimedia resources may be resources such as videos, pictures, and audio. When the sample multimedia resources are videos or pictures, having operated the sample multimedia resources may be having watched the sample videos or sample pictures. When the sample multimedia resources are audio, having operated the sample multimedia resources may be having listened to the sample audio. In the embodiments of the present disclosure, the sample multimedia resources may be newly released multimedia resources.

[0149] It should be noted that the preset neural network model can be trained to obtain a multimedia resource recommendation model with the function of recommending multimedia resources. In the process of training the preset neural network model, training samples are needed to train the preset neural network model. The training samples used in the embodiments of the present disclosure include but are not limited to: account features corresponding to each sample account, content features of sample multimedia resources, operation behavior sequences, and multimedia resource sequences. The following will introduce the methods for obtaining each training sample one by one.

[0150] In actual applications, the account characteristics of the sample accounts that have operated the sample multimedia resources can be obtained. The account characteristics of the sample account may include the account identification information (userid), the identification information of the device used when viewing the sample multimedia resources, the interest information corresponding to the account, age information, gender information, geographic location information, etc. In addition, the account characteristics of the sample account may also include the average summed account behavior sequence. The account characteristics of the sample account may be any one or more of the above-listed information, or may be some other information that can characterize the account characteristics, which is not limited in the embodiments of the present disclosure.

[0151] The averaged account behavior sequence may be, for example, the average sum of the encoding information of a preset number of multimedia resources that the account has recently operated for a long time, and the encoding information may be the embedding value corresponding to the identification information (e.g., photoid) of the multimedia resource. Embedding refers to a table lookup technology that can map high-dimensional, sparse input information to low-dimensional, dense output vectors. The embedding value may be a 32-dimensional float vector.

[0152] To facilitate understanding of the process of obtaining the average summed account behavior sequence, for example, the identification information of a preset number of videos that the account has recently watched for a long time can be determined. Assume that the identification information of 30 videos is determined. Then, the embedding values ​​corresponding to the identification information of the 30 videos can be determined by the table lookup technology. Finally, all the determined embedding values ​​can be added and divided by 30 to obtain the average sum of the encoding information of the 30 videos that the user has recently watched for a long time.

[0153] Since the sample multimedia resources are newly released multimedia resources, there may be fewer sample accounts that have operated the sample multimedia resources after the sample multimedia resources are released, but the operation behavior information of these sample accounts on the sample multimedia resources can still be collected. Among them, the operation behavior information can include whether the corresponding sample account has performed operations such as liking, commenting, forwarding, and sharing on the sample multimedia resources. In addition, it can also include the identification information of the device used when operating the sample multimedia resources, the duration of operating the sample multimedia resources, and the time difference between the operation time and the current time.

[0154] The label corresponding to each sample account can be determined based on the operation behavior information of each sample account on the sample multimedia resources. Specifically, the label corresponding to each sample account can be determined based on the information in the operation behavior information that indicates whether the corresponding sample account has performed operation behaviors such as liking, commenting, forwarding, and sharing on the sample multimedia resources. For example, if a sample account has liked the sample multimedia resource, then for the label with the like behavior as the target operation behavior, the sample user is labeled as having liked.

[0155] At least one target account may be screened from the plurality of sample accounts, and the target account may also be referred to as a valid account. The target account refers to an account whose operation behavior on the sample multimedia resource satisfies the first preset condition. In some optional embodiments, when the sample multimedia resource is a sample video, the target account may refer to an account whose viewing time of the sample video reaches a preset threshold within a preset historical time period.

[0156] The preset historical time period may be, for example, the last five days. The preset threshold may be, for example, half the length of the sample video. Then the target account may be, for example, an account whose sample video viewing time in the last five days exceeds half the length of the sample video.

[0157] It is understandable that some accounts will slide away the sample video before the number of sample videos watched reaches the preset threshold. These accounts have little reference value, so such accounts can be filtered out from the sample accounts.

[0158] After determining at least one target account, an operation behavior sequence can be determined based on the operation behavior information of the at least one target account on the sample multimedia resource. Specifically, the operation behavior information of each target account on the sample multimedia resource can be obtained, and then the operation behavior information can be combined to form an operation behavior sequence.

[0159] For example, obtain the identification information of the device used to operate the sample multimedia resources corresponding to each target account, the duration of the operation of the sample multimedia resources, and the time difference between the operation time and the current time, combine the above information corresponding to all target accounts together, and finally obtain the operation behavior sequence.

[0160] Since the above-mentioned target accounts generally operate not only sample multimedia resources, but also other multimedia resources other than sample multimedia resources. Based on this, multiple other multimedia resources operated by each target account can be collected. Specifically, multiple other multimedia resources operated by each target account can be found based on the identification information of the device used when operating the sample multimedia resources in the operation behavior information. Then, the target multimedia resource is determined from the multiple other multimedia resources operated by each target account.

[0161] The target multimedia resource refers to a multimedia resource that satisfies the second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period. The preset historical time period may be, for example, the last 5 days. The second preset condition may be, for example, that the duration of operating the multimedia resource reaches a preset threshold. When the multimedia resource is a video, the preset threshold may be, for example, half of the length of the corresponding video.

[0162] If it is necessary to limit the number of target multimedia resources that have been operated by each target account, in actual application, multiple target multimedia resources corresponding to the target account can be sorted in order from recent to earliest according to the operation time, and the target multimedia resources arranged in the front preset position can be selected, and the target multimedia resources arranged in the back can be eliminated.

[0163] Next, a multimedia resource sequence can be determined based on the multiple target multimedia resources that have been operated by each target account. Specifically, the identification information of each target multimedia resource that has been operated by all target accounts, the duration of operating the sample multimedia resource, the time difference between the operating time and the current time, etc. can be combined together to form a multimedia resource sequence.

[0164] For example, the target videos recently watched by the target account include beauty video A, cooking video B, and travel video C. The video IDs, viewing time, and the time difference between the viewing time and the current time corresponding to beauty video A, cooking video B, and travel video C can be combined together to form a multimedia resource sequence.

[0165] Next, we will introduce the method of obtaining content features of sample multimedia resources.

[0166] The content features of sample multimedia resources refer to the features obtained after understanding the multimodal information such as image information and audio information in the sample multimedia resources. The process of obtaining the content features of sample multimedia resources can be implemented as follows: extracting the features output by the intermediate network layer in the preset multimedia resource processing model, the multimedia resource processing model takes the sample multimedia resources as input; and determining the extracted features as the content features of the sample multimedia resources.

[0167] The multimedia resource processing model is a model other than the multimedia resource recommendation model, and is a model with other functions other than multimedia resource recommendation. The multimedia resource processing model may include a model obtained by training through a supervised method or an unsupervised method. The model obtained by training through a supervised method may be, for example, a multi-label or multi-classification model for multimedia resource classification, operational label classification, or topic label (hashtag) classification, and the model obtained by training through an unsupervised method may be, for example, a multimedia resource clustering model.

[0168] The multimedia resource processing model takes the relevant information of the sample multimedia resources as input and can perform relevant processing on the sample multimedia resources, such as classifying the sample multimedia resources. Since the intermediate network layer of the multimedia resource processing model can extract some features of the sample multimedia resources, these features are the features obtained after understanding the multimodal information such as image information and audio information in the sample multimedia resources. Based on this, the features output by the intermediate network layer can be extracted from the multimedia resource processing model as the content features of the sample multimedia resources.

[0169] It should be noted that due to the large number of sample multimedia resources in actual training, the multimedia resource processing model can be periodically offline to process these sample multimedia resources, extract the content features corresponding to each sample multimedia resource from the intermediate network layer, and update the content features of these sample multimedia resources in batches to the electronic device used to train the multimedia resource recommendation model, so as to be used in the process of training the multimedia resource recommendation model.

[0170] Through the above-described method, the account features corresponding to each of the multiple sample accounts, the content features of the sample multimedia resources, the operation behavior sequence, and the multimedia resource sequence can be obtained. After obtaining these features, the preset neural network model can be trained with these features as training samples. Specifically, the account features corresponding to each of the multiple sample accounts, the content features of the sample multimedia resources, the operation behavior sequence, and the multimedia resource sequence can be input into the preset neural network model for prediction processing to obtain the prediction results corresponding to each sample account. Among them, the prediction results are used to indicate the probability that the corresponding sample account will generate a target operation behavior on the sample multimedia resource.

[0171] Finally, according to the difference between the labels and prediction results corresponding to each sample account, the preset neural network model can be trained until the preset training end condition is met to obtain a multimedia resource recommendation model.

[0172] For example, the label corresponding to sample account A is "like" for sample video V, and the prediction result predicted by the preset neural network model indicates that the probability of sample account A liking sample video V is 80%. Based on this, the difference between the label corresponding to sample account A and the prediction result can be determined.

[0173] The process of training the preset neural network model according to the difference can be implemented as follows: adjusting the model parameters of the preset neural network model through the loss function of the preset neural network model. Wherein, the loss function in some optional embodiments can be a BCE loss function, which refers to a binary cross entropy loss function.

[0174] The preset neural network model can be trained by the above training method until the preset training end condition is met. For example, when the confidence of the prediction result output by the preset neural network model is higher than the preset threshold, a multimedia resource recommendation model can be obtained, indicating that the multimedia resource recommendation model can be used for online prediction.

[0175] It should be noted that after the multimedia resource recommendation model is used for online prediction, the multimedia resource recommendation model can be continuously trained offline to continuously improve the confidence of the multimedia resource recommendation model so that its online performance is continuously improved.

[0176] Based on this, the embodiment of the present disclosure also provides an incremental update solution for the model. Optionally, after obtaining the multimedia resource recommendation model, the method provided by the embodiment of the present disclosure may also include: sending the model parameters in the multimedia resource recommendation model to the model application device, so that the model application device updates the online multimedia resource recommendation model set in the model application device based on the received model parameters.

[0177] It is understandable that the training process of the multimedia resource recommendation model can be implemented in the model training device (also called the training end), and the recommendation task of multimedia resources through the multimedia resource recommendation model can be implemented in the model application device (also called the inference end). In order to continuously improve the performance of the model, the multimedia resource recommendation model can be continuously optimized and trained in the model training device while the multimedia resource recommendation model is used online on the model application device to recommend multimedia resources.

[0178] In practical applications, the model parameters in the optimized multimedia resource recommendation model can be updated from the model training device to the model application device according to a preset period, so that the online estimation capability of the multimedia resource recommendation model can be further improved.

[0179] The following is a detailed introduction to the model training process.

[0180] The preset neural network model may include multiple network layers, which may be divided into multiple first network layers, multiple second network layers, and multiple third network layers according to their functions. The multiple first network layers and the multiple second network layers may be the same network layer or may be independent network layers. Figure 2 As shown, multiple first network layers can be used to process the operation behavior sequence to extract the operation behavior features. Multiple second network layers can be used to process the multimedia resource sequence to extract the multimedia resource features. Multiple third network layers can use the features output by the multiple first network layers and the multiple second network layers, the account features corresponding to each of the multiple sample accounts, the content features of the sample multimedia resources, etc. as input to predict the prediction results.

[0181] Based on this, optionally, the account characteristics corresponding to each of the multiple sample accounts, the content characteristics of the sample multimedia resources, the operation behavior sequence, and the multimedia resource sequence are input into the preset neural network model for prediction processing, and the process of obtaining the prediction results corresponding to each sample account can be implemented as follows: the operation behavior sequence is aggregated through the multiple first network layers in the preset neural network model to obtain the operation behavior characteristics; the multimedia resource sequence is aggregated through the multiple second network layers in the preset neural network model to obtain the multimedia resource characteristics; the account characteristics corresponding to each of the multiple sample accounts, the content characteristics of the sample multimedia resources, the operation behavior characteristics, and the multimedia resource characteristics are input into the multiple third network layers in the preset neural network model to obtain the prediction results corresponding to each sample account.

[0182] By aggregating the operation behavior sequence and the multimedia resource sequence, the operation behavior sequence and the multimedia resource sequence can be converted into non-sequence features.

[0183] Optionally, multiple first network layers adopt a multi-head attention mechanism to aggregate operation behavior sequences; multiple second network layers adopt a multi-head attention mechanism to aggregate multimedia resource sequences.

[0184] It is worth noting that the multi-head attention mechanism can extract important information from the operation behavior sequence or the multimedia resource sequence. Multiple third network layers use the extracted important information as input to facilitate the operation of multiple third network layers.

[0185] Optionally, the operation behavior sequence is aggregated by presetting multiple first network layers in the neural network model to obtain the operation behavior characteristics. The process can be implemented as follows: the importance of each target account is determined by presetting multiple first network layers in the neural network model; the importance of each target account is used as a weight to perform weighted summation of the operation behavior information of each target account in the operation behavior sequence to obtain the operation behavior characteristics.

[0186] The above process of determining the importance of each target account through multiple first network layers in the preset neural network model can be implemented as follows: obtaining the attribute information of the sample multimedia resources and the playback environment information when each target account plays the sample multimedia resources; inputting the account characteristics, attribute information, playback environment information, content characteristics of the sample multimedia resources, and operation behavior sequence into the multiple first network layers in the preset neural network model to obtain the importance of each target account.

[0187] The attribute information of the sample multimedia resource may include identification information of the sample multimedia resource, release time, classification information of the sample multimedia resource, classification information of the music used in the sample multimedia resource, etc. The attribute information of the sample multimedia resource may be any one or more of the above-mentioned information, or other information that can characterize the characteristics of the multimedia resource.

[0188] The above playing environment information may also be called context information, which may include the time when the sample multimedia resource is played, the page where the sample multimedia resource is played, etc. The page where the sample multimedia resource is played may be, for example, a discovery page, a follow page, a local page, etc.

[0189] In the multi-head attention mechanism, such as Figure 3 As shown, account characteristics, attribute information, playback environment information, and content characteristics of sample multimedia resources can be set as parameter query, and the operation behavior sequence can be set as parameter key and value. The set query, key, and value are used as input, and the importance of each target account is obtained by processing multiple first network layers in the preset neural network model. Then, the importance of each target account can be used as a weight to perform weighted summation on the operation behavior information of each target account in the operation behavior sequence to obtain the operation behavior feature. The operation behavior feature is a feature that the preset neural network model needs to use later. For example, it can be used as the input of the MMoe (Modeling TaskRelationships in Multi-task Learning with Multi-gate Mixture-of-Experts) multi-task model. The MMoe multi-task model is a possible model structure for implementing the preset neural network model.

[0190] It should be noted that the operational behavior characteristics can be regarded as the aggregation result of the identification information of the target account. The identification information of the target account is obtained based on a large number of behavioral training of the target account. The identification information of the target account contains rich characteristics, and the newly released sample multimedia resources have not yet been described by a large number of behaviors generated by a large number of accounts. Therefore, the above method is used to achieve: the identification information of the target account containing rich characteristics is used to describe the relatively "cold" sample multimedia resources. The operational behavior characteristics can be used to describe what kind of account will like such sample multimedia resources.

[0191] Optionally, by presetting multiple second network layers in the neural network model, the multimedia resource sequence is aggregated to obtain the multimedia resource characteristics. The process can be implemented as follows: by presetting multiple second network layers in the neural network model, the importance of each target multimedia resource is determined; using the importance of each target multimedia resource as a weight, weighted summing up each target multimedia resource in the multimedia resource sequence to obtain the multimedia resource characteristics.

[0192] Optionally, through multiple second network layers in a preset neural network model, the process of determining the importance of each target multimedia resource can be implemented as follows: obtaining attribute information of sample multimedia resources and playback environment information when each target account plays the sample multimedia resources; inputting account characteristics, attribute information, playback environment information, content characteristics of sample multimedia resources, operation behavior sequence and multimedia resource sequence into multiple second network layers in the preset neural network model to obtain the importance of each target multimedia resource.

[0193] In the multi-head attention mechanism, such as Figure 4 As shown, the account characteristics, attribute information, playback environment information and content characteristics of sample multimedia resources can be set as the parameter query, the multimedia resource sequence, operation behavior sequence and content characteristics of sample multimedia resources can be set as the parameter key, and the multimedia resource sequence can be set as the parameter value. The set query, key and value are used as input, and the importance of each target multimedia resource is obtained by processing multiple second network layers in the preset neural network model. Then, the importance of each target multimedia resource can be used as a weight to perform weighted summation on each target multimedia resource to obtain the multimedia resource feature. The multimedia resource feature is the feature that the preset neural network model needs to use in the network layer after the second network layer. In addition, the multimedia resource feature can also be determined as the initial value of the identification information of the sample multimedia resource, and the identification information of the sample multimedia resource can be changed through training later.

[0194] Through the above-mentioned method of obtaining the identification information of the sample multimedia resources, it can be known that the initial value of the identification information of the newly released sample multimedia resources can be obtained through multiple second network layers in the preset neural network model. Multiple target multimedia resources can be directly input into the preset neural network model as raw data, and the preset neural network model can aggregate the multiple target multimedia resources without adding additional steps outside the preset neural network model to process the multiple target multimedia resources. The output is directly obtained from the input through the model, thereby realizing the end-to-end training process of the preset neural network model during the initialization process.

[0195] In the disclosed embodiments, by combining the content features, operation behavior sequences, and multimedia resource sequences of sample multimedia resources, features of newly released multimedia resources that have similar usage effects to multimedia resources that have been released for some time can be extracted, thereby avoiding the use of the same recommendation strategy for recommending newly released multimedia resources and multimedia resources that have been released for some time with similar features, and avoiding the problem of homogenization of newly released multimedia resources and multimedia resources that have been released for some time.

[0196] After obtaining the account characteristics, content characteristics, operation behavior characteristics and multimedia resource characteristics corresponding to each of the multiple sample accounts, the account characteristics, content characteristics, operation behavior characteristics and multimedia resource characteristics corresponding to each of the multiple sample accounts can be input into multiple third network layers in the preset neural network model to obtain the prediction results corresponding to each sample account.

[0197] It should be noted that, in addition to the account features corresponding to each of the multiple sample accounts, the content features of the sample multimedia resources, the operation behavior features, and the multimedia resource features, the attribute information, the playback environment information, and other information can also be input into multiple third network layers, and the third network layer processes these features and finally outputs the prediction results corresponding to each sample account. The prediction results are used to indicate the probability of the corresponding sample account generating the target operation behavior on the sample multimedia resource, and the target operation behavior includes operation behaviors such as likes, comments, forwarding, and sharing.

[0198] In the prediction process, since the prediction is generally about the probability of different accounts generating target operation behaviors on the same multimedia resource, the account is changing while the multimedia resource is fixed, so the information used to represent the characteristics of the multimedia resource can be reused for prediction. Generally speaking, the information used to represent the characteristics of the multimedia resource can be stored in the parameter server. During the training process, if the information used to represent the characteristics of the multimedia resource is needed, the information used to represent the characteristics of the multimedia resource can be directly obtained from the parameter server for use, without having to calculate the information used to represent the characteristics of the multimedia resource again.

[0199] The above-mentioned information used to represent the characteristics of multimedia resources can be the embedding value corresponding to the identification information of the multimedia resource. The embedding value corresponding to the identification information of the multimedia resource is obtained based on a large number of behavioral trainings generated by a large number of accounts on the multimedia resources. Therefore, for newly released multimedia resources, its corresponding embedding value may not exist in the parameter server. If the embedding value corresponding to a newly released multimedia resource does not exist in the parameter server, it is necessary to initialize the embedding value corresponding to the newly released multimedia resource. Specifically, the method for determining the multimedia resource characteristics introduced above in the embodiment of the present disclosure can be used to determine the multimedia resource characteristics corresponding to the newly released multimedia resource, and then the determined multimedia resource characteristics are used as the embedding values ​​corresponding to the newly released multimedia resource, and stored in the parameter server.

[0200] By adopting the present disclosure, for newly released sample multimedia resources, even if the sample multimedia resources lack a large number of behaviors generated by a large number of accounts as training samples, it is still possible to collect content features, operation behavior sequences, and multimedia resource sequences obtained by understanding multimodal information such as image information and audio information in the sample multimedia resources. Among them, the operation behavior sequence can indicate what kind of account is more interested in such sample multimedia resources, so the operation behavior sequence can describe the sample multimedia resources from the side. Based on account features, content features of sample multimedia resources, operation behavior features, and multimedia resource features, etc., the preset neural network model is trained, so that the multimedia resource recommendation model obtained after training can accurately recommend newly released multimedia resources.

[0201] Another embodiment of the present disclosure provides a multimedia resource recommendation method, which can be performed by an electronic device, which can be a terminal device such as a PC, a laptop, a smart phone, etc., or a server. The server can be a physical server including an independent host, or a virtual server, or a cloud server or a server cluster.

[0202] Figure 5 is a flowchart of a multimedia resource recommendation method according to an exemplary embodiment. Figure 5 As shown, the method includes the following steps.

[0203] In step S51, content features of the multimedia resources to be recommended are obtained. The multimedia resources to be recommended refer to multimedia resources whose duration between the release time and the current time is less than a preset threshold.

[0204] In step S52, at least one target account is determined from multiple reference accounts that have operated the multimedia resources to be recommended, and an operation behavior sequence is determined based on the operation behavior information of the at least one target account on the recommended multimedia resources. The target account refers to an account whose operation behavior on the recommended multimedia resources meets a first preset condition.

[0205] In step S53, multimedia resource characteristics are obtained, wherein the multimedia resource characteristics are determined based on multiple target multimedia resources respectively operated by each target account, and the target multimedia resource refers to a multimedia resource that meets a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period.

[0206] In step S54, the account characteristics corresponding to the account to be recommended, the content characteristics of the multimedia resources to be recommended, the operation behavior sequence and the multimedia resource characteristics are input into the multimedia resource recommendation model for prediction processing to obtain the prediction result corresponding to the account to be recommended. The prediction result is used to indicate the probability of the account to be recommended generating the target operation behavior on the multimedia resource to be recommended.

[0207] In step S55, the multimedia resource to be recommended is recommended based on the prediction result corresponding to the account to be recommended.

[0208] The above multimedia resource recommendation model can be adopted Figure 1 The model is obtained by training the model training method provided in the illustrated embodiment.

[0209] The account features corresponding to the account to be recommended are obtained in a similar manner to the account features corresponding to the sample account, the content features of the multimedia resources to be recommended are obtained in a similar manner to the content features of the sample multimedia resources, and the operation behavior sequences corresponding to the multimedia resources to be recommended are obtained in a similar manner to the operation behavior sequences corresponding to the sample multimedia resources. The specific methods for obtaining these features will not be described in detail in the embodiments of this disclosure, and the specific methods for obtaining them can be found in the methods described in the previous embodiments.

[0210] It should be noted that in the prediction process, since the probability of different accounts generating target operation behaviors on the same multimedia resource is generally predicted, the account is changing while the multimedia resource is fixed, so the information used to represent the characteristics of the multimedia resource can be reused for prediction. Generally speaking, the information used to represent the characteristics of the multimedia resource can be stored in the parameter server. After the model is launched, if the information used to represent the characteristics of the multimedia resource needs to be used, the information used to represent the characteristics of the multimedia resource can be directly obtained from the parameter server for use, without having to calculate the information used to represent the characteristics of the multimedia resource again, which can speed up the model prediction.

[0211] The above-mentioned information used to represent the characteristics of multimedia resources can be the embedding value corresponding to the identification information of the multimedia resource. The embedding value corresponding to the identification information of the multimedia resource is obtained based on a large number of behavioral trainings generated by a large number of accounts on the multimedia resources. Therefore, for newly released multimedia resources, its corresponding embedding value may not exist in the parameter server. If the embedding value corresponding to a newly released multimedia resource does not exist in the parameter server, the embedding value corresponding to the newly released multimedia resource needs to be initialized. Specifically, the multimedia resource feature corresponding to the newly released multimedia resource can be determined, and then the determined multimedia resource feature is used as the embedding value corresponding to the newly released multimedia resource, and it is stored in the parameter server.

[0212] It is worth noting that in the process of starting a newly released multimedia resource, that is, when recommending a newly released multimedia resource to an account to be recommended, an important indicator of which accounts to recommend the newly released multimedia resource is the embedding value corresponding to the identification information of the newly released multimedia resource. After the embedding value corresponding to the newly released multimedia resource is initialized, the embedding value corresponding to the newly released multimedia resource can be obtained from the parameter server, and the newly released multimedia resource can be started based on the obtained embedding value.

[0213] Optionally, the above process of determining multimedia resource features may be implemented as follows: determining a multimedia resource sequence based on a plurality of target multimedia resources respectively operated by each target account; and performing aggregation processing on the multimedia resource sequence to obtain multimedia resource features.

[0214] Optionally, the process of aggregating the multimedia resource sequences to obtain multimedia resource features can be implemented as follows: adopting a multi-head attention mechanism to aggregate the multimedia resource sequences to obtain multimedia resource features.

[0215] Optionally, a multi-head attention mechanism is used to aggregate the multimedia resource sequence, and the process of obtaining multimedia resource features can be implemented as follows: determining the importance of each target multimedia resource in the multimedia resource sequence; taking the importance of each target multimedia resource as a weight, performing weighted summation on each target multimedia resource to obtain the multimedia resource features.

[0216] Optionally, the process of determining the importance of each target multimedia resource in a multimedia resource sequence can be implemented as follows: obtaining attribute information of the multimedia resource to be recommended and playback environment information when each target account plays the multimedia resource to be recommended; determining the importance of each target multimedia resource based on account characteristics, attribute information, playback environment information, content characteristics of the multimedia resource to be recommended, operation behavior sequence, and multimedia resource sequence corresponding to the account to be recommended.

[0217] The method for determining the multimedia resource characteristics corresponding to the multimedia resources to be recommended is similar to the method for determining the multimedia resource characteristics corresponding to the sample multimedia resources. Since the method for determining the multimedia resource characteristics corresponding to the sample multimedia resources has been introduced in detail in the previous embodiment, it will not be repeated in the present disclosed embodiment. The process of determining the multimedia resource characteristics corresponding to the multimedia resources to be recommended can be implemented with reference to the contents recorded in the previous embodiment.

[0218] In addition, it should be noted that since the embedding value corresponding to the newly released multimedia resource is determined based on the target multimedia resource, that is, the embedding value of the newly released multimedia resource is obtained by referring to the information of other multimedia resources, this is because the newly released multimedia resource still lacks a large number of behaviors generated by a large number of accounts to train personalized embedding values, so it is chosen to determine the embedding value corresponding to the newly released multimedia resource based on the target multimedia resource. After the above-mentioned multimedia resources are released for a period of time, since a large number of behaviors generated by a large number of accounts on the above-mentioned multimedia resources can be collected, the large number of behaviors generated by a large number of accounts collected can be used to train and obtain the embedding value corresponding to the above-mentioned multimedia resources. Then the initial value of the embedding is replaced with the newly determined embedding value and stored in the parameter server. This is because, after a period of time after the multimedia resource is released, it is still expected to learn the personalized information of the multimedia resource through training.

[0219] During the prediction process, the account characteristics corresponding to the account to be recommended, the content characteristics of the multimedia resources to be recommended, the operation behavior sequence and the multimedia resource characteristics can be input into the multimedia resource recommendation model for prediction processing to obtain the prediction result corresponding to the account to be recommended, and the prediction result is used to indicate the probability of the account to be recommended to generate the target operation behavior for the recommended multimedia resource. Optionally, the target operation behavior includes a variety of operation behaviors, such as likes, comments, forwarding, sharing, etc. It should be noted that a multimedia resource recommendation model for predicting the probability of generating different operation behaviors can be trained, and the probability of the account to be recommended generating different operation behaviors for the recommended multimedia resources can be predicted through different models. Alternatively, if the multimedia resource recommendation model itself is a multi-task model, then the probability of the account to be recommended generating different operation behaviors for the recommended multimedia resources can be predicted at one time through the same multimedia resource recommendation model.

[0220] Based on this, optionally, based on the prediction results corresponding to the account to be recommended, the process of recommending the multimedia resources to be recommended can be implemented as follows: weighted summation of the probabilities of generating various operation behaviors to obtain the recommendation index of the multimedia resources to be recommended; based on the recommendation index of the multimedia resources to be recommended, recommending the multimedia resources to be recommended.

[0221] Specifically, the probability of generating various operation behaviors can be weighted and summed by a linear weighted formula to obtain the recommendation index of the multimedia resource to be recommended. The linear weighted formula can be, for example, an ensemble sort formula.

[0222] Optionally, based on the recommendation index of the multimedia resources to be recommended, the process of recommending the multimedia resources to be recommended can be implemented as follows: obtaining the recommendation index corresponding to each of multiple candidate multimedia resources, the candidate multimedia resources including the multimedia resources to be recommended; sorting the multiple candidate multimedia resources in descending order according to the recommendation index, and selecting the multimedia resource with a preset number of digits in the front for recommendation.

[0223] It is worth noting that after the multimedia resource recommendation model is used for online prediction, the multimedia resource recommendation model can be continuously trained offline to continuously improve the confidence of the multimedia resource recommendation model, so that its online performance can be continuously improved.

[0224] Based on this, the embodiment of the present disclosure also provides an incremental update solution for the model. Optionally, the method provided by the embodiment of the present disclosure also includes: receiving model parameters sent by the model training device according to a preset period; and updating the locally set multimedia resource recommendation model based on the received model parameters.

[0225] By adopting the present disclosure, for newly released multimedia resources to be recommended, even if the multimedia resources to be recommended lack a large number of behaviors generated by a large number of accounts as training samples, it is still possible to collect content features, operation behavior sequences, and multimedia resource sequences obtained after understanding multimodal information such as image information and audio information in the multimedia resources to be recommended. Among them, the operation behavior sequence can indicate what kind of account is more interested in such multimedia resources to be recommended, so the operation behavior sequence can describe the multimedia resources to be recommended from the side. Based on the account characteristics, the content characteristics of the multimedia resources to be recommended, the operation behavior characteristics, and the multimedia resource characteristics, etc., prediction results can be obtained to assist in the accurate recommendation of the newly released multimedia resources, and then the accuracy of the recommendation can be improved based on the prediction results.

[0226] Figure 6 A multimedia resource recommendation model training device is shown according to an exemplary embodiment. Figure 6 , the device comprises:

[0227] An acquisition unit 61 is configured to acquire content features of a sample multimedia resource, where the sample multimedia resource refers to a multimedia resource whose duration between a release time and a current time is less than a preset threshold;

[0228] A determination unit 62 is configured to determine operation behavior information of a plurality of sample accounts that have operated the sample multimedia resource on the sample multimedia resource;

[0229] The determining unit 62 is configured to determine, according to the operation behavior information of the multiple sample accounts on the sample multimedia resources, labels corresponding to the multiple sample accounts, wherein the labels are used to indicate whether the corresponding sample accounts generate target operation behaviors on the sample multimedia resources;

[0230] The determining unit 62 is configured to determine at least one target account from the multiple sample accounts, and determine an operation behavior sequence according to the operation behavior information of the at least one target account on the sample multimedia resource, wherein the target account refers to an account whose operation behavior on the sample multimedia resource meets a first preset condition;

[0231] The determining unit 62 is configured to determine a multimedia resource sequence according to a plurality of target multimedia resources respectively operated by each target account, wherein the target multimedia resource refers to a multimedia resource that meets a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period;

[0232] The prediction unit 63 is configured to input the account features corresponding to each of the multiple sample accounts, the content features of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence into a preset neural network model for prediction processing, and obtain a prediction result corresponding to each of the sample accounts, wherein the prediction result is used to indicate the probability that the corresponding sample account generates the target operation behavior on the sample multimedia resource;

[0233] The training unit 64 is configured to execute training of the preset neural network model according to the difference between the labels corresponding to each of the sample accounts and the prediction results until a preset training end condition is met to obtain a multimedia resource recommendation model.

[0234] Optionally, the acquiring unit 61 is configured to execute:

[0235] Extracting features of an intermediate network layer output in a preset multimedia resource processing model, wherein the multimedia resource processing model takes the sample multimedia resource as input;

[0236] The extracted features are determined as content features of the sample multimedia resources.

[0237] Optionally, the prediction unit 63 is configured to execute:

[0238] Aggregating the operation behavior sequence by presetting a plurality of first network layers in a neural network model to obtain operation behavior features;

[0239] Aggregating the multimedia resource sequence through the plurality of second network layers in the preset neural network model to obtain multimedia resource features;

[0240] The account characteristics corresponding to each of the multiple sample accounts, the content characteristics of the sample multimedia resources, the operation behavior characteristics and the multimedia resource characteristics are input into multiple third network layers in the preset neural network model to obtain prediction results corresponding to each of the sample accounts.

[0241] Optionally, the multiple first network layers adopt a multi-head attention mechanism to aggregate the operation behavior sequence;

[0242] The multiple second network layers adopt a multi-head attention mechanism to perform aggregation processing on the multimedia resource sequence.

[0243] Optionally, the prediction unit 63 is configured to execute:

[0244] Determine the importance of each target account by presetting multiple first network layers in the neural network model;

[0245] The importance of each target account is used as a weight, and the operation behavior information of each target account in the operation behavior sequence is weighted and summed to obtain the operation behavior feature.

[0246] Optionally, the prediction unit 63 is configured to execute:

[0247] Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource;

[0248] The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources and the operation behavior sequence are input into multiple first network layers in a preset neural network model to obtain the importance of each target account.

[0249] Optionally, the prediction unit 63 is configured to execute:

[0250] Determining the importance of each target multimedia resource through a plurality of second network layers in the preset neural network model;

[0251] The importance of each target multimedia resource is used as a weight, and weighted sum is performed on each target multimedia resource in the multimedia resource sequence to obtain a multimedia resource feature.

[0252] Optionally, the prediction unit 63 is configured to execute:

[0253] Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource;

[0254] The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence are input into multiple second network layers in the preset neural network model to obtain the importance of each target multimedia resource.

[0255] Optionally, the device further comprises:

[0256] The updating unit is configured to send the model parameters in the multimedia resource recommendation model to the model application device, so that the model application device updates the online multimedia resource recommendation model set in the model application device based on the received model parameters.

[0257] Figure 7 A multimedia resource recommendation device is shown according to an exemplary embodiment. Figure 7 , the device comprises:

[0258] The acquisition unit 71 is configured to acquire content features of a multimedia resource to be recommended, wherein the multimedia resource to be recommended refers to a multimedia resource whose duration between a release time and a current time is less than a preset threshold;

[0259] The determining unit 72 is configured to determine at least one target account from a plurality of reference accounts that have operated the multimedia resource to be recommended, and determine an operation behavior sequence according to the operation behavior information of the at least one target account on the multimedia resource to be recommended, wherein the target account refers to an account whose operation behavior on the multimedia resource to be recommended meets a first preset condition;

[0260] The acquisition unit 71 is configured to execute acquisition of multimedia resource features, wherein the multimedia resource features are determined based on a plurality of target multimedia resources respectively operated by each target account, and the target multimedia resources refer to multimedia resources that meet a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period;

[0261] The prediction unit 73 is configured to perform prediction processing by inputting the account characteristics corresponding to the account to be recommended, the content characteristics of the multimedia resource to be recommended, the operation behavior sequence and the multimedia resource characteristics into the multimedia resource recommendation model to obtain a prediction result corresponding to the account to be recommended, and the prediction result is used to indicate the probability that the account to be recommended will generate a target operation behavior on the multimedia resource to be recommended;

[0262] The recommendation unit 74 is configured to perform a recommendation on the multimedia resource to be recommended based on the prediction result corresponding to the account to be recommended.

[0263] Optionally, the target operation behavior includes multiple operation behaviors, and the recommendation unit 74 is configured to execute:

[0264] Performing weighted summation on the probabilities of generating various operation behaviors to obtain a recommendation index of the multimedia resource to be recommended;

[0265] The multimedia resource to be recommended is recommended based on the recommendation index of the multimedia resource to be recommended.

[0266] Optionally, the recommendation unit 74 is configured to execute:

[0267] Acquire recommendation indicators corresponding to each of a plurality of candidate multimedia resources, wherein the candidate multimedia resources include the multimedia resource to be recommended;

[0268] The plurality of candidate multimedia resources are sorted in descending order according to the recommendation index, and a multimedia resource ranked first by a preset number of places is selected for recommendation.

[0269] Optionally, the apparatus further includes an updating unit, wherein the updating unit is configured to execute:

[0270] Receiving model parameters sent by the model training device according to a preset period;

[0271] The locally set multimedia resource recommendation model is updated based on the received model parameters.

[0272] Optionally, the acquiring unit 71 is configured to execute:

[0273] Determining a multimedia resource sequence based on the multiple target multimedia resources respectively operated by the target accounts;

[0274] Aggregation processing is performed on the multimedia resource sequence to obtain multimedia resource features.

[0275] Optionally, the acquiring unit 71 is configured to execute:

[0276] A multi-head attention mechanism is adopted to aggregate the multimedia resource sequence to obtain multimedia resource features.

[0277] Optionally, the acquiring unit 71 is configured to execute:

[0278] Determining the importance of each target multimedia resource in the multimedia resource sequence;

[0279] The importance of each target multimedia resource is used as a weight, and weighted sum is performed on the target multimedia resources to obtain multimedia resource features.

[0280] Optionally, the acquiring unit 71 is configured to execute:

[0281] Acquire attribute information of the multimedia resource to be recommended and playback environment information when each target account plays the multimedia resource to be recommended;

[0282] The importance of each target multimedia resource is determined based on the account characteristics corresponding to the account to be recommended, the attribute information, the playback environment information, the content characteristics of the multimedia resource to be recommended, the operation behavior sequence and the multimedia resource sequence.

[0283] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0284] In one possible design, the above Figure 6 The structure of the multimedia resource recommendation model training device shown can be implemented as an electronic device, such as Figure 8As shown, the electronic device may include: a processor 911 and a memory 921. The memory 921 stores executable code. When the executable code is executed by the processor 911, the processor 911 can at least implement the above-mentioned Figures 1 to 4 The multimedia resource recommendation model training method provided in the illustrated embodiment.

[0285] Optionally, the electronic device may further include a communication interface 931 for communicating with other devices.

[0286] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 921 including instructions, and the instructions can be executed by a processor 911 of an electronic device to complete the above method. Alternatively, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0287] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, which implements the aforementioned Figures 1 to 4 The multimedia resource recommendation model training method provided in the illustrated embodiment.

[0288] In one possible design, the above Figure 7 The structure of the multimedia resource recommendation device shown can be implemented as an electronic device, such as Fig. 9 As shown, the electronic device may include: a processor 912 and a memory 922. The memory 922 stores executable code. When the executable code is executed by the processor 912, the processor 912 can at least implement the above-mentioned Figure 5 The multimedia resource recommendation method provided in the illustrated embodiment.

[0289] Optionally, the electronic device may further include a communication interface 932 for communicating with other devices.

[0290] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 922 including instructions, and the above instructions can be executed by the processor 912 of the electronic device to complete the above method. Alternatively, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0291] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, which implements the aforementioned Figure 5 The multimedia resource recommendation method provided in the illustrated embodiment.

[0292] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0293] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A multimedia resource recommendation model training method, characterized in that: include: Acquire content features of sample multimedia resources, where the sample multimedia resources refer to multimedia resources whose duration between release time and current time is less than a preset threshold; Determining operation behavior information of multiple sample accounts that have operated the sample multimedia resource on the sample multimedia resource; Determine, according to the operation behavior information of the multiple sample accounts on the sample multimedia resource, labels corresponding to the multiple sample accounts respectively, wherein the labels are used to indicate whether the corresponding sample accounts generate target operation behaviors on the sample multimedia resource; Determine at least one target account from the multiple sample accounts, and determine an operation behavior sequence according to the operation behavior information of the at least one target account on the sample multimedia resource, wherein the target account refers to an account whose operation behavior on the sample multimedia resource meets a first preset condition; Determine a multimedia resource sequence according to a plurality of target multimedia resources respectively operated by each target account, wherein the target multimedia resource refers to a multimedia resource that satisfies a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period; Inputting the account features corresponding to each of the multiple sample accounts, the content features of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence into a preset neural network model for prediction processing, and obtaining a prediction result corresponding to each of the sample accounts, wherein the prediction result is used to indicate the probability that the corresponding sample account will generate the target operation behavior on the sample multimedia resource; According to the difference between the labels corresponding to each of the sample accounts and the prediction results, the preset neural network model is trained until a preset training end condition is met to obtain a multimedia resource recommendation model.

2. The method according to claim 1, characterized in that The obtaining of content features of sample multimedia resources includes: Extracting features of an intermediate network layer output in a preset multimedia resource processing model, wherein the multimedia resource processing model takes the sample multimedia resource as input; The extracted features are determined as content features of the sample multimedia resources.

3. The method according to claim 1, characterized in that The inputting the account features corresponding to each of the plurality of sample accounts, the content features of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence into a preset neural network model for prediction processing to obtain prediction results corresponding to each of the sample accounts includes: Aggregating the operation behavior sequence by presetting a plurality of first network layers in a neural network model to obtain operation behavior features; Aggregating the multimedia resource sequence through the plurality of second network layers in the preset neural network model to obtain multimedia resource features; The account characteristics corresponding to each of the multiple sample accounts, the content characteristics of the sample multimedia resources, the operation behavior characteristics and the multimedia resource characteristics are input into multiple third network layers in the preset neural network model to obtain prediction results corresponding to each of the sample accounts.

4. The method according to claim 3, characterized in that The multiple first network layers adopt a multi-head attention mechanism to aggregate the operation behavior sequence; The multiple second network layers adopt a multi-head attention mechanism to perform aggregation processing on the multimedia resource sequence.

5. The method according to claim 4, characterized in that The operation behavior sequence is aggregated by using a plurality of first network layers in a preset neural network model to obtain operation behavior features, including: Determine the importance of each target account by presetting multiple first network layers in the neural network model; The importance of each target account is used as a weight, and the operation behavior information of each target account in the operation behavior sequence is weighted and summed to obtain the operation behavior feature.

6. The method according to claim 5, characterized in that The determining the importance of each target account by presetting a plurality of first network layers in the neural network model includes: Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource; The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources and the operation behavior sequence are input into multiple first network layers in a preset neural network model to obtain the importance of each target account.

7. The method according to claim 4, characterized in that The step of aggregating the multimedia resource sequence through the plurality of second network layers in the preset neural network model to obtain multimedia resource features includes: Determining the importance of each target multimedia resource through a plurality of second network layers in the preset neural network model; The importance of each target multimedia resource is used as a weight, and weighted sum is performed on each target multimedia resource in the multimedia resource sequence to obtain a multimedia resource feature.

8. The method according to claim 7, characterized in that Determining the importance of each target multimedia resource through the plurality of second network layers in the preset neural network model includes: Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource; The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence are input into multiple second network layers in the preset neural network model to obtain the importance of each target multimedia resource.

9. The method according to claim 1, characterized in that: After obtaining the multimedia resource recommendation model, the method further includes: The model parameters in the multimedia resource recommendation model are sent to the model application device, so that the model application device updates the online multimedia resource recommendation model set in the model application device based on the received model parameters.

10. A multimedia resource recommendation method, characterized in that: include: Acquire content features of a multimedia resource to be recommended, wherein the multimedia resource to be recommended refers to a multimedia resource whose duration between a release time and a current time is less than a preset threshold; Determine at least one target account from multiple reference accounts that have operated the multimedia resource to be recommended, and determine an operation behavior sequence according to the operation behavior information of the at least one target account on the multimedia resource to be recommended, wherein the target account refers to an account whose operation behavior on the multimedia resource to be recommended meets a first preset condition; Acquiring multimedia resource features, wherein the multimedia resource features are determined based on a plurality of target multimedia resources respectively operated by each target account, and the target multimedia resources refer to multimedia resources that meet a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period; Inputting the account characteristics corresponding to the account to be recommended, the content characteristics of the multimedia resource to be recommended, the operation behavior sequence and the multimedia resource characteristics into the multimedia resource recommendation model for prediction processing, and obtaining the prediction result corresponding to the account to be recommended, wherein the prediction result is used to indicate the probability that the account to be recommended will generate the target operation behavior on the multimedia resource to be recommended; The multimedia resource to be recommended is recommended based on the prediction result corresponding to the account to be recommended.

11. The method according to claim 10, characterized in that The target operation behavior includes multiple operation behaviors, and the recommending the multimedia resource to be recommended based on the prediction result corresponding to the account to be recommended includes: Performing weighted summation on the probabilities of generating various operation behaviors to obtain a recommendation index of the multimedia resource to be recommended; The multimedia resource to be recommended is recommended based on the recommendation index of the multimedia resource to be recommended.

12. The method according to claim 11, characterized in that The recommending the multimedia resource to be recommended based on the recommendation index of the multimedia resource to be recommended includes: Acquire recommendation indicators corresponding to each of a plurality of candidate multimedia resources, wherein the candidate multimedia resources include the multimedia resource to be recommended; The plurality of candidate multimedia resources are sorted in descending order according to the recommendation index, and a multimedia resource ranked first by a preset number of places is selected for recommendation.

13. The method according to claim 10, characterized in that The method further comprises: Receiving model parameters sent by the model training device according to a preset period; The locally set multimedia resource recommendation model is updated based on the received model parameters.

14. The method according to claim 10, characterized in that The step of obtaining multimedia resource features includes: Determining a multimedia resource sequence based on the multiple target multimedia resources respectively operated by the target accounts; Aggregation processing is performed on the multimedia resource sequence to obtain multimedia resource features.

15. The method according to claim 14, characterized in that The aggregating the multimedia resource sequence to obtain multimedia resource features includes: A multi-head attention mechanism is adopted to aggregate the multimedia resource sequence to obtain multimedia resource features.

16. The method according to claim 15, characterized in that The multi-head attention mechanism is used to aggregate the multimedia resource sequence to obtain multimedia resource features, including: Determining the importance of each target multimedia resource in the multimedia resource sequence; The importance of each target multimedia resource is used as a weight, and weighted sum is performed on the target multimedia resources to obtain multimedia resource features.

17. The method according to claim 16, characterized in that The determining the importance of each target multimedia resource in the multimedia resource sequence includes: Acquire attribute information of the multimedia resource to be recommended and playback environment information when each target account plays the multimedia resource to be recommended; The importance of each target multimedia resource is determined based on the account characteristics corresponding to the account to be recommended, the attribute information, the playback environment information, the content characteristics of the multimedia resource to be recommended, the operation behavior sequence and the multimedia resource sequence.

18. A multimedia resource recommendation model training device, characterized in that: include: An acquisition unit is configured to acquire content features of sample multimedia resources, where the sample multimedia resources refer to multimedia resources whose duration between release time and current time is less than a preset threshold; A determining unit, configured to determine operation behavior information of a plurality of sample accounts that have operated the sample multimedia resource on the sample multimedia resource; The determining unit is configured to determine, according to the operation behavior information of the multiple sample accounts on the sample multimedia resources, a label corresponding to each of the multiple sample accounts, wherein the label is used to indicate whether the corresponding sample account generates a target operation behavior on the sample multimedia resources; The determining unit is configured to determine at least one target account from the multiple sample accounts, and determine an operation behavior sequence according to the operation behavior information of the at least one target account on the sample multimedia resource, wherein the target account refers to an account whose operation behavior on the sample multimedia resource meets a first preset condition; The determining unit is configured to determine a multimedia resource sequence according to a plurality of target multimedia resources respectively operated by each target account, wherein the target multimedia resource refers to a multimedia resource that meets a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period; A prediction unit is configured to perform prediction processing by inputting account features corresponding to each of the multiple sample accounts, content features of the sample multimedia resources, the operation behavior sequence, and the multimedia resource sequence into a preset neural network model to obtain a prediction result corresponding to each of the sample accounts, wherein the prediction result is used to indicate the probability that the corresponding sample account generates the target operation behavior on the sample multimedia resource; The training unit is configured to execute training of the preset neural network model according to the difference between the labels corresponding to each of the sample accounts and the prediction results until a preset training end condition is met to obtain a multimedia resource recommendation model.

19. The device according to claim 18, characterized in that The acquisition unit is configured to execute: Extracting features of an intermediate network layer output in a preset multimedia resource processing model, wherein the multimedia resource processing model takes the sample multimedia resource as input; The extracted features are determined as content features of the sample multimedia resources.

20. The device according to claim 18, characterized in that The prediction unit is configured to perform: Aggregating the operation behavior sequence by presetting a plurality of first network layers in a neural network model to obtain operation behavior features; Aggregating the multimedia resource sequence through the plurality of second network layers in the preset neural network model to obtain multimedia resource features; The account characteristics corresponding to each of the multiple sample accounts, the content characteristics of the sample multimedia resources, the operation behavior characteristics and the multimedia resource characteristics are input into multiple third network layers in the preset neural network model to obtain prediction results corresponding to each of the sample accounts.

21. The device according to claim 20, characterized in that The multiple first network layers adopt a multi-head attention mechanism to aggregate the operation behavior sequence; The multiple second network layers adopt a multi-head attention mechanism to perform aggregation processing on the multimedia resource sequence.

22. The device according to claim 21, characterized in that The prediction unit is configured to perform: Determine the importance of each target account by presetting multiple first network layers in the neural network model; The importance of each target account is used as a weight, and the operation behavior information of each target account in the operation behavior sequence is weighted and summed to obtain the operation behavior feature.

23. The device according to claim 22, characterized in that The prediction unit is configured to perform: Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource; The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources and the operation behavior sequence are input into multiple first network layers in a preset neural network model to obtain the importance of each target account.

24. The device according to claim 21, characterized in that The prediction unit is configured to perform: Determining the importance of each target multimedia resource through a plurality of second network layers in the preset neural network model; The importance of each target multimedia resource is used as a weight, and weighted sum is performed on each target multimedia resource in the multimedia resource sequence to obtain a multimedia resource feature.

25. The device according to claim 24, characterized in that The prediction unit is configured to perform: Acquire attribute information of the sample multimedia resource and playback environment information when each target account plays the sample multimedia resource; The account characteristics, the attribute information, the playback environment information, the content characteristics of the sample multimedia resources, the operation behavior sequence and the multimedia resource sequence are input into multiple second network layers in the preset neural network model to obtain the importance of each target multimedia resource.

26. The device according to claim 18, characterized in that The device also includes: The updating unit is configured to send the model parameters in the multimedia resource recommendation model to the model application device, so that the model application device updates the online multimedia resource recommendation model set in the model application device based on the received model parameters.

27. A multimedia resource recommendation device, characterized in that: include: An acquisition unit is configured to acquire content features of a multimedia resource to be recommended, wherein the multimedia resource to be recommended refers to a multimedia resource whose duration between a release time and a current time is less than a preset threshold; a determining unit configured to determine at least one target account from a plurality of reference accounts that have operated the multimedia resource to be recommended, and determine an operation behavior sequence according to operation behavior information of the at least one target account on the multimedia resource to be recommended, wherein the target account refers to an account whose operation behavior on the multimedia resource to be recommended meets a first preset condition; The acquisition unit is configured to execute acquisition of multimedia resource features, wherein the multimedia resource features are determined based on a plurality of target multimedia resources respectively operated by each target account, and the target multimedia resources refer to multimedia resources that meet a second preset condition among the multimedia resources operated by the corresponding target account within a preset historical time period; A prediction unit is configured to perform prediction processing by inputting account features corresponding to the account to be recommended, content features of the multimedia resource to be recommended, the operation behavior sequence, and the multimedia resource features into a multimedia resource recommendation model to obtain a prediction result corresponding to the account to be recommended, wherein the prediction result is used to indicate the probability that the account to be recommended will generate a target operation behavior on the multimedia resource to be recommended; The recommendation unit is configured to perform a recommendation on the multimedia resource to be recommended based on the prediction result corresponding to the account to be recommended.

28. The device according to claim 27, characterized in that The target operation behavior includes multiple operation behaviors, and the recommendation unit is configured to execute: Performing weighted summation on the probabilities of generating various operation behaviors to obtain a recommendation index of the multimedia resource to be recommended; The multimedia resource to be recommended is recommended based on the recommendation index of the multimedia resource to be recommended.

29. The device according to claim 28, characterized in that The recommendation unit is configured to perform: Acquire recommendation indicators corresponding to each of a plurality of candidate multimedia resources, wherein the candidate multimedia resources include the multimedia resource to be recommended; The plurality of candidate multimedia resources are sorted in descending order according to the recommendation index, and a multimedia resource ranked first by a preset number of places is selected for recommendation.

30. The device according to claim 27, characterized in that The apparatus further includes an updating unit, wherein the updating unit is configured to execute: Receiving model parameters sent by the model training device according to a preset period; The locally set multimedia resource recommendation model is updated based on the received model parameters.

31. The device according to claim 27, characterized in that The acquisition unit is configured to execute: Determining a multimedia resource sequence based on the multiple target multimedia resources respectively operated by the target accounts; Aggregation processing is performed on the multimedia resource sequence to obtain multimedia resource features.

32. The device according to claim 31, characterized in that The acquisition unit is configured to execute: A multi-head attention mechanism is adopted to aggregate the multimedia resource sequence to obtain multimedia resource features.

33. The device according to claim 32, characterized in that The acquisition unit is configured to execute: Determining the importance of each target multimedia resource in the multimedia resource sequence; The importance of each target multimedia resource is used as a weight, and weighted sum is performed on the target multimedia resources to obtain multimedia resource features.

34. The device according to claim 33, characterized in that The acquisition unit is configured to execute: Acquire attribute information of the multimedia resource to be recommended and playback environment information when each target account plays the multimedia resource to be recommended; The importance of each target multimedia resource is determined based on the account characteristics corresponding to the account to be recommended, the attribute information, the playback environment information, the content characteristics of the multimedia resource to be recommended, the operation behavior sequence and the multimedia resource sequence.

35. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the multimedia resource recommendation model training method as described in any one of claims 1-9 or the multimedia resource recommendation method as described in any one of claims 10-17.

36. A computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the multimedia resource recommendation model training method described in any one of claims 1-9 or the multimedia resource recommendation method described in any one of claims 10-17.

37. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the multimedia resource recommendation model training method described in any one of claims 1-9 or the multimedia resource recommendation method described in any one of claims 10-17 is implemented.

Images