Multimedia data recall method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a multimedia data recall method and device, equipment and a storage medium, which are used for improving the recall effect of multimedia data in a cold start stage under the condition that the recall of the multimedia data in the cold start stage is not influenced. The method comprises the steps of obtaining object description data of a target object; adopting an object feature extraction network in a trained target prediction model to obtain corresponding object features based on the object description data; for each piece of candidate multimedia data, performing weighted fusion on corresponding comprehensive features obtained based on the identification information and the attribute information and corresponding attribute features obtained based on the attribute information through weighted parameters obtained based on the click frequency to obtain corresponding multimedia data features; and determining at least one piece of target multimedia data recalled for the target object in each piece of candidate multimedia data according to a feature similarity condition between the object feature and each piece of multimedia data feature.

Description

Background Art

[0002] With the development of the Internet, network platforms provide more and more conveniences for target objects, and target objects can conduct transactions or browse through the Internet. In order to increase exposure, multimedia data needs to be placed on the Internet.

[0003] Due to the different needs or interests of the target objects, different target objects have different needs for multimedia data. Different types of multimedia data usually need to be pushed to different target objects. Therefore, before delivering multimedia data to the target objects, it is necessary to determine the delivered multimedia data, that is, to recall the multimedia data. In other words, different multimedia data need to be recalled for different target objects.

[0004] In the multimedia data industry, each multimedia data corresponds to an information identification (Identity document, ID). When the multimedia data is clicked by the target object, the interaction record between the multimedia data and the target object can establish an association relationship with the information ID of the multimedia data. At present, a large amount of new multimedia data is created every day. Since the newly created multimedia data has not been clicked by the target object, the fine-grained features (such as information ID) of the newly created multimedia data lack the interaction records between the multimedia data and the target object associated with it.

[0005] In the initial stage of the delivery of newly created multimedia data, since the information ID of the new multimedia data has no associated interaction record, the multimedia data is considered to be in the cold start stage. When the multimedia data is clicked a set number of times by the target object, the information ID of the multimedia data is associated with a certain degree of interaction record between the multimedia data and the target object, and at this time, it can be considered that the multimedia data has passed the cold start stage. For multimedia data that has passed the cold start stage, since the information ID is associated with the interaction record, its interaction record is crucial for determining the recalled multimedia data for different target objects. Therefore, in the recall stage, the information ID is often one of the most important features.

[0006] Among the current recall schemes, there are information identification masking (ID Masking) and comparative learning schemes for multimedia data. This scheme may have the problem of over-weakening the information ID, that is, the information ID of the multimedia data is no longer referenced when determining the recalled multimedia data. Over-weakening the information ID of multimedia data will make the clicked multimedia data, like the newly created multimedia data, lack the interaction record with the target object, thereby affecting the recall effect of multimedia data that has passed the cold start period. Summary of the invention

[0007] The embodiments of the present application provide a multimedia data recall method, apparatus, device and storage medium, which are used to improve the recall effect of multimedia data in the cold start phase without affecting the recall of multimedia data that has passed the cold start period.

[0008] In a first aspect, an embodiment of the present application provides a multimedia data recall method, comprising:

[0009] Get object description data of the target object;

[0010] Using an object feature extraction network in a trained target prediction model to obtain corresponding object features based on the object description data;

[0011] Using the data feature extraction network in the target prediction model, for each candidate multimedia data, weighted fusion is performed on the corresponding comprehensive features obtained based on the identification information and the attribute information, and the corresponding attribute features obtained based on the attribute information, respectively, by using the weighted parameters obtained based on the click frequency, so as to obtain the corresponding multimedia data features; wherein the candidate multimedia data includes the click frequency, the identification information and the attribute information;

[0012] According to the feature similarity between the object feature and the features of each multimedia data, at least one target multimedia data to be recalled for the target object is determined from among the candidate multimedia data.

[0013] In a second aspect, an embodiment of the present application provides a multimedia data recall device, including:

[0014] A first acquisition unit, used to acquire object description data of a target object;

[0015] The second acquisition unit is used to use the object feature extraction network in the trained target prediction model to perform weighted fusion on the corresponding comprehensive features obtained based on the identification information and the attribute information, and the corresponding attribute features obtained based on the attribute information, for each candidate multimedia data, using a weighted parameter obtained based on the click frequency, to obtain a corresponding multimedia data feature; wherein the candidate multimedia data includes the click frequency, the identification information, and the attribute information;

[0016] A determination unit is used to determine at least one target multimedia data to be recalled for the target object from among the candidate multimedia data according to the feature similarity between the object feature and the features of the multimedia data.

[0017] In a possible implementation, the determination unit is further used to train the target prediction model in the following manner: obtain a training sample set; a training sample includes: object description data of a sample object, a sample data set of sample multimedia data, and an operation label of the sample object for the sample multimedia data; the sample data set includes identification information, attribute information and click frequency; the training sample set is used to perform multiple rounds of iterative training on the constructed initial prediction model to obtain a trained target prediction model, wherein in one round of iteration, the following operations are performed:

[0018] Using the initial prediction model, obtaining object features based on the object description data in the read training sample, and obtaining multimedia data features obtained by weighting the comprehensive features and the attribute features based on the corresponding sample data set, and outputting a prediction result based on the object features and the multimedia data features;

[0019] Based on the result difference between the prediction result and the corresponding operation label, the model parameters are adjusted.

[0020] In a possible implementation, the initial prediction model includes: an object feature extraction network, a global prediction network, and a data feature extraction network composed of a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork, and a gating subnetwork; when the determination unit obtains multimedia data features obtained by weighting comprehensive features and attribute features based on the corresponding sample data set, it is specifically used to:

[0021] Based on the identification information, attribute information, and click frequency in the sample data set, information embedding processing is performed respectively to obtain corresponding identification embedding results, attribute embedding results, and frequency embedding results;

[0022] Using the comprehensive feature extraction sub-network, based on the identification embedding result and the attribute embedding result, obtain the corresponding comprehensive feature, using the attribute feature extraction sub-network, based on the attribute embedding result, obtain the corresponding attribute feature, and using the gating sub-network, based on the frequency embedding result, obtain the weighting parameter;

[0023] The comprehensive features and the attribute features are weighted and fused according to the weighting parameters to obtain corresponding multimedia data features.

[0024] In a possible implementation, the initial prediction model includes: an object feature extraction network, a global prediction network, and a data feature extraction network composed of a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork and a gating subnetwork, a comprehensive prediction subnetwork connected to the comprehensive feature extraction subnetwork, and an attribute prediction subnetwork connected to the attribute feature extraction subnetwork;

[0025] The determining unit, when outputting the prediction result based on the object feature and the multimedia data feature, is specifically used to:

[0026] Using the comprehensive prediction subnetwork, based on the comprehensive features and the object features, obtain a first operation prediction result of the sample object on the sample multimedia data;

[0027] Using the attribute prediction subnetwork, based on the attribute feature and the object feature, obtain a second operation prediction result of the sample object on the sample multimedia data;

[0028] The global prediction network is used to obtain a third operation prediction result of the sample object on the sample multimedia data based on the multimedia features and the object features.

[0029] In a possible implementation, when the determining unit adjusts the model parameters based on the result difference between the prediction result and the corresponding operation label, it is specifically used to:

[0030] Calculating a first loss value based on a first result difference between the first operation prediction result and the corresponding operation label, calculating a second loss value based on a second result difference between the second operation prediction result and the corresponding operation label, and calculating a third loss value based on a third result difference between the third operation prediction result and the corresponding operation label;

[0031] Adjust model parameters according to the first loss value, the second loss value and the third loss value.

[0032] In a possible implementation, when the determining unit adjusts the model parameters based on the result difference between the prediction result and the corresponding operation label, it is specifically used to:

[0033] Calculating a first loss value based on a first result difference between the first operation prediction result and the corresponding operation label, and calculating a second loss value based on a second result difference between the second operation prediction result and the corresponding operation label;

[0034] When it is determined that the first loss value is not higher than the second loss value, calculating a distillation loss based on a result difference between the first operation prediction result and the second operation prediction result;

[0035] Adjust model parameters based on a third loss value calculated based on a third result difference between the third operation prediction result and the corresponding operation label, and the distillation loss.

[0036] In a possible implementation, the determination unit is further configured to construct a training sample set in the following manner: obtaining a historical operation data set recording multimedia data of respective operations of each business object;

[0037] Select each sample object from each business object, and count the click frequencies corresponding to each multimedia data based on the historical operation data set, and select corresponding sample multimedia data for each sample object; for each sample object, perform the following operations: combine the sample object with the corresponding sample multimedia data to obtain corresponding training samples; a training sample includes: object description data of a sample object, click frequency, identification information and attribute information of a sample multimedia data, and an operation label determined according to the operation of the sample object on the multimedia data.

[0038] In a possible implementation manner, after acquiring the object description data of the target object, the second determining unit is further configured to:

[0039] Determining a range of candidate multimedia data matching the target object according to the object description data of the target object;

[0040] In the candidate multimedia data range, candidate multimedia data corresponding to the target object are selected according to the content type of the multimedia data historically operated by the target object.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is used to store computer instructions; and the processor is used to execute computer instructions to implement the steps of the multimedia data recall method provided in the embodiment of the present application.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, the steps of the multimedia data recall method provided in the embodiment of the present application are implemented.

[0043] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; when the processor of an electronic device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, so that the electronic device executes the steps of the multimedia data recall method provided in the embodiment of the present application.

[0044] The beneficial effects of this application are as follows:

[0045] The embodiment of the present application provides a multimedia data recall method, device, equipment and storage medium. In the embodiment of the present application, the object description data of the target object is first obtained, and the object feature extraction network in the trained target prediction model is used to obtain the corresponding object features based on the object description data; then the multimedia data features are respectively outputted by the data feature extraction network in the target prediction model based on the identification information, attribute information and click frequency of the candidate multimedia data corresponding to each candidate multimedia data; finally, according to the feature similarity between the object feature and the features of each multimedia data, at least one target multimedia data to be recalled for the target object is determined in each candidate multimedia data. Among them, the data feature extraction network includes: a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork, and a gating subnetwork; the gating subnetwork is used to output a weighted parameter between the comprehensive feature and the attribute feature of a candidate multimedia data according to the click frequency of a candidate multimedia data. In the present application, during the training process of the target prediction model, the weighted parameter between the comprehensive feature and the attribute feature of the multimedia data is determined by the click frequency of the multimedia data, and then the importance of the identification information to the multimedia data is determined according to the click frequency of the multimedia data. Therefore, through the identification information, attribute information, and click frequency of each multimedia data, the characteristics of each multimedia data are determined based on the trained target prediction model, and the object characteristics are determined based on the trained target prediction model, so as to determine the click probability of each multimedia data through the characteristics of each multimedia data and the object characteristics, and then determine the target multimedia data to be recalled for the target object. In the above manner, the importance of identification information to multimedia data can be determined through click frequency without affecting the recall of multimedia data that has passed the cold start period, thereby improving the recall effect of multimedia data in the cold start stage.

[0046] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0049] Figure 2 A flowchart of a multimedia data recall method provided in an embodiment of the present application;

[0050] Figure 3 A training flow chart of a target prediction model provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the structure of a target prediction model provided in an embodiment of the present application;

[0052] Figure 5 A schematic diagram of the structure of another target prediction model provided in an embodiment of the present application;

[0053] Figure 6 A schematic diagram of a method for training a target prediction model provided in an embodiment of the present application;

[0054] Figure 7 A schematic diagram of obtaining object features provided in an embodiment of the present application;

[0055] Figure 8 A schematic diagram of obtaining comprehensive features provided in an embodiment of the present application;

[0056] Fig. 9 A schematic diagram of obtaining attribute features provided in an embodiment of the present application;

[0057] Fig.10 A schematic diagram of determining a first operation prediction result provided in an embodiment of the present application;

[0058] Fig.11 A schematic diagram of determining a second operation prediction result provided in an embodiment of the present application;

[0059] Fig.12 A schematic diagram of a multimedia data recall device provided in an embodiment of the present application;

[0060] Fig.13 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and beneficial effects of this application clearer, the technical solution in the embodiment of this application will be clearly and completely described in conjunction with the drawings in the embodiment of this application. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] In order to help those skilled in the art better understand the technical solution of the present application, some concepts involved in the present application are introduced below.

[0063] Multimedia cold start: Multimedia cold start refers to the process at the beginning of a brand new multimedia account or advertising multimedia series. When a multimedia account or multimedia series is just created, it needs to go through a period of cold start to build brand awareness, collect data, and optimize multimedia strategies. During the multimedia cold start stage, due to the lack of historical data and optimization strategies, the performance of the multimedia account or multimedia series may be relatively low. At this time, a series of experiments and strategy adjustments are needed to optimize multimedia materials, positioning, and delivery strategies.

[0064] In the present application, when multimedia data is clicked by the target object for a set number of times, it can be considered that the multimedia data has passed the cold start stage.

[0065] The word “exemplary” is used hereinafter to mean “serving as an example, example, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0066] The terms "first" and "second" in this document are used for descriptive purposes only and should not be understood as explicitly or implicitly indicating relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, unless otherwise specified, the meaning of "multiple" is two or more.

[0067] The following is a brief introduction to the design concept of the embodiment of the present application:

[0068] The cold start phase of multimedia refers to the process when a new multimedia series or multimedia account is just beginning to be launched. During this phase, multimedia owners need to build brand awareness, attract target audiences, and collect data to optimize multimedia effects. During the cold start phase of multimedia, due to the lack of historical data and optimization strategies, the performance of multimedia may be relatively low. Therefore, a series of strategic adjustments are needed to gradually improve the multimedia effect.

[0069] In the multimedia delivery stage, different target objects have different needs or interests, and different target objects have different needs for multimedia data. Therefore, different multimedia data needs to be recalled for different target objects.

[0070] In the initial stage of launching new multimedia data, since the identification information ID of the new multimedia data has no associated interaction records, the multimedia data is considered to be in the cold start stage. When the multimedia data is clicked a set number of times by the target object, the identification information ID of the multimedia data is associated with a certain degree of interaction records between the multimedia data and the target object. At this time, it can be considered that the multimedia data has passed the cold start stage. For multimedia data that has passed the cold start stage, since the identification information ID is associated with the interaction records, for different target objects, its interaction records are crucial to determine the recalled multimedia data. Therefore, in the recall stage, the identification information ID is often one of the most important features.

[0071] Among the current recall schemes, there are ID Masking and comparative learning schemes for multimedia data. This scheme may have the problem of over-weakening the ID information, that is, the ID information of the multimedia data is no longer referenced when determining the recalled multimedia data. Over-weakening the ID information of the multimedia data will make the clicked multimedia data, like the newly created multimedia data, lack the interaction record with the target object, thus affecting the recall effect of the multimedia data that has passed the cold start period.

[0072] Therefore, there is an urgent need for a multimedia data retrieval method that can improve the recall effect of multimedia data in the cold start phase without affecting the recall of multimedia data that has passed the cold start phase.

[0073] In view of this, the embodiments of the present application provide a multimedia data recall method, apparatus, device and storage medium. In the embodiments of the present application, the object description data of the target object is first obtained, and the object feature extraction network in the trained target prediction model is used to obtain the corresponding object features based on the object description data; then, for each candidate multimedia data, the corresponding comprehensive features obtained based on the identification information and the attribute information, and the corresponding attribute features obtained based on the attribute information are weighted and fused by using the weighting parameters obtained based on the click frequency to obtain the corresponding multimedia data features; finally, based on the feature similarity between the object feature and the features of each multimedia data, at least one target multimedia data for recalling the target object is determined in each candidate multimedia data.

[0074] In an embodiment of the present application, during the training process of the target prediction model, the weighted parameters between the comprehensive features and the attribute features of the multimedia data are determined by the click frequency of the multimedia data, and then the importance of the identification information to the multimedia data is determined according to the click frequency of the multimedia data. Therefore, the characteristics of each multimedia data are determined based on the trained target prediction model through the identification information, attribute information, and click frequency of each multimedia data, and the object characteristics are determined based on the trained target prediction model, so as to determine the click probability of each multimedia data through the characteristics of each multimedia data and the object characteristics, and then determine the target multimedia data to be recalled for the target object. In the above manner, the importance of the identification information to the multimedia data can be determined by the click frequency without affecting the recall of the multimedia data that has passed the cold start period, and then the recall effect of the multimedia data in the cold start stage can be improved.

[0075] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0076] refer to Figure 1 , Figure 1 Schematic diagram of an application scenario of an embodiment of the present application, the application scenario includes a target object 100 and an execution device 200. In some scenarios, the execution device 200 may be an electronic device, which may be a display device with a display function. In other scenarios, the execution device may be a server, which may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0077] In some embodiments, the target object 100 corresponds to a target object account, and when the user clicks on the multimedia data, a record of interaction with the multimedia data will be left in the target object account. The execution device 200 needs to determine the recalled multimedia data based on the interaction record with the multimedia data in the target object account for different target object accounts.

[0078] Figure 1 The figure is only an example. In fact, the number of target objects is not limited and is not specifically limited in the embodiments of the present application.

[0079] Based on the above application scenarios, the multimedia data recall method provided by the exemplary embodiment of the present application is described below in combination with the application scenarios described above according to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation methods of the present application are not limited in this respect.

[0080] refer to Figure 2 , Figure 2 A flowchart of a multimedia data recall method in an embodiment of the present application is provided as an example, and the method includes:

[0081] 201, obtaining object description data of a target object.

[0082] In some embodiments, the description data of the target object includes the image features and behavior features of the target object. The work is a distance, the image features may include the age and gender of the target object, and the behavior features may include the click records of the target object. Specifically, the behavior features such as the target object's article reading records, multimedia data click records, and multimedia data conversion records in the past 14 days may be obtained.

[0083] 202, using the object feature extraction network in the trained target prediction model to obtain corresponding object features based on the object description data.

[0084] In some embodiments, the object description data may be input into an object feature extraction network in a trained target prediction model, so that feature extraction is performed on the object description data through the object feature extraction network to obtain corresponding object features.

[0085] 203. Using the data feature extraction network in the target prediction model, for each candidate multimedia data, weighted fusion is performed on the corresponding comprehensive features obtained based on the identification information and the attribute information, and the corresponding attribute features obtained based on the attribute information, respectively, through the weighted parameters obtained based on the click frequency, to obtain the corresponding multimedia data features.

[0086] The candidate multimedia data includes click frequency, identification information and attribute information.

[0087] In some embodiments, the data feature extraction network in the target prediction model can perform feature extraction on the identification information and attribute information of the candidate multimedia data to determine the comprehensive features of the candidate multimedia data, and perform feature extraction on the attribute information of the candidate multimedia data to obtain attribute features.

[0088] In some embodiments, a data feature extraction network in a target prediction model can be used to determine weighting parameters between attribute features and comprehensive features based on the click frequency of candidate multimedia data, and weight the attribute features and comprehensive features based on the weighting parameters to obtain multimedia data features of the candidate multimedia data.

[0089] Specifically, the data feature extraction network includes: a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork, and a gating subnetwork. In some scenarios, the comprehensive feature extraction subnetwork can be used to extract features from identification information and attribute information to obtain comprehensive features of candidate multimedia data. The attribute feature extraction subnetwork can be used to extract features from attribute information to obtain attribute features of candidate multimedia data. Furthermore, the gating subnetwork can be used to output a weighting parameter between a comprehensive feature and an attribute feature of a candidate multimedia data based on the click frequency of a candidate multimedia data. Furthermore, the comprehensive features and attribute features of the candidate multimedia data can be weighted according to the weighting parameter to obtain the multimedia data features of the candidate multimedia data.

[0090] 204 , determining at least one target multimedia data to be recalled for the target object from among the candidate multimedia data according to the feature similarity between the object feature and the features of the multimedia data.

[0091] In some embodiments, the inner product between the object feature and the multimedia data feature can be determined, and the inner product can be used as the prediction result. Further, based on the inner product, the target multimedia data to be recalled for the target object in each candidate multimedia data is determined. As an example, the inner product between the object feature and the multimedia data feature is the probability of each candidate pair of media data being predicted to be clicked by the target object, and then the N multimedia data with the highest interest rate are used as the target multimedia data to be recalled for the target object. For example, there are 10 candidate multimedia data, and the inner product results with the object feature are 0.82, 0.47, 0.75, 0.95, 0.61, 0.83, 0.79, 0.91, 0.87, and 0.74 respectively. If it is set to recall 6 media data for the target object, the multimedia data corresponding to the top 6 inner product results are used as the target pair of media data.

[0092] In some embodiments, the target prediction model is obtained by iteratively training the initial prediction model using the training sample set. Figure 3 As shown, Figure 3 The training process of the target prediction model is shown as an example. The target prediction model can be trained in the following way, as follows:

[0093] 301, obtain a training sample set.

[0094] A training sample includes: object description data of a sample object, a sample data set of sample multimedia data, and an operation label of a sample object on a sample multimedia data. The sample data set includes identification information, attribute information, and click frequency. The operation label is used to describe whether the sample object performs a click operation on the sample multimedia data.

[0095] 302, using the training sample set, performing multiple rounds of iterative training on the constructed initial prediction model to obtain a trained target prediction model.

[0096] In one round of iteration, the following operations are performed:

[0097] 3021, using the initial prediction model, obtaining object features based on the object description data in the read training samples, and obtaining multimedia data features obtained by weighting the comprehensive features and attribute features based on the corresponding sample data set, and outputting prediction results based on the object features and the multimedia data features.

[0098] In some embodiments, the initial prediction model includes: an object feature extraction network, a global prediction network, and a data feature extraction network composed of a comprehensive feature extraction sub-network, an attribute feature extraction sub-network, and a gating sub-network, such as Figure 4 As shown. Based on the corresponding sample data set, the multimedia data features obtained by weighting the comprehensive features and attribute features can be achieved in the following way: based on the identification information, attribute information, and click frequency in the sample data set, information embedding processing is performed respectively to obtain the corresponding identification embedding results, attribute embedding results, and frequency embedding results. Then, a comprehensive feature extraction sub-network can be used to obtain the corresponding comprehensive features based on the identification embedding results and the attribute embedding results, and an attribute feature extraction sub-network can be used to obtain the corresponding attribute features based on the attribute embedding results, and a gating sub-network can be used to obtain the weighting parameters based on the frequency embedding results. Furthermore, the comprehensive features and attribute features can be weighted fused according to the weighting parameters to obtain the corresponding multimedia data features.

[0099] In some embodiments, the initial prediction model includes: an object feature extraction network, a global prediction network, and a data feature extraction network composed of a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork and a gating subnetwork, a comprehensive prediction subnetwork connected to the comprehensive feature extraction subnetwork, and an attribute prediction subnetwork connected to the attribute feature extraction subnetwork, such as Figure 5As shown. Based on the object features and the multimedia data features, outputting the prediction results includes: using a comprehensive prediction sub-network to obtain a first operation prediction result of the sample object on the sample multimedia data based on the comprehensive features and the object features. Further, an attribute prediction sub-network can be used to obtain a second operation prediction result of the sample object on the sample multimedia data based on the attribute features and the object features. Finally, a global prediction network is used to obtain a third operation prediction result of the sample object on the sample multimedia data based on the multimedia features and the object features.

[0100] 3022, adjusting model parameters based on the result difference between the prediction result and the corresponding operation label.

[0101] In some embodiments, a first loss value may be calculated based on a first result difference between a first operation prediction result and a corresponding operation label, a second loss value may be calculated based on a second result difference between a second operation prediction result and a corresponding operation label, and a third loss value may be calculated based on a third result difference between a third operation prediction result and a corresponding operation label. Further, model parameters may be adjusted according to the first loss value, the second loss value, and the third loss value.

[0102] In one possible implementation, the model parameters are adjusted based on the difference between the prediction result and the corresponding operation label, which can be achieved as follows:

[0103] A first loss value is calculated based on a first result difference between a first operation prediction result and a corresponding operation label, and a second loss value is calculated based on a second result difference between a second operation prediction result and a corresponding operation label; when it is determined that the first loss value is not higher than the second loss value, a distillation loss is calculated based on the result difference between the first operation prediction result and the second operation prediction result; and model parameters are adjusted based on a third loss value calculated by a third result difference between a third operation prediction result and the corresponding operation label, and the distillation loss.

[0104] In some embodiments, the training sample set may be constructed in the following manner:

[0105] Acquire a historical operation data set recording multimedia data of respective operations of each business object;

[0106] Select each sample object from each business object, and count the click frequencies corresponding to each multimedia data according to the historical operation data set, and select the corresponding sample multimedia data for each sample object. The click frequency can be determined according to the number of times the identification information of the multimedia data appears in the training sample, and the click frequency can be used to distinguish whether the multimedia data is in the cold start stage or the hot start stage.

[0107] For each sample object, the following operations are performed: the sample object is combined with the corresponding sample multimedia data to obtain corresponding training samples; a training sample includes: object description data of a sample object, click frequency, identification information and attribute information of a sample multimedia data, and an operation label determined according to the operation of the sample object on a multimedia data.

[0108] In some embodiments, after obtaining the object description data of the target object, a range of candidate multimedia data matching the target object can be determined according to the object description data of the target object. Further, in the range of candidate multimedia data, each candidate multimedia data corresponding to the target object can be selected according to the content type of multimedia data historically operated by the target object.

[0109] As an example, according to the object description data of the target object, the candidate multimedia range matching the target object is determined to be multimedia data of the food category and the sports category. The candidate multimedia data corresponds to different content types, and in the candidate multimedia data range, multimedia data of the food category and the sports category are selected as candidate multimedia data.

[0110] Optionally, in the embodiment of the present application, the multimedia data may be an advertisement. Below, the multimedia data recall method provided in the embodiment of the present application is described in detail by taking the multimedia data as an advertisement.

[0111] In the embodiment of the present application, the training of the target prediction model can be achieved in the following manner. For specific steps, see Figure 6 As shown, the details are as follows:

[0112] 601, determine a sample set.

[0113] In some embodiments, the sample set includes positive samples and negative samples, wherein the advertisements in the positive samples are those clicked by the sample subjects, and the advertisements in the negative samples are those obtained by random sampling, that is, the advertisements in the negative samples are those not clicked by the sample subjects.

[0114] 602, determine sample object characteristics.

[0115] In some embodiments, the sample advertisement corresponding to each sample in the sample set corresponds to sample object data, wherein the sample object data includes the image characteristics of the sample object (such as age, gender, etc.), and the behavioral characteristics of the sample object, such as obtaining the behavioral characteristics of the sample object's article reading records, advertisement click records, and advertisement conversion records within a set time period.

[0116] 603, obtaining advertisement fine-grained features.

[0117] In some embodiments, the fine-grained features of advertisements include identification information of advertisements. Advertisement data includes multiple types of data. Each advertisement corresponds to an identification information, namely, an advertisement ID, and the interaction record between the user and the advertisement is associated with the identification information ID of the advertisement.

[0118] 604, obtaining generalized features of the advertisement.

[0119] In some embodiments, the advertisement data also includes other data besides the advertisement ID, and features of the advertisement besides the advertisement ID are extracted as generalized features, such as advertisement type, advertisement category, and city where the advertisement is placed.

[0120] 605, determine the click frequency of the advertisement.

[0121] In some scenarios, the number of times the advertisement ID appears cumulatively in the training samples can be determined as the click frequency. In some embodiments, the click frequency corresponding to the advertisement ID can be used to determine whether the advertisement is in the cold start stage or the hot start stage. As an example, when the click frequency corresponding to the advertisement ID exceeds a set number of times, it can be determined that the advertisement has passed the cold start stage and is in the hot start stage.

[0122] 606, iteratively train the initial prediction model using the samples to obtain a target prediction model.

[0123] In some embodiments, the initial prediction model includes: an object feature extraction network, a global prediction network, and a data feature extraction network composed of a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork, and a gating subnetwork. As an example, the object feature extraction network can be called a user tower, the comprehensive feature extraction subnetwork can be called a memory tower, and the attribute feature extraction subnetwork can be called a generalization tower.

[0124] In some embodiments, the feature, memory feature, and generalization feature of the sample object can be mapped into a 64-dimensional vector through an embedding layer to map the ID type data field. For continuous features, such as the consumption amount of the sample object, the bucketing method can be used to discretize the features to obtain discrete ID features and map them into a 64-dimensional embedding vector.

[0125] Furthermore, the sample object data may include multiple features. After each of the multiple features included in the sample object data is word-embedded through the embedding layer, word embedding vectors corresponding to the multiple features can be obtained. Further, the multiple word embedding vectors can be connected, and the connected sample object data can be input into the user tower to obtain object features. As an example, the sample object data includes N object features. After the N object features are word-embedded to obtain N word embedding vectors, the N word embedding vectors can be spliced. As an example, the word embedding vector can be a 64-dimensional vector. After splicing the N word vectors, an N*64-dimensional long vector is obtained.

[0126] Furthermore, the N*64-dimensional long vector can be input into the user tower, and after passing through several layers of fully connected networks, it is finally normalized to obtain the object features. As an example, the normalization operation can be implemented by the L2 Normalization operation, and the obtained object features can be 64-dimensional features. The object features can be represented by user_embedding, such as Figure 7 shown.

[0127] In some embodiments, the advertising memory features and the advertising generalization features can be input into the memory tower. In some scenarios, the advertising features and the advertising generalization features can be word-embedded to obtain embedded word vectors respectively. Furthermore, the embedded word vectors corresponding to the advertising features and the advertising generalization features can be spliced ​​and input into the memory tower to obtain comprehensive features. As an example, the memory features and the generalization features include a total of M features. After the M features are word-embedded to obtain M word embedding vectors, the M word embedding vectors can be spliced. As an example, the word embedding vector can be a 64-dimensional vector. After splicing the M word vectors, an M*64-dimensional long vector is obtained.

[0128] Furthermore, the M*64-dimensional long vector can be input into the memory tower, and after passing through several layers of fully connected networks, the comprehensive features are output. As an example, the obtained comprehensive features can be 64-dimensional features, and the comprehensive features can be represented by item_warm_embedding, such as Figure 8 shown.

[0129] In some embodiments, the advertising memory feature and the advertising generalization feature can be input into the generalization tower. In some scenarios, the advertising feature and the advertising generalization feature can be word-embedded to obtain the embedded word vectors respectively. Furthermore, the embedded word vectors corresponding to the advertising generalization features can be spliced ​​and input into the generalization tower to obtain the attribute feature. As an example, the generalization feature includes a total of K features. After the K features are word-embedded to obtain K word embedding vectors, the K word embedding vectors can be spliced. As an example, the word embedding vector can be a 64-dimensional vector. After splicing the K word vectors, a K*64-dimensional long vector is obtained.

[0130] Furthermore, the K*64-dimensional long vector can be input into the generalization tower, and after passing through several layers of fully connected networks, the attribute features are output. As an example, the obtained attribute features can be 64-dimensional features, and the attribute features can be represented by item_cold_embedding, such as Fig. 9 shown.

[0131] In some embodiments, the click frequency of the advertisement can be input into the gated subnetwork, and after a single-layer full connection, the weight coefficient w is obtained through the logistics function. The weight coefficient w is used to weight the comprehensive features and attribute features to obtain multimedia data features, that is, advertisement features. As an example, the advertisement features meet the conditions described in the following formula:

[0132] item_embedding=L2_Normalization(w*item_warm_embedding+(1-w)*item_cold_embedding)

[0135] In some embodiments, after determining the advertisement features and the object features, the inner product of the advertisement features and the object features can be used as the prediction result, and the final loss function loss_main can be based on infoNCE. Specifically, the third operation prediction result of the sample object for the sample advertisement can be obtained based on the advertisement features and the object features through the global prediction network.

[0136] In some scenarios, since negative samples are randomly selected, a smaller temperature hyperparameter is often required in infoNCE to make the target prediction model pay more attention to difficult negative samples.

[0137] In some scenarios, since the generalization tower does not have the ad memory feature, underfitting problems are prone to occur. Therefore, during training, a dynamic distillation network can be used to distill the memory tower information into the generalization network.

[0138] In some embodiments, the initial prediction model further includes an integrated prediction sub-network connected to the integrated feature extraction sub-network. The integrated prediction sub-network can be used to obtain a first operation prediction result of the sample object for the sample advertisement based on the integrated features and object features, as Fig.10 shown. Similarly, the initial prediction model further includes an attribute prediction sub-network connected to the attribute feature extraction sub-network. The attribute prediction sub-network can be used to obtain a second operation prediction result of the sample object for the sample advertisement based on the attribute features and object features, as Fig.11 shown.

[0139] Furthermore, based on the first result difference between the first operation prediction result and the corresponding operation label, a first loss value is calculated, and based on the second result difference between the second operation prediction result and the corresponding operation label, a second loss value is calculated, and based on the third result difference between the third operation prediction result and the corresponding operation label, a third loss value is calculated. The model parameters are adjusted according to the first loss value, the second loss value, and the third loss value.

[0140] Specifically, assume that the operation label of the sample is yi, the first operation prediction result is ywi, and the second operation prediction result is yci. First, determine the first result difference between the first operation prediction result and the operation label, and calculate the first loss value. The first loss value is denoted as loss(yi, ywi). Then, determine the second result difference between the second operation prediction result and the operation label, and calculate the second loss value. The second loss value can be denoted as loss(yi, yci). In some scenarios, if loss(yi, ywi) < loss(yi, yci), indicating that the memory loss is less than the generalization loss, then the distillation loss loss1 = loss(ywi, yci); otherwise loss1 = 0.

[0141] Furthermore, it can be determined that the final loss is loss_main + alpha * loss1. Where alpha represents a preset hyperparameter. In some scenarios, alpha is usually less than 0.1. In the actual application process, this parameter needs to be tried multiple times to find a suitable hyperparameter value.

[0142] Based on the above process, the initial prediction model is iteratively trained to obtain a trained target prediction model.

[0143] In some embodiments, the amount of multimedia data faced by the online recall stage is huge, so the calculation of the recall stage can be completed with the help of the vector engine. Specifically, the feature extraction of multimedia data can be performed through the data feature extraction network in the target prediction model. Write item_embedding (item_embedding = L2_Normalization (w*item_warm_embedding + (1-w)*item_cold_embedding)) into the vector engine. When it is determined that multimedia data needs to be recalled for the target object, the object description data of the target object can be calculated in real time through the object feature extraction network in the trained target prediction model. After the object features are obtained, the inner product is performed with the multimedia data features in the vector engine to determine the target multimedia data to be recalled.

[0144] Based on the same technical concept, see Fig.12 As shown, Fig.12 An exemplary embodiment of the present application provides a multimedia data recall device 1200. The multimedia data recall device 1200 includes:

[0145] The first acquisition unit 1201 is used to acquire object description data of a target object;

[0146] The second acquisition unit 1202 is used to use the object feature extraction network in the trained target prediction model to perform weighted fusion on the corresponding comprehensive features obtained based on the identification information and the attribute information, and the corresponding attribute features obtained based on the attribute information, for each candidate multimedia data, using the weighted parameters obtained based on the click frequency, to obtain the corresponding multimedia data features; wherein the candidate multimedia data includes the click frequency, the identification information and the attribute information;

[0147] The determination unit 1203 is configured to determine at least one target multimedia data to be recalled for the target object from among the candidate multimedia data according to the feature similarity between the object feature and the features of the multimedia data.

[0148] In a possible implementation, the determination unit 1203 is further configured to train the target prediction model in the following manner:

[0149] Acquire a training sample set; a training sample includes: object description data of a sample object, a sample data set of a sample multimedia data, and an operation label of a sample object for a sample multimedia data; the sample data set includes identification information, attribute information and click frequency;

[0150] The training sample set is used to perform multiple rounds of iterative training on the constructed initial prediction model to obtain the trained target prediction model. In one round of iteration, the following operations are performed:

[0151] Using the initial prediction model, obtaining object features based on the object description data in the read training samples, and obtaining multimedia data features obtained by weighting the comprehensive features and attribute features based on the corresponding sample data set, and outputting prediction results based on the object features and the multimedia data features;

[0152] Based on the difference between the predicted results and the corresponding action labels, the model parameters are adjusted.

[0153] In a possible implementation, the initial prediction model includes: an object feature extraction network, a global prediction network, and a data feature extraction network composed of a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork, and a gating subnetwork; when the determination unit 1203 obtains multimedia data features obtained by weighting the comprehensive features and the attribute features based on the corresponding sample data set, it is specifically used to:

[0154] Based on the identification information, attribute information, and click frequency in the sample data set, information embedding processing is performed respectively to obtain corresponding identification embedding results, attribute embedding results, and frequency embedding results;

[0155] A comprehensive feature extraction sub-network is used to obtain corresponding comprehensive features based on the identity embedding results and the attribute embedding results, and an attribute feature extraction sub-network is used to obtain corresponding attribute features based on the attribute embedding results, and a gating sub-network is used to obtain weighting parameters based on the frequency embedding results;

[0156] The comprehensive features and attribute features are weighted and fused according to the weighting parameters to obtain the corresponding multimedia data features.

[0157] In a possible implementation, the initial prediction model includes: an object feature extraction network, a global prediction network, and a data feature extraction network composed of a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork and a gating subnetwork, a comprehensive prediction subnetwork connected to the comprehensive feature extraction subnetwork, and an attribute prediction subnetwork connected to the attribute feature extraction subnetwork;

[0158] The determining unit 1203, when outputting the prediction result based on the object feature and the multimedia data feature, is specifically used to:

[0159] Using a comprehensive prediction sub-network, based on comprehensive features and object features, obtain a first operation prediction result of the sample object on the sample multimedia data;

[0160] Using the attribute prediction subnetwork, based on the attribute features and the object features, obtain a second operation prediction result of the sample object on the sample multimedia data;

[0161] A global prediction network is used to obtain a third operation prediction result of the sample object on the sample multimedia data based on the multimedia features and the object features.

[0162] In a possible implementation, the determining unit 1203, when adjusting the model parameters based on the result difference between the prediction result and the corresponding operation label, is specifically configured to:

[0163] Calculating a first loss value based on a first result difference between a first operation prediction result and a corresponding operation label, calculating a second loss value based on a second result difference between a second operation prediction result and a corresponding operation label, and calculating a third loss value based on a third result difference between a third operation prediction result and a corresponding operation label;

[0164] Adjust the model parameters according to the first loss value, the second loss value, and the third loss value.

[0165] In a possible implementation, the determining unit 1203, when adjusting the model parameters based on the result difference between the prediction result and the corresponding operation label, is specifically configured to:

[0166] Calculating a first loss value based on a first result difference between the first operation prediction result and the corresponding operation label, and calculating a second loss value based on a second result difference between the second operation prediction result and the corresponding operation label;

[0167] Upon determining that the first loss value is not higher than the second loss value, calculating the distillation loss based on the result difference between the first operation prediction result and the second operation prediction result;

[0168] The model parameters are adjusted based on a third loss value calculated based on a third result difference between a third operation prediction result and a corresponding operation label, and a distillation loss.

[0169] In a possible implementation, the determining unit 1203 is further configured to construct a training sample set in the following manner:

[0170] Acquire a historical operation data set recording multimedia data of respective operations of each business object;

[0171] Select each sample object from each business object, and count the click frequencies corresponding to each multimedia data according to the historical operation data set, and select corresponding sample multimedia data for each sample object;

[0172] For each sample object, the following operations are performed: the sample object is combined with the corresponding sample multimedia data to obtain corresponding training samples; a training sample includes: object description data of a sample object, click frequency, identification information and attribute information of a sample multimedia data, and an operation label determined according to the operation of the sample object on a multimedia data.

[0173] In a possible implementation, the second determining unit 1202, after acquiring the object description data of the target object, acquires the data feature extraction network in the target prediction model based on the data sets corresponding to each of the candidate multimedia data, and before acquiring the data feature extraction network in the target prediction model, is further configured to:

[0174] Determining a range of candidate multimedia data matching the target object according to the object description data of the target object;

[0175] In the candidate multimedia data range, candidate multimedia data corresponding to the target object are selected according to the content type of the multimedia data historically operated by the target object.

[0176] For the convenience of description, the above parts are divided into units (or modules) according to their functions and described separately. Of course, when implementing this application, the functions of each unit (or module) can be implemented in the same or multiple software or hardware.

[0177] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0178] After introducing the multimedia data recall method and device according to the exemplary embodiment of the present application, next, an electronic device for multimedia data recall according to another exemplary embodiment of the present application is introduced.

[0179] Based on the same inventive concept as the above method embodiment of the present application, an electronic device is also provided in the embodiment of the present application, and the electronic device may be a server. In this embodiment, the structure of the electronic device may be as follows: Fig.13 As shown, it includes a memory 1301 and one or more processors 1302.

[0180] The memory 1301 is used to store computer programs executed by the processor 1302. The memory 1301 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0181] The memory 1301 may be a volatile memory, such as a random-access memory (RAM); the memory 1301 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1301 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1301 may be a combination of the above memories.

[0182] The processor 1302 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1302 is configured to implement the above multimedia data recall method when calling the computer program stored in the memory 1301 .

[0183] The specific connection medium between the memory 1301 and the processor 1302 is not limited in the embodiment of the present application. Fig.13 In the embodiment, the memory 1301 and the processor 1302 are connected via a bus 1303. The bus 1303 is connected to the processor 1302 via a bus 1303. Fig.13 The connections between the other components are only for illustration and are not intended to be limiting. The bus 1303 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.13 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0184] The memory 1301 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the multimedia data recall method of the embodiment of the present application. The processor 1302 is used to execute the above multimedia data recall method.

[0185] In some possible implementations, various aspects of the multimedia data recall method provided in the present application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the multimedia data recall method according to various exemplary implementations of the present application described above in this specification.

[0186] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0187] The program product of the embodiment of the present application can adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on a computing device. However, the program product of the present application is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, device or device.

[0188] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0189] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0190] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0191] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0192] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0193] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0194] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0196] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A multimedia data recall method, characterized in that: include: Get object description data of the target object; Using an object feature extraction network in a trained target prediction model to obtain corresponding object features based on the object description data; Using the data feature extraction network in the target prediction model, for each candidate multimedia data, weighted fusion is performed on the corresponding comprehensive features obtained based on the identification information and the attribute information, and the corresponding attribute features obtained based on the attribute information, respectively, by using the weighted parameters obtained based on the click frequency, so as to obtain the corresponding multimedia data features; wherein the candidate multimedia data includes the click frequency, the identification information and the attribute information; According to the feature similarity between the object feature and the features of each multimedia data, at least one target multimedia data to be recalled for the target object is determined from among the candidate multimedia data.

2. The method according to claim 1, characterized in that The target prediction model is trained in the following way: Acquire a training sample set; a training sample includes: object description data of a sample object, a sample data set of sample multimedia data, and an operation label of the sample object for the sample multimedia data; the sample data set includes identification information, attribute information and click frequency; The training sample set is used to perform multiple rounds of iterative training on the constructed initial prediction model to obtain a trained target prediction model, wherein in one round of iteration, the following operations are performed: Using the initial prediction model, obtaining object features based on the object description data in the read training sample, and obtaining multimedia data features obtained by weighting the comprehensive features and the attribute features based on the corresponding sample data set, and outputting a prediction result based on the object features and the multimedia data features; Based on the result difference between the prediction result and the corresponding operation label, the model parameters are adjusted.

3. The method according to claim 2, characterized in that The initial prediction model includes: an object feature extraction network, a global prediction network, and a data feature extraction network composed of a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork, and a gating subnetwork; the multimedia data features obtained by weighting the comprehensive features and the attribute features based on the corresponding sample data set include: Based on the identification information, attribute information, and click frequency in the sample data set, information embedding processing is performed respectively to obtain corresponding identification embedding results, attribute embedding results, and frequency embedding results; Using the comprehensive feature extraction sub-network, based on the identification embedding result and the attribute embedding result, obtain the corresponding comprehensive feature, using the attribute feature extraction sub-network, based on the attribute embedding result, obtain the corresponding attribute feature, and using the gating sub-network, based on the frequency embedding result, obtain the weighting parameter; The comprehensive features and the attribute features are weighted and fused according to the weighting parameters to obtain corresponding multimedia data features.

4. The method according to claim 2, characterized in that The initial prediction model includes: an object feature extraction network, a global prediction network, and a data feature extraction network composed of a comprehensive feature extraction subnetwork, an attribute feature extraction subnetwork and a gating subnetwork, a comprehensive prediction subnetwork connected to the comprehensive feature extraction subnetwork, and an attribute prediction subnetwork connected to the attribute feature extraction subnetwork; The outputting a prediction result based on the object feature and the multimedia data feature includes: Using the comprehensive prediction subnetwork, based on the comprehensive features and the object features, obtain a first operation prediction result of the sample object on the sample multimedia data; Using the attribute prediction subnetwork, based on the attribute feature and the object feature, obtain a second operation prediction result of the sample object on the sample multimedia data; The global prediction network is used to obtain a third operation prediction result of the sample object on the sample multimedia data based on the multimedia features and the object features.

5. The method according to claim 4, characterized in that The adjusting the model parameters based on the result difference between the prediction result and the corresponding operation label includes: Calculating a first loss value based on a first result difference between the first operation prediction result and the corresponding operation label, calculating a second loss value based on a second result difference between the second operation prediction result and the corresponding operation label, and calculating a third loss value based on a third result difference between the third operation prediction result and the corresponding operation label; Adjust model parameters according to the first loss value, the second loss value and the third loss value.

6. The method according to claim 4, characterized in that The adjusting the model parameters based on the result difference between the prediction result and the corresponding operation label includes: Calculating a first loss value based on a first result difference between the first operation prediction result and the corresponding operation label, and calculating a second loss value based on a second result difference between the second operation prediction result and the corresponding operation label; When it is determined that the first loss value is not higher than the second loss value, calculating a distillation loss based on a result difference between the first operation prediction result and the second operation prediction result; The model parameters are adjusted based on a third loss value calculated based on a third result difference between the third operation prediction result and the corresponding operation label, and the distillation loss.

7. The method according to any one of claims 2 to 6, characterized in that: The training sample set is constructed in the following way: Acquire a historical operation data set recording multimedia data of respective operations of each business object; Selecting each sample object from each business object, and counting the click frequency corresponding to each multimedia data according to the historical operation data set, and selecting corresponding sample multimedia data for each sample object; For each sample object, the following operations are performed: the sample object is respectively combined with corresponding sample multimedia data to obtain corresponding training samples; A training sample includes: object description data of a sample object, click frequency, identification information and attribute information of a sample multimedia data, and an operation label determined according to the operation of the sample object on the multimedia data.

8. The method according to any one of claims 1 to 6, characterized in that: After obtaining the object description data of the target object, the method further includes: Determining a range of candidate multimedia data matching the target object according to the object description data of the target object; In the candidate multimedia data range, candidate multimedia data corresponding to the target object are selected according to the content type of the multimedia data historically operated by the target object.

9. A multimedia data recall device, characterized in that: include: A first acquisition unit, used to acquire object description data of a target object; The second acquisition unit uses the object feature extraction network in the trained target prediction model to perform weighted fusion on the corresponding comprehensive features obtained based on the identification information and the attribute information, and the corresponding attribute features obtained based on the attribute information, for each candidate multimedia data, using a weighted parameter obtained based on the click frequency, to obtain a corresponding multimedia data feature; wherein the candidate multimedia data includes the click frequency, the identification information, and the attribute information; A determination unit is used to determine at least one target multimedia data to be recalled for the target object from among the candidate multimedia data according to the feature similarity between the object feature and the features of the multimedia data.

10. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer instructions; The processor is used to obtain computer instructions stored in the memory and execute the method according to any one of claims 1 to 8 according to the computer instructions.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images