Method and apparatus for managing machine learning model on basis of distillation, device and medium

Abstract

Provided are a method and apparatus for managing a machine learning model on the basis of distillation, a device and a medium. The method comprises: using a first reference sample associated with a first reference object in an application to determine a first machine learning model, the first reference sample comprising a first reference classification for a first reference media item specified by the first reference object; using the first machine learning model to determine a prediction for a second reference classification of a second reference object in the application for a second reference media item; on the basis of the second reference object, the second reference media item and the prediction for the second reference classification, generating a second reference sample; and using the second reference sample to determine a second machine learning model, the second machine learning model describing an association relationship between objects in the application and classifications of the objects for media items. In this way, more reference samples can be obtained from sparse reference samples, thereby improving the accuracy of machine learning models while reducing interference with the normal use of applications.

Description

Methods, apparatus, devices, and media for managing machine learning models based on distillation. Technical Field

[0001] Implementations of this disclosure generally relate to the field of computers, and in particular to methods, apparatus, devices, and computer-readable storage media for managing machine learning models based on distillation. Background Technology

[0002] Machine learning techniques have been widely used to perform various tasks. For example, in recommendation scenarios, machine learning models (e.g., recommendation models) can be used to recommend various media items to users within an application. To improve recommendation accuracy, users can be asked questions to determine if they like the recommended media items. However, too many questions can negatively impact the application's usability, resulting in a small number of collected samples and making it difficult to use these sparse samples to update the machine learning model. Therefore, it is desirable to address the problem of insufficient sample size and update the machine learning model in a more accurate manner. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for managing a machine learning model is provided. In this method, a first machine learning model is determined using first reference samples associated with a first reference object in an application. The first reference samples include a first reference classification specified by the first reference object for a first reference media item. The first machine learning model describes the association between a first object provided with the first media item and the first classification of the first object for the first media item. Using the first machine learning model, a prediction of a second reference classification for a second reference media item is determined for a second reference object in the application. Based on the predictions of the second reference object, the second reference media item, and the second reference classification, a second reference sample is generated. Using the second reference sample, a second machine learning model is determined. The second machine learning model describes the association between objects in the application and the classification of objects for media items.

[0004] In a second aspect of this disclosure, an apparatus for managing machine learning models is provided. The apparatus includes: a first determining module configured to determine a first machine learning model using first reference samples associated with a first reference object in an application, the first reference samples including a first reference classification for a first reference media item specified by the first reference object, the first machine learning model describing the association between a first object provided with the first media item and the first classification of the first object for the first media item; a prediction module configured to determine a prediction of a second reference classification for a second reference object in the application using the first machine learning model; a generating module configured to generate a second reference sample based on the prediction of the second reference object, the second reference media item, and the second reference classification; and a second determining module configured to determine a second machine learning model using the second reference sample, the second machine learning model describing the association between objects in the application and the classification of objects for media items.

[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to a first aspect of this disclosure when executed by the at least one processing unit.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to implement the method according to a first aspect of this disclosure.

[0007] In a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to a first aspect of this disclosure.

[0008] It should be understood that the content described in this content section is not intended to limit the key or essential features of the implementation of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] In the following detailed description, the above and other features, advantages, and aspects of the various implementations of this disclosure will become more apparent, taken in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 shows a block diagram of an application environment according to one implementation of the present disclosure;

[0011] Figure 2 shows a block diagram for managing a machine learning model according to some implementations of this disclosure;

[0012] Figure 3A shows a block diagram illustrating the structure of the feature space of a first machine learning model according to some implementations of this disclosure;

[0013] Figure 3B shows a block diagram illustrating the structure of the feature space of a second machine learning model according to some implementations of this disclosure;

[0014] Figure 4 shows a block diagram illustrating the structure of a machine learning model according to some implementations of this disclosure;

[0015] Figure 5 shows a block diagram of a process for managing a machine learning model based on distillation, according to some implementations of this disclosure;

[0016] Figure 6 shows a flowchart of a method for managing a machine learning model according to some implementations of this disclosure;

[0017] Figure 7 shows a block diagram of an apparatus for managing a machine learning model according to some implementations of this disclosure; and

[0018] Figure 8 shows a block diagram of a device capable of implementing various implementations of the present disclosure. Detailed Implementation

[0019] Implementations of this disclosure will now be described in more detail with reference to the accompanying drawings. While some implementations of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the implementations set forth herein. Rather, these implementations are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and implementations of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] In the description of the implementation methods disclosed herein, the term "comprising" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one implementation" or "the implementation" should be understood as "at least one implementation". The term "some implementations" should be understood as "at least some implementations". Other explicit and implicit definitions may also be included below. As used herein, the term "model" can represent the relationships between various data. For example, the aforementioned relationships can be obtained based on various currently known and / or future-developed technical solutions.

[0021] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0022] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0023] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0024] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0025] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0026] The term "in response to" as used herein refers to a state in which a corresponding event occurs or a condition is satisfied. It will be understood that the timing of subsequent actions performed in response to such event or condition is not necessarily strongly correlated with the time when the event occurs or the condition is met. For example, in some cases, subsequent actions may be performed immediately upon the occurrence of the event or the fulfillment of the condition; while in others, they may be performed some time after the occurrence of the event or the fulfillment of the condition.

[0027] Example Environment

[0028] Machine learning techniques have been widely used to perform a variety of tasks. For example, in recommendation scenarios, machine learning models (e.g., recommendation models) can be used to recommend various media items to objects in an application. Referring to Figure 1, which illustrates a block diagram 100 of an application environment according to one implementation of this disclosure, media items 120 can be provided to objects (e.g., users of application 110) in application 110. Media items 120 can include various types, such as, but not limited to, videos, short videos, music, text, images, games, or rich media data including combinations of the above types. For ease of description, video will be used as an example of a media item in the context of this disclosure.

[0029] To improve the accuracy of recommendations, questionnaires can be used to present questions to the target audience. For example, the target audience can be asked to categorize the recommended media items, or to express their liking for the recommended media items. A questionnaire page 130 can be provided in application 110, where questions (e.g., asking about the categorization of the viewed media items) can be presented to various users of application 110, and responses to the questions can be received from the users.

[0030] Page 130 may include a control 138 for rejecting a response, and the page can be canceled in response to receiving an interaction request with control 138. Page 130 may further include a control 136 for submitting a response, and may include one or more predefined categories. For example, control 132 corresponds to "Category 1", ..., and control 134 corresponds to "Category N". An object can select a desired category and press control 136 to submit the selected category. Responses from the object can be collected and used to learn the association between the object and the media item, thereby improving the accuracy of recommendations.

[0031] However, too many questionnaires can hinder the normal use of the application, resulting in a small number of collected samples and making it difficult to use these sparse samples to update the machine learning model. Therefore, it is desirable to address the problem of insufficient sample size and update the machine learning model in a more accurate manner.

[0032] Overview of the distillation process

[0033] To at least partially address the shortcomings of the prior art, a method for managing machine learning models is proposed according to one implementation of this disclosure. Referring to Figure 2, which illustrates a block diagram 200 for managing machine learning models according to some implementations of this disclosure, a summary of one implementation of this disclosure is provided. As shown in Figure 2, a machine learning model 210 can be trained using collected sparse samples (e.g., reference sample 211, etc.), and the machine learning model 220 can be updated using more samples generated by the machine learning model 210 (e.g., reference sample 221, etc.).

[0034] Specifically, a first machine learning model (e.g., machine learning model 210) can be determined using a first reference sample (e.g., reference sample 211) associated with a first reference object in the application. The first reference sample may include a first reference classification (e.g., reference classification 214) specified by the first reference object (e.g., reference object 212) for a first reference media item (e.g., reference media item 213). Here, the first machine learning model describes the association between a first object provided with the first media item and a first classification of the first object for the first media item. In other words, a questionnaire can be provided to a small number of objects in the application and classifications can be collected, and the machine learning model 210 can be trained using a small number of samples.

[0035] A trained first machine learning model can be used to determine a prediction (e.g., a prediction of the reference classification 224) of a second reference object (e.g., reference object 222) in the application for a second reference media item (e.g., reference media item). Based on the predictions of the second reference object, the second reference media item, and the second reference classification, a second reference sample (e.g., reference sample 221) is generated. Here, the prediction of the second reference classification is a soft label; although not true ground truth data, this prediction is generated by the trained machine learning model 210, thus the accuracy of the reference classification prediction 224 is high, and it can be used as training data. Further, a second machine learning model (e.g., machine learning model 220) can be determined using the second reference sample. Here, the second machine learning model can describe the relationship between objects in the application and the classification of objects for media items.

[0036] By utilizing some implementation methods disclosed herein, a small number of reference samples can be obtained with minimal interference to application usage. In this way, more reference samples can be obtained from sparse reference samples, improving the accuracy of the machine learning model while reducing interference with normal application operation.

[0037] Detailed process of distillation

[0038] Having outlined some implementations of this disclosure, further details regarding methods for managing machine learning models will be described below. According to some implementations of this disclosure, historical data can be used to train machine learning model 210. For example, a first reference sample (also called a training sample) can be obtained. A questionnaire can be provided in the application to obtain a first reference classification. Specifically, the application can provide a first reference media item and a first reference question associated with the first reference media item. The first reference classification is determined based on the first reference response submitted by the first reference object to the first reference question.

[0039] For the example in Figure 1, the responses submitted by objects can be determined; for example, objects can interact with controls 132, ..., and 134. Assuming an object presses control 132 and clicks control 136, the corresponding response can be determined to be "Category 1". Reference samples can be constructed based on the determined categories, and machine learning model 210 can be trained using multiple collected reference samples. Using some implementations of this disclosure, questionnaires can be provided and categories collected only to a small number of objects in the application (e.g., one ten-thousandth or a different proportion of the number of objects in the application). In this way, it can be ensured that the vast majority of objects in the application can use the application normally. During application operation, questionnaires related to the provided media items and the responses of each object to the questionnaires can be collected.

[0040] According to some implementations of this disclosure, the first reference sample may further include object information of the first reference object and media information of the first reference media item. It should be understood that the object information may include various aspects, such as, but not limited to, the object's identifier, object-related device information (e.g., operating system type, model, etc.), etc. The media information may include various aspects, such as, but not limited to, the media item's identifier, the media item's duration, the media item's content, etc. The object information, media information, and reference classification can be mapped to a feature space, and the machine learning model 210 can be used to learn the relationships between the three.

[0041] According to some implementations of this disclosure, the collected samples can be mapped to a feature space. The first machine learning model and the second machine learning model can have different feature spaces. More information is provided below with reference to Figures 3A and 3B. Figure 3A shows a block diagram 300A illustrating the structure of the feature space of the first machine learning model according to some implementations of this disclosure. As shown in Figure 3A, the feature space 310 can include features corresponding to object information 311 and media information 312, respectively. An encoder can be used to transform the collected object information 311 and media information 312 into corresponding object features and media features, respectively.

[0042] According to some implementations of this disclosure, the first sample may further include first additional information, which includes at least one of the following: a first object embedding associated with the first reference object, a first media embedding associated with the first reference media item, first environmental information associated with the first reference object and the first reference media item, and a first posterior label associated with the first reference classification. As shown in the dashed box portion of Figure 3A, the feature space 310 may further include features corresponding to the embedding information 313, the context information 314, and the posterior label 315, respectively.

[0043] Here, embedding information 313 can represent object embeddings related to the object and media embeddings related to the media, determined by the encoder of the recommendation model. For example, object embeddings and media embeddings can be directly concatenated to determine embedding features corresponding to embedding information 313. Context information 314 can represent, for example, context information about the object browsing media items, such as the start time of browsing, the duration of browsing, the date of browsing, etc. A corresponding encoder can be used to transform context information 314 into context features. Posterior information 315 can represent the posterior probability associated with the object's submitted response, and a corresponding encoder can be used to transform posterior information 315 into posterior features.

[0044] According to some implementation methods of this disclosure, the above-mentioned multiple features can be concatenated to generate the final features for input into the first machine learning model. Utilizing some implementation methods of this disclosure, factors that may affect classification can be described from multiple perspectives, thereby improving the accuracy of the first machine learning model.

[0045] According to some implementations of this disclosure, the final features described above can be input into the first machine learning model, and the first machine learning model can be updated. Specifically, in determining the first machine learning model, the first machine learning model can determine a first prediction for the first reference classification based on the object information of the first reference object and the media information of the first reference media item. The first machine learning model can be updated based on the first difference between the first reference classification and the first prediction.

[0046] Alternatively and / or additionally, first additional information may be further considered. Specifically, in the process of determining the first prediction of the first reference category, the first machine learning model may further determine the first prediction of the first reference category based on the first additional information. In other words, the first prediction of the first reference category may be determined based on the object information of the first reference object, the media information of the first reference media item, and the first additional information. Utilizing some implementations of this disclosure, the powerful learning capabilities of machine learning models can be leveraged to acquire knowledge about the classification based on historical data of whether a small number of objects have submitted responses.

[0047] Referring to Figure 4, which illustrates a block diagram 400 of the structure of a machine learning model according to some implementations of this disclosure, the machine learning model may include multiple inputs for inputting object information 310, media information 312, embedding information 313, context information 314, and posterior information 315, respectively. This information may be encoded into features in a feature space and input to a shared network layer 420. Here, the dimension of the shared network layer 420 may be, for example, 256, and alternatively and / or additionally, a self-attention module 422 may be applied to the shared network layer 420.

[0048] The machine learning model can have branches corresponding to multiple categories, for example, branch 431 corresponds to category 1, branch 432 to category 2, branch 433 to category 3, ..., branch 434 to category N. Here, a category can represent a negative evaluation of a media item (e.g., negative categories such as boring, outdated, etc.). For example, category 1 can indicate that a media item belongs to negative category 1, category 2 can indicate that a media item belongs to negative category 2, and so on. It should be understood that most applications rely too heavily on predictive models with positive feedback, which often leads to over-optimization of short-term goals while ignoring long-term effects. For example, users may show short-term positive feedback to some novel media items, but if such videos appear frequently, it may lead to a decrease in recommendation accuracy. According to some implementations of this disclosure, the recommendation model is adjusted through negative feedback. During application operation, questionnaires can be proactively provided to a smaller number of people to collect more comprehensive negative feedback. The collected negative feedback can be used to train the machine learning model, and the machine learning model can be used to adjust the output of the recommendation model, thereby reducing negative feedback.

[0049] As shown in Figure 4, each branch can include multiple network layers. For example, branch 431 can include multiple network layers (with dimensions of 64, 16, and 1, respectively). The corresponding loss can be determined based on the output of each branch. For example, features can be input into the machine learning model, and the prediction of the reference class can be determined. The loss can be determined using the difference between the collected ground truth of the reference class and the prediction of the reference class. Each class can correspond to a loss; for example, class 1 corresponds to loss 441, class 2 corresponds to loss 442, class 3 corresponds to loss 443, ..., class N corresponds to loss 444. Furthermore, the individual losses can be weighted and summed to determine the final loss 450. It should be understood that although Figure 4 shows N branches, alternatively and / or additionally, the machine learning model may include only one branch.

[0050] It should be understood that although the process of updating the first machine learning model has only been described above using a single first reference object as an example, alternatively and / or additionally, the first reference object may include multiple first reference objects. That is, multiple first reference objects may be objects provided with a questionnaire. According to some implementations of this disclosure, the first machine learning model can be trained using multiple collected first reference samples so that the first machine learning model can accurately represent the association between a first object provided with a first media item and the first classification of the first object for the first media item. Alternatively and / or additionally, the first machine learning model can be used to determine the prediction of a second reference classification for a second reference object in the application for a second reference media item. In this way, the first machine learning model can be used to generate more samples from sparse samples, thereby determining the second machine learning model.

[0051] According to some implementations of this disclosure, the second reference question associated with the second reference media item is not provided to the second reference object. In other words, the second reference object is an object in the application that is not provided with a questionnaire. According to some implementations of this disclosure, the second reference object includes multiple second reference objects, and the second number of the multiple second reference objects is greater than the first number of the multiple first reference objects. In this way, interference with the normal use of the application can be reduced, and a large number of training samples can be obtained without providing a questionnaire to every object in the application.

[0052] Referring back to Figure 3B, which illustrates further details regarding the second machine learning model, this figure shows a block diagram 300B of the structure of the feature space of the second machine learning model according to some implementations of this disclosure. The feature space 320 of the second machine learning model can be similar to the feature space 310 of the first machine learning model. The difference is that, since no questionnaire is provided to the second reference object, the feature space 320 does not include posterior information. As shown in Figure 3B, the feature space 320 may include object features corresponding to object information 321 and media features corresponding to media information 322, respectively.

[0053] According to some implementations of this disclosure, in the process of determining the prediction of the second reference classification using the first machine learning model, the prediction of the second reference classification can be determined by the first machine learning model based on the second object information of the second reference object and the second media information of the second reference media item. It should be understood that the prediction of the second reference classification can be used as a soft label in the training samples, and a second reference sample can be generated using the second object information, the second media information, and the soft label, and the second machine learning model can be trained using the second reference sample.

[0054] According to some implementations of this disclosure, the second reference sample may further include second additional information, which includes at least one of the following: a second object embedding associated with the second reference object, a second media embedding associated with the second reference media item, and second environmental information associated with the second reference object and the second reference media item. As shown in FIG3B, the feature space 320 may further include embedding information 323, for example, representing the object embedding of the object and the media embedding of the media determined by the encoder of the recommendation model. For example, the object embedding and the media embedding can be directly concatenated to determine the embedding features corresponding to the embedding information 323.

[0055] According to some implementations of this disclosure, in the process of determining the prediction of the second reference classification, the first machine learning model can determine the prediction of the second reference classification based on the second additional information. In other words, the second object information, the second media information, and the second additional information can be input into the first machine learning model to determine the prediction of the second reference classification in a more accurate manner. Utilizing some implementations of this disclosure, the accuracy of the prediction of the second reference classification can be improved by leveraging various details in the additional information, thereby providing more accurate training data for the subsequent training process. Furthermore, the accuracy of the second machine learning model can be improved.

[0056] According to some implementations of this disclosure, in the process of determining the second machine learning model using the second reference sample, the second machine learning model can determine additional predictions for the second classification based on the second object information and the second media information; and update the second machine learning model based on the difference between the second classification prediction and the additional predictions for the second classification. It should be understood that although the second classification prediction is not the actual ground truth label collected, it is generated using a trained first machine learning model, and therefore can accurately reflect the second classification of the second object for the second media item to a certain extent. A corresponding loss function can be generated based on the difference, and the second machine learning model can be updated in the direction that minimizes this loss function.

[0057] According to some implementations of this disclosure, in determining the additional prediction for the second category, a second machine learning model can determine the additional prediction for the second reference category based on the second additional information. In other words, the additional prediction for the second reference category can be determined based on object information 321, media information 322, embedding information 323, and context information 324. In this way, more factors associated with the object and media can be fully considered in determining the additional prediction, thereby improving the accuracy of the second machine learning model.

[0058] According to some implementations of this disclosure, the second machine learning model can have a structure similar to that shown in Figure 4. The difference lies in that the data input to the second machine learning model does not include the posterior information 315 as shown in the dashed box. Furthermore, the input information for the first machine learning model at the multiple input ends in Figure 4 (e.g., input object information 310, media information 312, embedding information 313, and context information 314) can be replaced with input information for the second machine learning model (e.g., input object information 320, media information 322, embedding information 323, and context information 324). The second machine learning model can be trained in a similar manner; specifically, the final loss can be determined using the loss output at each branch, and the second machine learning model can be updated in the direction that minimizes this final loss.

[0059] The steps for determining the first and second machine learning models have been described separately. Hereinafter, the overall process for determining the second machine learning model is described with reference to Figure 5. According to some implementations of this disclosure, the machine learning model can be determined based on a knowledge distillation process. Figure 5 shows a block diagram 500 of a process for managing machine learning models based on distillation, according to some implementations of this disclosure. As shown in Figure 5, teacher model 510 corresponds to the first machine learning model, and student model 520 corresponds to the second machine learning model. Knowledge in teacher model 510 can be transferred to student model 520 based on a knowledge distillation process.

[0060] Specifically, multiple samples 512 can be obtained, representing samples determined through a questionnaire. Specifically, a questionnaire can be provided to a subset of the users in the application (e.g., one in ten thousand, or another proportion), asking them to categorize the media item they just viewed. Here, samples 512 can include positive samples (denoted by P) and negative samples (denoted by N). Feedback in positive samples can be negative (e.g., believing the media item belongs to a certain type of inappropriate video), and feedback in negative samples can be positive (e.g., believing the media item does not belong to a certain type of inappropriate video).

[0061] The teacher model 510 can be trained using media information 311, object information 312, embedding information 313, contextual information 314, and posterior information 315 associated with multiple samples 512. During training, the loss function can be defined according to Formula 1:

[0062] In Formula 1, i represents the i-th object in the application, j represents the j-th media item in the application, and This represents the classification of the i-th object for the j-th media item (from ground truth data, and denoted by the superscript T). CE() represents the cross-entropy loss. The teacher model 510 can be trained using Equation 1 and sample 512 to obtain the trained teacher model 510.

[0063] Multiple samples 522 can be generated using the teacher model 510 without requiring questionnaires to be provided to each object in the application. In other words, the teacher model 510 can be used to predict an object's classification of a media item. In this way, soft labels for multiple samples can be determined, and corresponding positive and negative samples can be generated. Then, the soft labels in the multiple samples 522 can be filtered using a filter 530. It should be understood that here, the soft label represents the probability that the object considers the media item to belong to the negative category, and therefore ranges from [0,1]. Labels close to 0 and labels close to 1 can be filtered from a large number of soft labels, while soft labels close to the middle value can be ignored. Specifically, only soft labels in the range of [0,0.2] and [0.8,1] can be retained, and corresponding negative and positive samples can be generated. In this way, the accuracy of the training samples can be improved, thereby improving the accuracy of the machine learning model obtained using the training samples.

[0064] Specifically, it can be determined whether soft labels are used as training data based on the following formula 2.

[0065] According to some implementations of this disclosure, the student model 520 (corresponding to the second machine learning model) can be trained using the training data 1 output from the filter 530. For example, the student model 520 can be trained using media information 321, object information 322, embedding information 323, and context information 324 associated with multiple samples 522. During training, the loss function can be defined according to Formula 3:

[0066] In Formula 3, i represents the i-th object in the application, j represents the j-th media item in the application, and Let S represent the prediction (soft label, denoted by the superscript S) of the classification of the i-th object for the j-th media item. CE() represents the cross-entropy loss. Student model 520 can be trained using Equation 2 and sample 522 to obtain the trained student model 520.

[0067] According to some implementations of this disclosure, the second machine learning model can be further updated using the first reference sample. The loss function can be defined based on Equation 4. In Equation 4, the loss function used to update the student model can be determined based on a weighted sum of the losses shown in Equations 1 and 3. For example, α can represent the weight that adjusts the ratio between the two losses.

[0068] According to some implementations of this disclosure, a second machine learning model can be used to recommend media items. For example, the second machine learning model can determine a prediction of the target category of a target object for a target media item; and based on the prediction of the target category, provide the target media item to the target object. Suppose it is desired to recommend media items to an object, object information and media information can be input into the second machine learning model, and the object's negative evaluation of the media item can be determined. Media items with lower negative evaluations can be preferentially recommended to the object.

[0069] Alternatively and / or additionally, a second machine learning model can be combined with an existing recommendation model. For example, the recommendation model can determine the initial recommendation metrics associated with objects and media items. Further, a final recommendation metric can be determined based on the initial recommendation metric and negative reviews output by the recommendation prediction module, and then a media item can be recommended to the object based on the final recommendation metric. For example, the final recommendation metric can be determined based on a weighted sum of the initial recommendation metric and negative reviews. Utilizing some implementations of this disclosure, in the process of recommending media items, on the one hand, recommendation metrics determined based on existing technical solutions can be considered, and on the other hand, negative reviews determined based on a small number of questionnaires can be considered, thereby recommending media items to the object in a more accurate manner.

[0070] By utilizing some implementation methods disclosed herein, a small number of reference samples can be obtained with minimal interference to application usage. In this way, more reference samples can be obtained from sparse reference samples, improving the accuracy of the machine learning model while reducing interference with normal application operation.

[0071] Example process

[0072] Figure 6 illustrates a flowchart of a method 600 for managing a machine learning model according to some implementations of this disclosure. At block 610, a first machine learning model is determined using a first reference sample associated with a first reference object in the application. The first reference sample includes a first reference classification specified by the first reference object for a first reference media item. The first machine learning model describes the association between a first object provided with the first media item and the first classification of the first object for the first media item. At block 620, using the first machine learning model, a prediction of a second reference classification for a second reference object in the application for a second reference media item is determined. At block 630, a second reference sample is generated based on the predictions of the second reference object, the second reference media item, and the second reference classification. At block 640, a second machine learning model is determined using the second reference sample. The second machine learning model describes the association between objects in the application and the classification of objects for media items.

[0073] According to some implementations of this disclosure, the first reference sample further includes object information of the first reference object and media information of the first reference media item, and determining the first machine learning model includes: determining a first prediction of the first reference classification based on the object information of the first reference object and the media information of the first reference media item by the first machine learning model; and updating the first machine learning model based on the first difference between the first reference classification and the first prediction.

[0074] According to some implementations of this disclosure, the first sample further includes first additional information, which includes at least one of the following: a first object embedding associated with the first reference object, a first media embedding associated with the first reference media item, first environmental information associated with the first reference object and the first reference media item, and a first posterior label associated with the first reference classification. The first prediction for determining the first reference classification further includes: determining the first prediction for the first reference classification by a first machine learning model based on the first additional information.

[0075] According to some implementations of this disclosure, the first reference classification is obtained based on the following: in the application, providing a first reference media item and a first reference question associated with the first reference media item to a first reference object; and determining the first reference classification based on the first reference response submitted by the first reference object to the first reference question.

[0076] According to some implementations of this disclosure, determining the prediction of the second reference classification using the first machine learning model includes: determining the prediction of the second reference classification by the first machine learning model based on the second object information of the second reference object and the second media information of the second reference media item.

[0077] According to some implementations of this disclosure, the second reference sample further includes second additional information, which includes at least one of the following: a second object embedding associated with the second reference object, a second media embedding associated with the second reference media item, second environmental information associated with the second reference object and the second reference media item, and the prediction for determining the second reference classification further includes: a prediction for determining the second reference classification by a first machine learning model based on the second additional information.

[0078] According to some implementations of this disclosure, determining a second machine learning model using a second reference sample includes: determining additional predictions for a second classification based on second object information and second media information using the second machine learning model; and updating the second machine learning model based on the difference between the predictions for the second classification and the additional predictions for the second classification.

[0079] According to some implementations of this disclosure, determining the second machine learning model using the second reference sample further includes: updating the second machine learning model using the first reference sample.

[0080] According to some implementations of this disclosure, the second reference sample further includes second additional information, which includes at least one of the following: a second object embedding associated with the second reference object, a second media embedding associated with the second reference media item, second environmental information associated with the second reference object and the second reference media item, and the additional prediction for determining the second classification further includes: determining the additional prediction for the second reference classification by a second machine learning model based on the second additional information.

[0081] According to some implementations of this disclosure, the first reference object includes multiple first reference objects, the second reference object includes multiple second reference objects, and the second number of the multiple second reference objects is greater than the first number of the multiple first reference objects.

[0082] According to some implementations of this disclosure, the second reference issue associated with the second reference media item is not provided to the second reference object, and the classification represents negative evaluations of the media item.

[0083] According to some implementations of this disclosure, the method further includes: determining a prediction of the target object's target classification for the target media item using a second machine learning model; and providing the target media item to the target object based on the prediction of the target classification.

[0084] Example devices and equipment

[0085] Figure 7 shows a block diagram of an apparatus 700 for managing a machine learning model according to some implementations of the present disclosure. The apparatus 700 includes: a first determining module configured to determine a first machine learning model using first reference samples associated with a first reference object in an application, the first reference samples including a first reference classification for a first reference media item specified by the first reference object, the first machine learning model describing the association between a first object provided with the first media item and the first classification of the first object for the first media item; a prediction module configured to determine a prediction of a second reference classification for a second reference object in the application using the first machine learning model; a generating module configured to generate a second reference sample based on the prediction of the second reference object, the second reference media item, and the second reference classification; and a second determining module configured to determine a second machine learning model using the second reference sample, the second machine learning model describing the association between objects in the application and the classification of objects for media items.

[0086] According to some implementations of this disclosure, the first reference sample further includes object information of the first reference object, media information of the first reference media item, and the first determining module is further configured to: determine a first prediction of the first reference classification by the first machine learning model based on the object information of the first reference object and the media information of the first reference media item; and update the first machine learning model based on a first difference between the first reference classification and the first prediction.

[0087] According to some implementations of this disclosure, the first sample further includes first additional information, which includes at least one of the following: a first object embedding associated with the first reference object, a first media embedding associated with the first reference media item, first environmental information associated with the first reference object and the first reference media item, and a first posterior label associated with the first reference classification. The first determining module is further configured to: determine the first prediction of the first reference classification by the first machine learning model based on the first additional information.

[0088] According to some implementations of this disclosure, the first reference classification is obtained based on the following: in the application, providing the first reference media item and a first reference question associated with the first reference media item to the first reference object; and determining the first reference classification based on a first reference response submitted by the first reference object to the first reference question.

[0089] According to some implementations of this disclosure, the prediction module is further configured to: determine the prediction of the second reference classification by the first machine learning model based on the second object information of the second reference object and the second media information of the second reference media item.

[0090] According to some implementations of this disclosure, the second reference sample further includes second additional information, which includes at least one of the following: a second object embedding associated with the second reference object, a second media embedding associated with the second reference media item, second environmental information associated with the second reference object and the second reference media item, and the prediction module is further configured to: determine the prediction of the second reference classification by the first machine learning model based on the second additional information.

[0091] According to some implementations of this disclosure, the second determining module is further configured to: determine additional predictions for the second classification based on the second object information and the second media information by the second machine learning model; and update the second machine learning model based on the difference between the predictions for the second classification and the additional predictions for the second classification.

[0092] According to some implementations of this disclosure, the second determining module is further configured to: update the second machine learning model using the first reference sample.

[0093] According to some implementations of this disclosure, the second reference sample further includes second additional information, which includes at least one of the following: a second object embedding associated with the second reference object, a second media embedding associated with the second reference media item, second environmental information associated with the second reference object and the second reference media item, and the second determining module is further configured to: determine the additional prediction of the second reference classification by the second machine learning model based on the second additional information.

[0094] According to some implementations of this disclosure, the first reference object includes a plurality of first reference objects, the second reference object includes a plurality of second reference objects, and the second number of the plurality of second reference objects is greater than the first number of the plurality of first reference objects.

[0095] According to some implementations of this disclosure, a second reference issue associated with the second reference media item is not provided to the second reference object, and the classification represents a negative evaluation of the media item.

[0096] According to some implementations of this disclosure, the apparatus further includes a processing module configured to: determine a prediction of a target category for a target media item by the second machine learning model; and provide the target media item to the target object based on the prediction of the target category.

[0097] Figure 7 shows a block diagram of a device 700 capable of implementing various implementations of the present disclosure. It should be understood that the computing device 700 shown in Figure 7 is merely exemplary and should not constitute any limitation on the functionality and scope of the implementations described herein. The computing device 700 shown in Figure 7 can be used to implement the methods described above.

[0098] As shown in Figure 7, the computing device 700 is in the form of a general-purpose computing device. Components of the computing device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage devices 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processing unit 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in the memory 720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the computing device 700.

[0099] Computing device 700 typically includes multiple computer storage media. Such media can be any available media accessible to computing device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 730 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data (e.g., training data for training) and accessible within computing device 700.

[0100] The computing device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 7, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. The memory 720 may include a computer program product 725 having one or more program modules configured to perform various methods or actions of various implementations of the present disclosure.

[0101] The communication unit 740 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 700 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 700 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0102] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 700 can also communicate as needed with one or more external devices (not shown) via communication unit 740. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with computing device 700, or with any device (e.g., network card, modem, etc.) that enables computing device 700 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interface (not shown).

[0103] According to an implementation of this disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the method described above. According to an implementation of this disclosure, a computer program product is provided, on which a computer program is stored, which, when executed by a processor, implements the method described above.

[0104] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0105] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0106] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0108] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for managing machine learning models, comprising: A first machine learning model is determined using a first reference sample associated with a first reference object in the application, the first reference sample including a first reference classification for a first reference media item specified by the first reference object, and the first machine learning model describes the association between a first object provided with a first media item and the first classification of the first object for the first media item. Using the first machine learning model, determine the prediction of the second reference object in the application for the second reference media item's second reference category; Based on the predictions made by the second reference object, the second reference media item, and the second reference classification, a second reference sample is generated; as well as A second machine learning model is determined using the second reference sample. The second machine learning model describes the relationship between objects in the application and the classification of those objects for media items.

2. The method according to claim 1, wherein the first reference sample further includes object information of the first reference object, media information of the first reference media item, and determining the first machine learning model includes: The first machine learning model determines a first prediction for the first reference category based on the object information of the first reference object and the media information of the first reference media item. as well as The first machine learning model is updated based on the first difference between the first reference classification and the first prediction.

3. The method of claim 2, wherein the first sample further comprises first additional information, the first additional information comprising at least one of the following: a first object embedding associated with the first reference object, a first media embedding associated with the first reference media item, first environmental information associated with the first reference object and the first reference media item, and a first posterior label associated with the first reference classification. The first prediction for determining the first reference classification further includes: The first prediction of the first reference classification is determined by the first machine learning model based on the first additional information.

4. The method of claim 1, wherein the first reference classification is obtained based on: In the application, the first reference media item and a first reference question associated with the first reference media item are provided to the first reference object; and The first reference category is determined based on the first reference response submitted by the first reference object to the first reference question.

5. The method of claim 1, wherein determining the prediction of the second reference classification using the first machine learning model comprises: The prediction for the second reference classification is determined by the first machine learning model based on the second object information of the second reference object and the second media information of the second reference media item.

6. The method of claim 5, wherein the second reference sample further includes second additional information, the second additional information including at least one of the following: a second object embedding associated with the second reference object, a second media embedding associated with the second reference media item, second environmental information associated with the second reference object and the second reference media item, and the prediction for determining the second reference classification further includes: The prediction for the second reference classification is determined by the first machine learning model based on the second additional information.

7. The method of claim 6, wherein determining the second machine learning model using the second reference sample comprises: The second machine learning model determines additional predictions for the second classification based on the second object information and the second media information. as well as The second machine learning model is updated based on the difference between the prediction of the second category and the additional prediction of the second category.

8. The method of claim 1, wherein determining the second machine learning model using the second reference sample further comprises: The second machine learning model is updated using the first reference sample.

9. The method of claim 7, wherein the second reference sample further includes second additional information, the second additional information including at least one of the following: a second object embedding associated with the second reference object, a second media embedding associated with the second reference media item, second environmental information associated with the second reference object and the second reference media item, and the additional prediction for determining the second classification further includes: The second machine learning model determines the additional predictions for the second reference classification based on the second additional information.

10. The method of claim 1, wherein the first reference object includes a plurality of first reference objects, the second reference object includes a plurality of second reference objects, and the second number of the plurality of second reference objects is greater than the first number of the plurality of first reference objects.

11. The method of claim 1, wherein a second reference issue associated with the second reference media item is not provided to the second reference object, and the classification represents a negative evaluation of the media item.

12. The method of claim 1, further comprising: The second machine learning model determines the prediction of the target object's target category for the target media item; as well as Based on the prediction of the target classification, the target media item is provided to the target object.

13. An apparatus for managing a machine learning model, comprising: A first determining module is configured to determine a first machine learning model using a first reference sample associated with a first reference object in the application, the first reference sample including a first reference classification for a first reference media item specified by the first reference object, the first machine learning model describing the association between a first object provided with the first media item and the first classification of the first object for the first media item. The prediction module is configured to use the first machine learning model to determine a prediction of the second reference object in the application for the second reference media item's second reference category; A generation module is configured to generate a second reference sample based on the predictions of the second reference object, the second reference media item, and the second reference classification. as well as The second determining module is configured to determine a second machine learning model using the second reference sample, the second machine learning model describing the relationship between objects in the application and the objects' classification for media items.

14. An electronic device comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 12 when executed by the at least one processing unit.

15. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 12.

16. A computer instruction product comprising computer instructions, wherein the computer instructions, when executed by a processor, implement the method according to any one of claims 1 to 12.

Images