Method and apparatus for managing machine learning model, device, and medium
By updating the machine learning model by predicting the probability of an object submitting a response and correcting the weights, the bias problem in the training samples is solved, the model's consideration of objects that refuse to submit a response is improved, and the recommendation accuracy is enhanced.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- BEIJING ZITIAO NETWORK TECH CO LTD
- Filing Date
- 2025-01-20
- Publication Date
- 2026-07-23
AI Technical Summary
Existing machine learning models suffer from bias in their training samples when dealing with objects that refuse to submit responses, causing the recommendation model to ignore the opinions of objects that refuse to submit responses and affecting the accuracy of recommendations.
By using a first machine learning model to predict the probability of an object submitting a response, determining correction weights, and updating a second machine learning model based on reference responses and correction weights, the association between the object and the media item is described, and data bias in the training samples is corrected.
This increases the importance of training samples for objects that refuse to submit responses in the machine learning model, enhances the model's consideration of objects with different submission probabilities, and improves the accuracy and comprehensiveness of the recommendation model.
Smart Images
Figure CN2025073467_23072026_PF_FP_ABST
Abstract
Description
Methods, apparatus, devices, and media for managing machine learning models Technical Field
[0001] Implementations of this disclosure generally relate to the field of computers, and in particular to methods, apparatus, devices, and computer-readable storage media for managing machine learning models. Background Technology
[0002] Machine learning techniques have been widely used to perform various tasks. For example, in recommendation scenarios, machine learning models (e.g., recommendation models) can be used to recommend various media items to users in an application. To improve the accuracy of recommendations, users can be presented with questions asking whether they like the recommended media items. Some users may agree to answer the questions and submit responses, while others may refuse to answer. In this case, the collected responses only reflect the opinions of those who agreed to submit responses, but not those who refused to submit responses. If the recommendation model is updated based on the collected responses, it may cause the model to ignore users who refused to submit responses. Therefore, it is desirable to reduce the bias in the training samples and update the machine learning model in a more accurate manner. Summary of the Invention
[0003] In a first aspect of this disclosure, a method for managing a machine learning model is provided. In this method, a first machine learning model is used to determine predictions of submission events between an object and a media item provided in an application, the predictions of which represent the probability that the object will submit a response to a question associated with the media item. Based on the predictions of the submission events, correction weights associated with the object are determined. A reference media item and a reference question associated with the reference media item are provided to the object in the application. In response to receiving a reference response submitted by the object for the reference question, a second machine learning model is updated based on the reference response and the correction weights, the second machine learning model describing the association between the object and the reference media item.
[0004] In a second aspect of this disclosure, an apparatus for managing a machine learning model is provided. The apparatus includes: a prediction determination module configured to determine, using a first machine learning model, a prediction of a submission event between an object and a media item provided in an application, the prediction of the submission event representing the probability that the object submits a response to a question associated with the media item; a weight determination module configured to determine a correction weight associated with the object based on the prediction of the submission event; a providing module configured to provide the object with a reference media item and a reference question associated with the reference media item in the application; and an updating module configured to, in response to receiving a reference response submitted by the object for the reference question, update a second machine learning model based on the reference response and the correction weight, the second machine learning model describing the association between the object and the reference media item.
[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to a first aspect of this disclosure when executed by the at least one processing unit.
[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to implement the method according to a first aspect of this disclosure.
[0007] In a fifth aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, implements the method according to a first aspect of this disclosure.
[0008] It should be understood that the content described in this content section is not intended to limit the key or essential features of the implementation of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0009] In the following detailed description, the above and other features, advantages, and aspects of the various implementations of this disclosure will become more apparent, taken in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein:
[0010] Figure 1 shows a block diagram of an application environment according to one implementation of the present disclosure;
[0011] Figure 2 shows a block diagram for managing a machine learning model according to some implementations of this disclosure;
[0012] Figure 3 shows a block diagram of the process of updating a machine learning model according to some implementations of this disclosure;
[0013] Figure 4 shows a block diagram illustrating the problems of some implementations according to this disclosure;
[0014] Figure 5 shows a flowchart of a method for managing a machine learning model according to some implementations of this disclosure;
[0015] Figure 6 shows a block diagram of an apparatus for managing a machine learning model according to some implementations of the present disclosure; and
[0016] Figure 7 shows a block diagram of a device capable of implementing various implementations of the present disclosure. Detailed Implementation
[0017] Implementations of this disclosure will now be described in more detail with reference to the accompanying drawings. While some implementations of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the implementations set forth herein. Rather, these implementations are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and implementations of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0018] In the description of the implementation methods disclosed herein, the term "comprising" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one implementation" or "the implementation" should be understood as "at least one implementation". The term "some implementations" should be understood as "at least some implementations". Other explicit and implicit definitions may also be included below. As used herein, the term "model" can represent the relationships between various data. For example, the aforementioned relationships can be obtained based on various currently known and / or future-developed technical solutions.
[0019] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.
[0020] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.
[0021] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.
[0022] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.
[0023] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.
[0024] The term "in response to" as used herein refers to a state in which a corresponding event occurs or a condition is satisfied. It will be understood that the timing of subsequent actions performed in response to such event or condition is not necessarily strongly correlated with the time when the event occurs or the condition is met. For example, in some cases, subsequent actions may be performed immediately upon the occurrence of the event or the fulfillment of the condition; while in others, they may be performed some time after the occurrence of the event or the fulfillment of the condition.
[0025] Example Environment
[0026] Machine learning techniques have been widely used to perform a variety of tasks. For example, in recommendation scenarios, machine learning models (e.g., recommendation models) can be used to recommend various media items to objects in an application. Referring to Figure 1, which illustrates a block diagram 100 of an application environment according to one implementation of this disclosure, media items 120 can be provided to objects (e.g., users of application 110) in application 110. Media items 120 can include various types, such as, but not limited to, videos, short videos, music, text, images, games, or rich media data including combinations of the above types. For ease of description, video will be used as an example of a media item in the context of this disclosure.
[0027] To improve the accuracy of recommendations, questionnaires can be used to present questions to the users. For example, questions could be asked whether they liked the recommended media items, or they could be asked to categorize the recommended media items. Different users may respond differently to media item 120. For instance, some users might like media item 120, watch all media items, and potentially perform actions such as liking, commenting, or sharing. Alternatively and / or additionally, some users might not like media item 120 and might skip it and browse to the next media item. To further optimize the performance of the recommendation model, a questionnaire page 130 can be provided in application 110. For example, questions can be presented to various users of application 110 (e.g., how did you feel about the video you just viewed?), and responses to question 130 can be received from the users.
[0028] Page 130 includes a control 134 for rejecting a response, and the page can be canceled in response to receiving an interaction request with control 134. Page 130 may further include a control 132 for submitting a response, and may include one or more predefined responses. For example, control 140 corresponds to the positive response "I like it", control 142 corresponds to the neutral response "I neither like nor dislike it", and control 144 corresponds to the negative response "I don't like it". An object can select a desired response and press control 132 to submit the selected response. Responses from objects can be collected and used to learn the association between the object and the media item (e.g., the object's level of liking for the media item), thereby improving the accuracy of recommendations.
[0029] However, as shown in Figure 1, some users agree to answer questions and submit responses, while others refuse to answer. In this case, the collected responses only reflect the opinions of those who agree to submit responses, but not those who refuse. If the recommendation model is updated based on these collected responses, it may cause the model to ignore users who refuse to submit responses. Therefore, it is desirable to reduce the bias in the training samples and update the machine learning model in a more accurate manner.
[0030] Overview of Managing Machine Learning Models
[0031] To at least partially address the shortcomings of the prior art, a method for managing machine learning models is proposed according to one implementation of this disclosure. This method can eliminate bias in training data, thereby improving the accuracy of the machine learning model. Referring to Figure 2, which describes an overview of one implementation of this disclosure, Figure 2 shows a block diagram 200 for managing a machine learning model according to some implementations of this disclosure. As shown in Figure 2, a prediction 240 of a submission event between an object 230 and a media item 232 provided in the application can be determined using a first machine learning model (e.g., machine learning model 210). The prediction of the submission event can represent the probability that the object 230 will submit a response to a question associated with the media item 232.
[0032] The question shown on page 130 can be presented to object 230, asking whether the object likes the media item 120 that was just viewed. Here, machine learning model 210 can be a model used to predict whether the object will answer the question, and prediction 240 can represent the probability (e.g., between 0 and 1) that the object will answer the question. Machine learning model 210 can be trained using historical data samples; for example, if the object was presented with one question every day for the past 30 days but only received one response, then prediction 240 could be represented as a probability of "1 / 30". Alternatively and / or additionally, whether the object submits a response can further depend on relevant information about the media item; for example, if the media item is music, more responses may be collected; if the media item is sports, fewer responses may be collected. In this case, prediction 240 further depends on the specific information of the media item.
[0033] A correction weight 250 associated with object 230 can be determined based on the prediction 240 of the submitted event. In the application, a reference media item 234 and a reference question 260 associated with the reference media item 234 can be provided to object 230. In response to receiving a reference response 262 submitted by object 230 for the reference question 260, a second machine learning model (e.g., machine learning model 220) can be updated based on the reference response 262 and the correction weight 250. The second machine learning model can describe the association between object 230 and reference media item 234. It should be understood that here, machine learning model 220 can represent the degree of liking of reference media item 234 by object 230, which can represent a recommendation metric for recommending reference media item 234 to object 230. The higher the degree of liking, the higher the probability of recommending the reference media item 234 to object 230.
[0034] Using the implementation method of this disclosure, data bias in training samples can be corrected based on the probability of object submission and response, thereby increasing the importance of training samples corresponding to objects with lower submission probabilities. In this way, training data corresponding to different objects with different submission probabilities can be considered more comprehensively and accurately, thereby improving the accuracy of the trained machine learning model.
[0035] Detailed process of managing machine learning models
[0036] Having described an overview of some implementations according to this disclosure, further details regarding methods for managing machine learning models will be described below. Figure 3 shows a block diagram 300 of a process for updating a machine learning model according to some implementations of this disclosure. As shown in Figure 3, multiple modules can be used to implement the technical solutions described above. According to some implementations of this disclosure, the submission prediction module 310 can be used to predict whether an object will submit a response to a received question. In other words, relevant information about an object can be input into the submission prediction module 310, and the submission prediction module 310 can output the submission probability of that object. Alternatively and / or additionally, relevant information about media items can be further input into the submission prediction module 310, in which case the submission prediction module 310 can operate in a more granular manner and output the submission probability of that object for questions associated with the media item.
[0037] According to some implementations of this disclosure, historical data can be used to train the machine learning model 210. For example, a first reference sample (also called a training sample) can be obtained. The first reference sample includes first object information of a first reference object, first media information of a first reference media item, and a first reference submission event between the first reference object and the first reference media item, wherein the first reference media item is provided to the first reference object; and the first machine learning model is updated based on the first reference sample. Using some implementations of this disclosure, the powerful learning capability of the machine learning model can be utilized to obtain knowledge about the submission probability based on historical data of whether an object submits a response. During application operation, questionnaires related to the provided media items and the responses of each object to the questionnaires can be collected.
[0038] In the process of acquiring the first reference sample, a first reference media item and a first reference question associated with the first reference media item can be provided to the first reference object; and based on the first reference response submitted by the first reference object to the first reference question, a first reference submission event in the first reference sample can be determined. For the example in Figure 1, assuming an interaction request is received for control 132 (i.e., a click on the submit control), response 311 can be determined to indicate a "positive" submission event, and a positive sample is constructed. Assuming an interaction request is received for control 134 (i.e., a click on the cancel control), response 311 can be determined to indicate a "negative" submission event, and a negative sample is constructed.
[0039] It should be understood that object information 312 may include various aspects, such as, but not limited to, the object's identifier, object-related device information (e.g., operating system type, model, etc.), etc. Media information may include various aspects, such as, but not limited to, the media item's identifier, the media item's duration, the media item's content, etc. Object information, media information, and submission events can be mapped to a feature space, and machine learning model 210 can be used to learn the relationships between the three.
[0040] According to some implementations of this disclosure, a machine learning model 210 can be used to output a predicted probability 313, also known as a submission probability. The submission probability can be represented as P(submit|show), where submit represents the event of an object submitting a response, and show represents the event of providing a question to the object in the application. Therefore, P(submit|show) represents the probability that, given that the object has received a question, it will submit a response to that question. A probabilistic model can be constructed using the machine learning model 210 to output the submission probability P(submit|show).
[0041] According to some implementations of this disclosure, in the process of updating the first machine learning model based on the first reference sample, the first machine learning model can determine a first prediction of the first reference submission event based on the first object information and the first media information; and update the first machine learning model based on the first difference between the first reference submission event and the first prediction. Specifically, an initial machine learning model 210 can be obtained, a loss function can be constructed based on the first difference, and the machine learning model 210 can be trained in the direction that minimizes the loss function. According to some implementations of this disclosure, a large number of reference samples can be generated based on a large amount of historical data from a large number of users. Furthermore, the machine learning model 210 can be continuously updated iteratively. In this way, the machine learning model 210 can continuously accumulate knowledge about the submission probability, thereby improving the accuracy of the machine learning model 210.
[0042] According to some implementations of this disclosure, after determining the prediction of a submission event (i.e., the submission probability), the corresponding correction weight can be determined based on the submission probability. Specifically, the correction weight can decrease as the prediction of the submission event increases, for example, it can be inversely proportional to the prediction of the submission event. Continuing to refer to Figure 3, the correction weight 250 can be determined by the sample correction module 320. Assuming the submission probability is represented as P(submit|show), the correction weight can be represented as W = 1 / P(submit|show). The submission probability 313 (i.e., P(submit|show)) output by the machine learning model 210 can be used to correct the recommendation prediction model. Specifically, each training sample can be multiplied by the correction weight W = 1 / P(submit|show), and then the sample with the correction weight can be used for training. It should be understood that the formulas here are merely illustrative, and alternatively and / or additionally, other formulas can be used to determine the correction weights, as long as the correction weights decrease as the submission probability increases.
[0043] According to some implementations of this disclosure, the second machine learning model can be updated based on a reference response and correction weights. Specifically, a second reference sample can be determined based on the reference response, the second reference sample including second object information of the object, second media information of the reference media item, and the reference response; and the second machine learning model can be updated based on the second reference sample and correction weights. Here, the second reference sample refers to the training sample used to update the machine learning model 220. Specifically, in the liking prediction module 330, the predicted probability 334 of liking / disliking can be determined.
[0044] According to some implementations of this disclosure, the first object information may differ from the second object information, and the first media information may differ from the second media information. Specifically, there may be overlap between the first object information and the second object information, and there may be overlap between the first media information and the second media information. Using some implementations of this disclosure, the dimensions of relevant features can be selected based on the respective concerns of the first and second machine learning models, thereby improving the accuracy of each machine learning model.
[0045] In the liking prediction module 330, the response 331 to the question can be obtained, that is, the object's response (like / dislike) to the question associated with the media item. Training samples can be constructed based on object information 332, media information 333, and response 331 to train the machine learning model 220. Here, object information 332 can include various aspects, such as, but not limited to, the object's identifier, object-related device information (e.g., operating system type, model, etc.), the time when the object received the question, etc. Media information 333 can include various aspects, such as, but not limited to, the media item's identifier, the media item's duration, the media's resolution, the media's category, etc. Object information 332, media information 333, and response 331 can be mapped to a feature space, and the machine learning model 220 can be used to learn the relationship between the three.
[0046] According to some implementations of this disclosure, in the process of updating the second machine learning model based on the second reference sample and correction weights, the second machine learning model can determine a second prediction of the reference response based on the second object information and the second media information; and update the second machine learning model based on the second difference between the reference response and the second prediction and the correction weights. For example, object information 332 and media information 333 can be input into machine learning model 220, and a second prediction from machine learning model 222 can be received, allowing a second difference between the second prediction and the true value in response 331 to be determined. Then, machine learning model 220 is updated based on the second difference and correction weights W = 1 / P(submit|show). Using some implementations of this disclosure, since the submission probability is less than or equal to 1, the correction weights are greater than or equal to 1. In this way, the influence of the response corresponding to the object with the lower submission probability can be strengthened, thereby enabling machine learning model 220 to consider the perspective of the object corresponding to the lower submission probability more.
[0047] According to some implementations of this disclosure, in the process of updating the second machine learning model based on the second difference and the correction weights, the loss used to update the second machine learning model can be determined based on the product of the second difference and the correction weights; and the second machine learning model is updated based on the loss. In the context of this disclosure, the second difference represents the degree of influence of the response corresponding to the object with the lower submission probability on the parameters of the machine learning model. Determining the loss based on the product of the second difference and the correction weights can strengthen the influence of the response corresponding to the object with the lower submission probability, thereby enabling the machine learning model 220 to consider the perspective of the object corresponding to the lower submission probability more.
[0048] By utilizing some implementations of this disclosure, data bias in training samples can be corrected based on the probability of object submission and response, thereby increasing the importance of training samples corresponding to objects with lower submission probabilities. In this way, training data corresponding to different objects with different submission probabilities can be considered more comprehensively and accurately, thereby improving the accuracy of the trained machine learning model.
[0049] According to some implementations of this disclosure, the response may include at least one of the following: an evaluation of the media item specified by the object; or a category of the media item specified by the object. See Figure 4 for further details, which shows a block diagram 400 of a question according to some implementations of this disclosure. As shown in Figure 4, page 410 shows a question asking whether the object liked the video it just viewed. Control 412 corresponds to a positive evaluation "I like it," and control 414 corresponds to a negative evaluation "I don't like it." Page 420 shows a question asking about the category of the media item. Control 422 corresponds to category 1 (e.g., low-quality videos) for which the recommendation frequency is expected to be reduced, and control 424 corresponds to category N for which the recommendation frequency is expected to be reduced. Using some implementations of this disclosure, the specific content of the question can be tailored to the query objective in order to gather more information from the response that helps improve recommendation accuracy.
[0050] The methods described above can be used in applications to recommend media items to users. Experimental data shows that when the question concerns liking levels, using the above method increases the level of liking reported in the questionnaire. When the question concerns media item categories for which a reduced recommendation frequency is desired, using the above method reduces the proportion of low-quality videos reported in the questionnaire.
[0051] According to some implementations of this disclosure, the second machine learning model described above can be used to select media items to recommend to an object. Specifically, the second machine learning model can select a target media item from multiple media items based on second object information; and provide the target media item to the object. Assuming that it is desired to recommend media items to an object, object features and media features can be input into machine learning model 220, and it can be determined whether the object likes the media item. Media items with higher liking can be preferentially recommended to the object.
[0052] Alternatively and / or additionally, the machine learning model 220 can be combined with existing recommendation models. For example, the recommendation model can determine the initial recommendation metrics associated with objects and media items. Further, a final recommendation metric can be determined based on the initial recommendation metric and the liking level output by the recommendation prediction module, and then a media item can be recommended to the object based on the final recommendation metric. For example, the final recommendation metric can be determined based on a weighted sum of the initial recommendation metric and the liking level. Utilizing some implementations of this disclosure, in the process of recommending media items, on the one hand, recommendation metrics determined based on existing technical solutions can be considered, and on the other hand, a corrected liking level determined based on a questionnaire can be considered, thereby recommending media items to the object in a more accurate manner.
[0053] Using the implementation method of this disclosure, data bias in training samples can be corrected based on the probability of object submission and response, thereby increasing the importance of training samples corresponding to objects with lower submission probabilities. In this way, training data corresponding to different objects with different submission probabilities can be considered more comprehensively and accurately, thereby improving the accuracy of the trained machine learning model.
[0054] Example process
[0055] Figure 5 illustrates a flowchart of a method 500 for managing a machine learning model according to some implementations of this disclosure. At box 510, a first machine learning model is used to determine a prediction of a submission event between an object and a media item provided in the application, the prediction representing the probability that the object will submit a response to a question associated with the media item. At box 520, a correction weight associated with the object is determined based on the prediction of the submission event. At box 530, a reference media item and a reference question associated with the reference media item are provided to the object in the application. At box 540, in response to receiving a reference response submitted by the object for the reference question, a second machine learning model is updated based on the reference response and the correction weight, the second machine learning model describing the association between the object and the reference media item.
[0056] According to some implementations of this disclosure, the first machine learning model is determined based on the following: obtaining a first reference sample, the first reference sample including first object information of a first reference object, first media information of a first reference media item, and a first reference submission event between the first reference object and the first reference media item, the first reference media item being provided to the first reference object; and updating the first machine learning model based on the first reference sample.
[0057] According to some implementations of this disclosure, obtaining the first reference sample includes: providing a first reference media item and a first reference question associated with the first reference media item to a first reference object; and determining a first reference submission event in the first reference sample based on a first reference response submitted by the first reference object to the first reference question.
[0058] According to some implementations of this disclosure, updating the first machine learning model based on the first reference sample includes: determining a first prediction of the first reference submission event by the first machine learning model based on the first object information and the first media information; and updating the first machine learning model based on the first difference between the first reference submission event and the first prediction.
[0059] According to some implementations of this disclosure, the correction weight decreases as the prediction of the submitted event increases.
[0060] According to some implementations of this disclosure, updating the second machine learning model based on the reference response and correction weights includes: determining a second reference sample based on the reference response, the second reference sample including second object information of the object, second media information of the reference media item, and the reference response; and updating the second machine learning model based on the second reference sample and correction weights.
[0061] According to some implementations of this disclosure, updating the second machine learning model based on the second reference sample and correction weights includes: determining a second prediction of the reference response by the second machine learning model based on the second object information and the second media information; and updating the second machine learning model based on the second difference between the reference response and the second prediction and the correction weights.
[0062] According to some implementations of this disclosure, updating the second machine learning model based on the second difference and the correction weights includes: determining a loss for updating the second machine learning model based on the product of the second difference and the correction weights; and updating the second machine learning model based on the loss.
[0063] According to some implementations of this disclosure, the first object information is different from the second object information, and the first media information is different from the second media information.
[0064] According to some implementations of this disclosure, the response includes at least one of the following: an evaluation of the media item specified by the object; or a classification of the media item specified by the object.
[0065] According to some implementations of this disclosure, the method further includes: selecting a target media item from multiple media items based on second object information by a second machine learning model; and providing the target media item to the object.
[0066] Example devices and equipment
[0067] Figure 6 shows a block diagram of an apparatus 600 for managing a machine learning model according to some implementations of the present disclosure. The apparatus includes: a prediction determination module 610 configured to determine, using a first machine learning model, a prediction of a submission event between an object and a media item provided in an application, the prediction of the submission event representing the probability that the object submits a response to a question associated with the media item; a weight determination module 620 configured to determine a correction weight associated with the object based on the prediction of the submission event; a provision module 630 configured to provide the object with a reference media item and a reference question associated with the reference media item in the application; and an update module 640 configured to, in response to receiving a reference response submitted by the object for the reference question, update a second machine learning model based on the reference response and the correction weight, the second machine learning model describing the association between the object and the reference media item.
[0068] According to some implementations of this disclosure, the first machine learning model is determined based on the following: obtaining a first reference sample, the first reference sample including first object information of a first reference object, first media information of a first reference media item, and a first reference submission event between the first reference object and the first reference media item, the first reference media item being provided to the first reference object; and updating the first machine learning model based on the first reference sample.
[0069] According to some implementations of this disclosure, obtaining the first reference sample includes: providing a first reference media item and a first reference question associated with the first reference media item to a first reference object; and determining a first reference submission event in the first reference sample based on a first reference response submitted by the first reference object to the first reference question.
[0070] According to some implementations of this disclosure, the update module is further configured to: determine a first prediction of the first reference submission event by the first machine learning model based on the first object information and the first media information; and update the first machine learning model based on the first difference between the first reference submission event and the first prediction.
[0071] According to some implementations of this disclosure, the correction weight decreases as the prediction of the submitted event increases.
[0072] According to some implementations of this disclosure, the update module is further configured to include: determining a second reference sample based on a reference response, the second reference sample including second object information of the object, second media information of the reference media item, and a reference response; and updating a second machine learning model based on the second reference sample and correction weights.
[0073] According to some implementations of this disclosure, the update module is further configured to: determine a second prediction of the reference response by the second machine learning model based on the second object information and the second media information; and update the second machine learning model based on the second difference between the reference response and the second prediction and the correction weights.
[0074] According to some implementations of this disclosure, the module is further configured to: determine a loss for updating the second machine learning model based on the product of the second difference and the correction weights; and update the second machine learning model based on the loss.
[0075] According to some implementations of this disclosure, the first object information is different from the second object information, and the first media information is different from the second media information.
[0076] According to some implementations of this disclosure, the response includes at least one of the following: an evaluation of the media item specified by the object; or a classification of the media item specified by the object.
[0077] According to some implementations of this disclosure, it further includes a processing module configured to: select a target media item from multiple media items based on second object information by a second machine learning model; and provide the target media item to the object.
[0078] Figure 7 shows a block diagram of a device 700 capable of implementing various implementations of the present disclosure. It should be understood that the computing device 700 shown in Figure 7 is merely exemplary and should not constitute any limitation on the functionality and scope of the implementations described herein. The computing device 700 shown in Figure 7 can be used to implement the methods described above.
[0079] As shown in Figure 7, the computing device 700 is in the form of a general-purpose computing device. Components of the computing device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage devices 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processing unit 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in the memory 720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the computing device 700.
[0080] Computing device 700 typically includes multiple computer storage media. Such media can be any available media accessible to computing device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 730 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data (e.g., training data for training) and accessible within computing device 700.
[0081] The computing device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 7, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. The memory 720 may include a computer program product 725 having one or more program modules configured to perform various methods or actions of various implementations of the present disclosure.
[0082] The communication unit 740 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 700 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 700 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.
[0083] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 700 can also communicate as needed with one or more external devices (not shown) via communication unit 740. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with computing device 700, or with any device (e.g., network card, modem, etc.) that enables computing device 700 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interface (not shown).
[0084] According to an implementation of this disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the method described above. According to an implementation of this disclosure, a computer program product is provided, on which a computer program is stored, which, when executed by a processor, implements the method described above.
[0085] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0086] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0087] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0089] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.
Claims
1. A method for managing machine learning models, comprising: A first machine learning model is used to determine a prediction of a submission event between an object and a media item provided in the application, the prediction of the submission event representing the probability that the object submits a response to a question associated with the media item; Based on the prediction of the submission event, determine the correction weight associated with the object; The application provides the object with reference media items and reference questions associated with the reference media items; as well as In response to receiving a reference response submitted by the object for the reference question, a second machine learning model is updated based on the reference response and the correction weights, the second machine learning model describing the association between the object and the reference media item.
2. The method of claim 1, wherein the first machine learning model is determined based on: Obtain a first reference sample, the first reference sample including first object information of a first reference object, first media information of a first reference media item, and a first reference submission event between the first reference object and the first reference media item, wherein the first reference media item is provided to the first reference object; and The first machine learning model is updated based on the first reference sample.
3. The method according to claim 2, wherein obtaining the first reference sample comprises: Provide the first reference media item and the first reference issue associated with the first reference media item to the first reference object; as well as Based on the first reference response submitted by the first reference object to the first reference question, the first reference submission event in the first reference sample is determined.
4. The method of claim 2, wherein updating the first machine learning model based on the first reference sample comprises: The first prediction of the first reference submission event is determined by the first machine learning model based on the first object information and the first media information; as well as The first machine learning model is updated based on the first difference between the first reference submission event and the first prediction.
5. The method of claim 1, wherein the correction weight decreases as the prediction of the submission event increases.
6. The method of claim 1, wherein updating the second machine learning model based on the reference response and the correction weights comprises: A second reference sample is determined based on the reference response. The second reference sample includes the second object information of the object, the second media information of the reference media item, and the reference response. as well as The second machine learning model is updated based on the second reference sample and the correction weights.
7. The method of claim 6, wherein updating the second machine learning model based on the second reference sample and the correction weights comprises: The second prediction of the reference response is determined by the second machine learning model based on the second object information and the second media information; as well as The second machine learning model is updated based on the second difference between the reference response and the second prediction, and the correction weights.
8. The method of claim 7, wherein updating the second machine learning model based on the second difference and the correction weights comprises: Based on the product of the second difference and the correction weight, the loss used to update the second machine learning model is determined; as well as The second machine learning model is updated based on the loss.
9. The method according to claim 1, wherein the first object information is different from the second object information, and the first media information is different from the second media information.
10. The method of claim 1, wherein the response comprises at least one of the following: The evaluation of the media item specified by the object; or The category for the media item specified by the object.
11. The method of claim 1, further comprising: The second machine learning model selects the target media item from multiple media items based on the second object information; as well as Provide the target media item to the object.
12. An apparatus for managing a machine learning model, comprising: A prediction determination module is configured to use a first machine learning model to determine a prediction of a submission event between an object and a media item provided in the application, the prediction of the submission event representing the probability that the object submits a response to a question associated with the media item; A weight determination module is configured to determine a correction weight associated with the object based on the prediction of the submission event; A module is provided, configured to provide the object with a reference media item and a reference question associated with the reference media item in the application; An update module is configured to update a second machine learning model based on the reference response and the correction weights in response to receiving a reference response submitted by the object for the reference question. The second machine learning model describes the association between the object and the reference media item.
13. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 11.
14. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 11.
15. A computer instruction product comprising computer instructions, wherein the computer instructions, when executed by a processor, implement the method according to any one of claims 1 to 11.