Method and apparatus for selective learning of media samples, electronic device, and storage medium

Abstract

The present disclosure provides a method and device for selective learning of media samples, electronic equipment and storage medium. The method comprises: determining a first media sample and a second media sample from a media sample set, wherein the noise content of the first media sample is less than that of the second media sample; constructing a first cross-entropy loss of the first media sample according to the true label and the predicted label of the first media sample; constructing a regularization loss of the second media sample according to the prior label probability distribution and the predicted label probability distribution of the second media sample; constructing a total sample loss according to the first cross-entropy loss and the regularization loss; and training the model based on the total sample loss so as to realize selective learning of the first media sample and the second media sample by the model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of machine learning, and more particularly to a selective learning method and device for media samples, an electronic device and a storage medium. BACKGROUND

[0002] In practical applications, the collected data is usually incomplete due to the non-standard and imperfect data collection process, privacy issues, etc. This problem is particularly serious in the medical field. These incomplete data sets often contain many complex missing features, which can hinder the effectiveness of machine learning models and seriously affect the performance of machine learning. SUMMARY

[0003] Therefore, the present disclosure provides a selective learning method and device for media samples, an electronic device and a storage medium.

[0004] One aspect of the present disclosure provides a selective learning method for media samples, comprising: determining a first media sample and a second media sample from a media sample set, wherein the noise content of the first media sample is less than that of the second media sample; constructing a first cross-entropy loss of the first media sample according to a true label and a predicted label of the first media sample; constructing a regularization loss of the second media sample according to a prior label probability distribution and a predicted label probability distribution of the second media sample; constructing a total sample loss according to the first cross-entropy loss and the regularization loss; and training a model based on the total sample loss to facilitate selective learning of the first media sample and the second media sample by the model.

[0005] Another aspect of the present disclosure provides a selective learning device for media samples, comprising: a media sample determination module configured to determine a first media sample and a second media sample from a media sample set, wherein the noise content of the first media sample is less than that of the second media sample; a first cross-entropy loss module configured to construct a first cross-entropy loss of the first media sample according to a true label and a predicted label of the first media sample; a regularization loss module configured to construct a regularization loss of the second media sample according to a prior label probability distribution and a predicted label probability distribution of the second media sample; a total sample loss module configured to construct a total sample loss according to the first cross-entropy loss and the regularization loss; and a selective learning module configured to train a model based on the total sample loss to facilitate selective learning of the first media sample and the second media sample by the model.

[0006] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the selective learning method of media samples of the present disclosure.

[0007] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, implement the selective learning method of media samples of the present disclosure.

[0008] According to the above-mentioned embodiments of the present disclosure, a simple and effective selective learning method for incomplete data is proposed, which can selectively learn from the imputed samples. Specifically, the selective learning method can dynamically identify the quality of the imputed samples during the training process. For high-quality samples with high confidence, by adopting empirical risk minimization, better performance on these samples can be ensured. For low-quality imputed samples with high confidence, a regularization term is introduced to prevent the model from overfitting to them, so that the model performance can be maintained. In addition, by dynamically applying regularization to low-quality samples, selective learning can be performed on the imputed data, so that a more reliable model can be trained using the imputed samples, and an accurate and robust classification model can be obtained. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure, taken in conjunction with the accompanying drawings, in which:

[0010] Figure 1A A graph showing the change of reconstruction accuracy rate with respect to different missing rates when the input data contains a large number of inaccurate features is schematically shown;

[0011] Figure 1B A schematic diagram of the optimal decision boundary is schematically shown;

[0012] Figure 1C A schematic diagram of the biased decision boundary is schematically shown;

[0013] Figure 2 An exemplary system architecture to which the selective learning method of media samples can be applied according to embodiments of the present disclosure is schematically shown;

[0014] Figure 3 A flowchart of the selective learning method of media samples according to embodiments of the present disclosure is schematically shown;

[0015] Figure 4 An overview diagram of the selective learning method according to embodiments of the present disclosure is schematically shown;

[0016] Figure 5 a block diagram of a selective learning device of a media sample according to an embodiment of the present disclosure is illustrated schematically; and

[0017] Figure 6 a block diagram of an electronic device adapted to implement a selective learning method of a media sample according to an embodiment of the present disclosure is illustrated schematically. DETAILED DESCRIPTION

[0018] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely exemplary of the present disclosure, and is not intended to limit the scope of the present disclosure. In the following detailed description of the embodiments of the present disclosure, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it would be apparent to one skilled in the art that the present disclosure can be practiced without these specific details. In other instances, well-known structures and functions have been omitted to avoid unnecessarily complicating the present disclosure with details that will be readily apparent to those skilled in the art.

[0019] The terms used herein are merely used to describe specific embodiments, and are not intended to limit the present disclosure. The terms "include" and "have" used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0020] All terms used herein, including technical and scientific terms, have the same meanings as those generally understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the present specification, and should not be interpreted in an idealized or excessively formal manner.

[0021] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should be generally interpreted as including one or more of the items enumerated in the list (e.g., "a system having at least one of A, B, and C" should include, but not be limited to, a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C, etc.).

[0022] In embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of data involved (for example, including but not limited to user personal information) comply with the provisions of relevant laws and regulations, are used for legal purposes, and do not violate public order and good customs. In particular, necessary measures are taken to prevent illegal access to user personal information data, and to maintain user personal information security, network security, and national security.

[0023] In embodiments of the present disclosure, the authorization or consent of the user is obtained before the user's personal information is acquired or collected.

[0024] To improve data quality, ensure the accuracy and reliability of analysis, and minimize or eliminate the impact of missing data on model training and testing, missing data needs to be processed first. The current missing data processing methods can be divided into two categories: deletion method and imputation method. The deletion method directly removes samples or attributes containing missing features from the dataset. Although this method is simple and effective for large datasets with few missing features, it may lead to information loss and bias in the case of non-random missing data. The imputation method estimates and fills in the missing data in the dataset to maintain the integrity of the dataset, which includes single imputation (SI) and multiple imputation (MI). In practical applications, the choice of imputation method depends on the data characteristics, missing data patterns, and specific requirements. Compared with the deletion method, the imputation method can reduce information loss. However, the accuracy of imputation must be considered because inaccurate imputation can have a significant impact on the results.

[0025] Data imputation methods are often used to address the challenges posed by incomplete data during training and testing. These methods estimate and impute missing features based on observed data, aiming to improve data utilization and minimize the negative impact on model performance. The imputed data is used to train machine learning models.

[0026] However, when the missing rate of the original data is high, imputation may introduce noise features, thereby changing the original data distribution. To verify this point, the method examines the reconstruction accuracy (DRA) of the heart disease dataset under different missing rates and different imputation methods.

[0027] Figure 1A The figure schematically shows the change of reconstruction accuracy with respect to different missing rates when the input data contains a large number of inaccurate features.

[0028] As Figure 1A shown, the imputed data contains a large number of inaccurate features, and the reconstruction accuracy gradually decreases as the missing rate increases. When training a machine learning model using imputed data, these inaccurate features will have a negative impact on the model.

[0029] Figure 1B The figure schematically shows the optimal decision boundary.

[0030] As Figure 1B shown, when training the model using ideal complete data, the model can basically learn the ideal classification boundary.

[0031] Figure 1CA schematic diagram showing a biased decision boundary is shown.

[0032] In Figure 1C the presence of low-quality imputed samples, the classification boundary can deviate from the ideal state, as the model overfits to the low-quality imputed samples. Such errors can lead to serious consequences in practical applications.

[0033] Therefore, there is a need for a method that enables the model to learn from imputed data and make more reliable predictions when faced with uncertainty caused by imputation, thereby reducing the negative impact of inaccurate imputation.

[0034] One traditional approach to address this issue is to regularize the model to prevent it from overfitting on low-quality imputed samples. For example, label smoothing (LS) can be applied to the original classification loss function by softening the original true labels. Penalizing confidence (PC) proposes a confidence penalty term to prevent deep neural networks from overfitting. The goal of these methods can be seen as reducing the classification loss indiscriminately while regularizing all training samples, but this can introduce other problems. On the one hand, for high-quality samples that should be classified with high confidence, these methods also aim to reduce their confidence to prevent the model from overfitting to these samples through regularization. On the other hand, for low-quality imputed samples, these methods still try to reduce their classification loss, although these samples themselves may not be correctly classified due to their low quality.

[0035] Regularization is a widely used technique in machine learning, aiming to prevent overfitting and enhance the generalization ability of the model. Common regularization methods are generally divided into implicit regularization and explicit regularization. Implicit regularization methods do not add explicit constraints to the objective function, but indirectly affect the model complexity by modifying the learning process or optimization algorithm, including methods such as label smoothing (LS) and penalizing confidence (PC). Explicit regularization methods, on the other hand, limit model complexity by imposing additional constraints or penalties on parameters, thereby reducing the risk of overfitting. These methods often focus more on specific samples in the data, such as difficult-to-classify samples, like focal loss (FL), and class-imbalanced samples, like dual focal loss (DFL).

[0036] The inventors have found in implementing the present disclosure that, although multiple imputation (MI) methods take uncertainty into account in the imputation process and can mitigate the impact of inaccurate imputed samples to some extent, they cannot completely eliminate the impact of low-quality imputed samples in model training. This requires the use of regularization methods to prevent the model from overfitting to low-quality samples. However, implicit regularization methods usually apply regularization to all samples without distinction, which leads to a decrease in the model's prediction confidence for high-quality samples that are easy to classify. On the other hand, explicit regularization methods achieve regularization by increasing the weight of samples with large losses, which can cause the model to overfit to low-quality samples even more, which can also reduce the model's performance when applied to low-quality samples, causing model bias. Therefore, current regularization methods cannot effectively solve the model bias problem when the data set contains inaccurate imputed samples.

[0037] Embodiments of the present disclosure provide a method, apparatus, electronic device and storage medium for selective learning of media samples. The method comprises: determining a first media sample and a second media sample from a media sample set, wherein the noise content of the first media sample is less than the noise content of the second media sample; constructing a first cross-entropy loss of the first media sample according to a true label and a predicted label of the first media sample; constructing a regularization loss of the second media sample according to a prior label probability distribution and a predicted label probability distribution of the second media sample; constructing a total sample loss according to the first cross-entropy loss and the regularization loss; and training a model based on the total sample loss, so as to achieve selective learning of the first media sample and the second media sample by the model.

[0038] Figure 2 An exemplary system architecture 200 to which the selective learning method of media samples can be applied according to embodiments of the present disclosure is schematically shown. It should be noted that the system architecture shown in FIG. 1 is merely an example of a system architecture to which embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that embodiments of the present disclosure cannot be applied to other devices, systems, environments or scenarios.

[0039] As Figure 2 shown, the system architecture 200 according to this embodiment can include a first terminal device 201, a second terminal device 202, a third terminal device 203, a network 204 and a server 205. The network 204 is a medium for providing a communication link between the first terminal device 201, the second terminal device 202, the third terminal device 203 and the server 205. The network 204 can include various connection types, such as wired and / or wireless communication links, etc.

[0040] The user can use the first terminal device 201, the second terminal device 202, and the third terminal device 203 to interact with the server 205 through the network 204 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 201, the second terminal device 202, and the third terminal device 203, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only as examples).

[0041] The first terminal device 201, the second terminal device 202, and the third terminal device 203 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.

[0042] The server 205 can be a server providing various services, such as a background management server supporting websites browsed by the user using the first terminal device 201, the second terminal device 202, and the third terminal device 203 (only as examples). The background management server can analyze and process received user requests, etc., and feed back the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal device.

[0043] It should be noted that the selective learning method of the media sample provided by the embodiments of the present disclosure can generally be executed by the server 205. Accordingly, the selective learning device of the media sample provided by the embodiments of the present disclosure can generally be arranged in the server 205. The selective learning method of the media sample provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 205 and capable of communicating with the first terminal device 201, the second terminal device 202, the third terminal device 203, and / or the server 205. Accordingly, the selective learning device of the media sample provided by the embodiments of the present disclosure can also be arranged in a server or a server cluster different from the server 205 and capable of communicating with the first terminal device 201, the second terminal device 202, the third terminal device 203, and / or the server 205. Alternatively, the selective learning method of the media sample provided by the embodiments of the present disclosure can also be executed by the first terminal device 201, the second terminal device 202, or the third terminal device 203, or by other terminal devices different from the first terminal device 201, the second terminal device 202, or the third terminal device 203. Accordingly, the selective learning device of the media sample provided by the embodiments of the present disclosure can also be arranged in the first terminal device 201, the second terminal device 202, or the third terminal device 203, or in other terminal devices different from the first terminal device 201, the second terminal device 202, or the third terminal device 203.

[0044] For example, the media sample set can be originally stored in any one of the first terminal device 201, the second terminal device 202, or the third terminal device 203 (for example, the first terminal device 201, but not limited thereto), or stored on an external storage device and imported into the first terminal device 201. Then, the first terminal device 201 can locally perform the selective learning method of the media sample provided by the embodiments of the present disclosure, or send the media sample set to other terminal devices, servers, or server clusters, and perform the selective learning method of the media sample provided by the embodiments of the present disclosure by other terminal devices, servers, or server clusters receiving the media sample set.

[0045] It should be understood that Figure 2 The number of terminal devices, networks, and servers in the system is only illustrative. Any number of terminal devices, networks, and servers can be provided according to implementation needs.

[0046] Figure 3 An illustrative flowchart of the selective learning method of the media sample according to the embodiments of the present disclosure is shown.

[0047] As Figure 3 The method includes operations S301-S305.

[0048] In operation S301, a first media sample and a second media sample are determined from a media sample set, wherein the noise content of the first media sample is less than the noise content of the second media sample.

[0049] According to the embodiments of the present disclosure, the media samples in the media sample set can include any one of the following: text samples, image samples, audio samples, etc., and can not be limited thereto. In the case of including media samples with different noise contents directly collected (without interpolation) in the media sample set, the first media sample with less noise content can be determined as a high-quality sample and the second media sample with more noise content can be determined as a low-quality sample according to the amount of noise content.

[0050] According to an embodiment of the present disclosure, the media sample set can include incomplete media samples, i.e., media samples with missing features, at the initial collection. In this case, the missing features of the media samples with missing features need to be imputed first, and the method can include: obtaining a training sample set, the training sample set including media samples with missing features, wherein the media samples with missing features are represented by a multi-dimensional feature vector and a multi-dimensional mask vector corresponding to the multi-dimensional feature vector, the media samples with missing features also have a true label, and each mask element in the multi-dimensional mask vector indicates whether a corresponding feature vector in the multi-dimensional feature vector is missing or not by using a first value or a second value; obtaining a multi-dimensional weight determined for the multi-dimensional feature vector; calculating a weighted distance between the media sample with missing features and other media samples based on the multi-dimensional weight, wherein the other media samples represent other media samples in the training sample set that have the same true label as the media sample with missing features and are different from the media sample with missing features; determining a preset number of target other media samples closest to the media sample with missing features in terms of the weighted distance; and imputing a missing feature vector of the media sample with missing features according to a target feature vector of the target other media samples to obtain an imputed media sample, wherein the target feature vector and the missing feature vector have a corresponding relationship.

[0051] For example, consider a training sample where is a D-dimensional feature vector, is its corresponding true label. When some features of the sample are missing, a D-dimensional mask vector is used to indicate whether each feature is missing or not. Specifically, indicates that the feature is available, and indicates that the feature is missing, is the th dimension in the feature vector. The obtained training sample set can be represented as where is the number of training samples in the training sample set, is the th training sample in the training sample set, and the goal of incomplete data imputation is to estimate the missing features by finding a data generation function. After imputing all incomplete sample features in the training sample set , a complete media sample set is obtained for the training of the model.

[0052] In the present method, a multiple imputation method can be used to estimate the missing data by using neighbor information, which is called neighbor-based imputation (NBI). With the training sample For example, for the imputation of the missing features, the method first uses the random forest model to obtain the feature importance of the training samples , which represents the D-dimensional weight determined for the D-dimensional feature vector. Then, the D-dimensional weight is used to calculate the weighted distance between the training sample and other samples in the training sample set to ensure that the calculated distance more accurately reflects the true relationship between the samples. Based on the calculated weighted distance, the NBI identifies the nearest neighbor samples of the training sample that belong to the same category. For each missing feature in the training sample , the corresponding values of its nearest neighbor samples can be used to construct a distribution representing the missing feature, achieving the imputation of the missing feature.

[0053] According to an embodiment of the present disclosure, the target feature vector described above can have multiple. The imputation of the missing feature vector of the media sample to be imputed according to the target feature vector of the target other media sample can include: in response to determining that the missing feature is a continuous feature, imputing according to the mean of the multiple target feature vectors; and in response to determining that the missing feature is a discrete feature, imputing according to the mode of the multiple target feature vectors.

[0054] Specifically, for continuous features, a Gaussian distribution can be constructed and imputed with its mean. For discrete features, a multinomial distribution can be constructed and imputed with its mode. After imputing all missing features, the imputed sample can be obtained.

[0055] According to an embodiment of the present disclosure, based on the obtained media sample set described above, the imputed media sample obtained after imputing the missing features of the media sample to be imputed with missing features can be included. In this case, high-quality samples and low-quality samples can be determined according to the imputation accuracy. Specifically, in the case where the imputation accuracy of a first media sample is greater than the imputation accuracy of a second media sample, the first media sample with higher imputation accuracy can be determined as a high-quality sample, and the second media sample with lower imputation accuracy can be determined as a low-quality sample.

[0056] For example, assume that the imputed media sample set is composed of high-quality samples and low-quality samples. Among them, the high-quality samples can include original samples without missing features and accurately imputed media samples, and the low-quality samples can include imputed media samples with inaccurate imputation.

[0057] According to embodiments of this disclosure, before performing the above-described operation S301, the model can be trained for e rounds first. Specifically, the third cross-entropy loss of the media samples can be calculated based on the true and predicted labels of the media samples in the media sample set, and the model can be trained for a preset number of rounds based on the third cross-entropy loss.

[0058] For example, given the complete dataset after interpolation The model can be trained using Empirical Risk Minimization (ERM). Formally, it can be represented as formula (1).

[0059] Formula (1)

[0060] In formula (1), This can represent the third cross-entropy loss function.

[0061] It should be noted that in classification tasks, It can include cross-entropy loss and squared loss, etc., without being limited here.

[0062] The above embodiments of this disclosure can accurately identify low-quality samples.

[0063] According to embodiments of this disclosure, the media sample set may include multiple media samples. High-quality samples and low-quality samples can also be determined by cross-entropy loss values. Specifically, the above operation S301 may include: for each media sample in the media sample set, calculating a second cross-entropy loss value for the media sample based on the true label and predicted label of the media sample, thereby obtaining multiple second cross-entropy loss values ​​determined for multiple media samples in the media sample set; determining a first media sample and a second media sample based on the multiple second cross-entropy loss values, wherein the second cross-entropy loss value of the first media sample is less than the second cross-entropy loss value of the second media sample.

[0064] For example, the formula (1) can be used to... Viewed as a widely used cross-entropy loss Establish the second cross-entropy loss as shown in formula (2).

[0065] Formula (2)

[0066] In formula (2), Indicates the number of categories for the sample label. Indicates sample In category The real label below, Then the corresponding model for the sample Category The probability of predicting the label.

[0067] Low-quality imputed samples typically have higher cross-entropy loss. Inspired by this, low-quality samples can be automatically identified by utilizing the loss value of the training samples. Specifically, during training, given... For each sample, the second cross-entropy loss value can be calculated according to formula (2). If the second cross-entropy loss value of the first media sample is less than that of the second media sample, the first media sample with the lower second cross-entropy loss value is determined as a high-quality sample, and the second media sample with the lower second cross-entropy loss value is determined as a low-quality sample.

[0068] According to embodiments of this disclosure, determining the first media sample and the second media sample based on a plurality of second cross-entropy loss values ​​may include: sorting the plurality of media samples in the media sample set in ascending order based on the plurality of second cross-entropy loss values ​​to obtain an ascending order sorting result; determining a first value based on the product of a predetermined ratio and the total number of samples of the plurality of media samples; determining a second value based on the difference between the total number of samples and the first value; determining the media sample corresponding to the first second value elements in the ascending order sorting result as the first media sample; and determining the media sample corresponding to the last first value elements in the ascending order sorting result as the second media sample.

[0069] According to embodiments of this disclosure, in form, the corresponding distribution of the imputed samples can follow Hubler's... - The pollution model satisfies formula (3).

[0070] Formula (3)

[0071] In formula (3), The distribution representing high-quality samples, This represents the distribution of low-quality samples. This is a parameter representing the proportion of inaccurately imputed data in the overall dataset. For a specific model and dataset, It is a fixed but unknown constant. To simplify calculations in practical applications, this disclosure can treat it as an adjustable hyperparameter, and the aforementioned predetermined ratio can be determined based on it.

[0072] Then, the samples can be sorted in ascending order according to the calculated second cross-entropy loss value, for example, formula (4) can be obtained.

[0073] Formula (4)

[0074] In formula (4), denotes the second cross-entropy loss value of the i-th media sample, and the sorted elements can be denoted as . denotes the second cross-entropy loss value of the i-th media sample. Subsequently, the Huber's -robust loss in formula (3) can be used to determine the low-quality samples. The proportion of low-quality samples is defined as , and the media samples with low loss can be regarded as high-quality samples, which can be denoted as a set as shown in formula (5) . .

[0075] formula (5)

[0076] Correspondingly, the remaining media samples with high loss can be regarded as low-quality samples, which can be denoted as a set as shown in formula (6) .

[0077] formula (6)

[0078] In the above manner, the sample belonging to the distribution or can be estimated in a non-parametric manner, and the media sample set is divided into high-quality samples and low-quality samples to obtain the first media sample and the second media sample.

[0079] The inventors found in the process of implementing the present disclosure that the data imputation method mainly estimates and imputes the missing features based on the observed data. However, the sample set obtained by using the imputation method contains low-quality samples with inaccurate imputation, and there are often a large number of inaccurate features in the low-quality samples. These features can distort the original distribution of the data and cause the model to learn inaccurate information. On this basis, using the above empirical risk minimization method to optimize all samples can cause the model to learn biased information. Specifically, during the model training process, low-quality samples can introduce bias and noise, thereby hindering the model from effectively capturing the potential real information in the data. In this case, the model can overfit these low-quality data during the training process, causing the decision boundary to deviate, and thus leading to a decrease in performance on unseen data.

[0080] Therefore, a method is needed to reduce the negative impact of inaccurate imputation samples and enable the model to effectively learn from imputed data.

[0081] An effective way to prevent the model from overfitting low-quality samples is regularization. However, the goal of the related regularization method can generally be regarded as reducing the classification loss of all samples without distinction while regularizing all training samples. On the one hand, for simple samples that should be classified with high confidence, these methods will reduce their confidence in order to prevent overfitting. On the other hand, for low-quality samples that may not be correctly classified, such methods will also reduce their loss, causing the model to learn a bias. Therefore, the related regularization method cannot solve the model bias problem caused by the overfitting of the model to inaccurate imputed samples. In order to solve this problem, it is necessary to develop an effective and robust selective learning (SLID) method for imputed samples. For example, regularization can be imposed in the loss function to prevent the model from overfitting biased samples.

[0082] In view of the fact that the related regularization method cannot solve the model bias problem caused by inaccurate imputed data introduced by the imputation technique, the present disclosure proposes a selective learning method for incomplete data based on the quality of imputed data. The selective learning method proposed by the present disclosure belongs to an explicit regularization method. The method follows the following principles: for high-quality samples, use cross-entropy loss to let the model normally learn them. For low-quality samples, impose regularization on them based on Kullback-Leibler (KL) loss. And accordingly, operations S302~S303 can be constructed.

[0083] In operation S302, a first cross-entropy loss of the first media sample is constructed according to the true label and the predicted label of the first media sample.

[0084] According to an embodiment of the present disclosure, after the samples are divided into high-quality samples and low-quality samples, the goal of the method is to enable the model to effectively learn from the high-quality samples in the set while minimizing the impact of inaccurate imputed samples in the set . To this end, for high-quality samples, the first cross-entropy loss as shown in formula (7) can be used to train the model.

[0085] Formula (7)

[0086] In operation S303, a regularization loss of the second media sample is constructed according to the prior label probability distribution and the predicted label probability distribution of the second media sample.

[0087] According to an embodiment of the present disclosure, for low-quality samples, regularization can be imposed on them.

[0088] In this case, the prior class probability of this kind of sample can be calculated first The Kullback-Leibler (KL) divergence between the predicted class probability and the prior label probability distribution is obtained. Then, the regularization loss as shown in formula (8) can be constructed for the low-quality sample set . .

[0089] Formula (8)

[0090] In formula (8), is the KL divergence for measuring the difference between two distributions, which can be calculated by formula (9).

[0091] Formula (9)

[0092] In formula (9), denotes the prior label probability distribution, represents the predicted label probability distribution predicted by the model, and correspond to the prior probability and the predicted probability of the class respectively.

[0093] In operation S304, the total loss of the sample is constructed according to the first cross-entropy loss and the regularization loss.

[0094] According to an embodiment of the present disclosure, based on formulas (7)-(9), the total loss of the sample can be represented as formula (10).

[0095] Formula (10)

[0096] In formula (10), is a hyperparameter for representing the strength of regularization.

[0097] Through the above embodiments of the present disclosure, a novel regularization method is proposed, which can dynamically distinguish high-quality and low-quality samples and apply regularization only to low-quality samples. This method enables the model to accurately classify high-quality samples while mitigating the negative impact of low-quality samples, so that the model can effectively learn from high-quality samples while minimizing the impact of low-quality samples on the model, and achieve selective learning of incomplete data.

[0098] In operation S305, the model is trained based on the total loss of the sample, so that the model can achieve selective learning of the first media sample and the second media sample.

[0099] According to an embodiment of the present disclosure, the selective learning method aims to solve the problem of low-quality samples in the data set due to inaccurate imputation. The main goal of selective learning is to effectively learn the imputed data set while mitigating the negative impact of low-quality samples.

[0100] Figure 4 An overview diagram of the selective learning method according to an embodiment of the present disclosure is schematically shown.

[0101] As Figure 4 shown, in the model training phase, high-quality samples following the distribution and low-quality samples following the distribution can be automatically distinguished. The empirical risk minimization is used for high-quality samples to optimize the model. At the same time, regularization is applied to low-quality samples to encourage the model to assign low confidence scores to these samples.

[0102] Through the above embodiments of the present disclosure, a simple and effective selective learning method for incomplete data is proposed, which can selectively learn from imputed samples. Specifically, the selective learning method can dynamically identify the quality of imputed samples during training. For high-quality samples with high confidence, by using empirical risk minimization, better performance on these samples can be ensured. For low-quality imputed samples with high confidence, a regularization term is introduced to prevent the model from overfitting to them, so that the model performance can be maintained and these samples can be identified during testing to avoid making decisions. In addition, by dynamically applying regularization to low-quality samples, selective learning can be performed on imputed data, so that a more reliable model can be trained using imputed samples, and an accurate and robust classification model can be obtained. A large number of experiments prove that the selective learning method improves the accuracy and reliability of the model.

[0103] Figure 5 A block diagram of a selective learning device for media samples according to an embodiment of the present disclosure is schematically shown.

[0104] As Figure 5 shown, the selective learning device for media samples 500 includes a media sample determination module 510, a first cross-entropy loss module 520, a regularization loss module 530, a total sample loss module 540, and a selective learning module 550.

[0105] The media sample determination module 510 is configured to determine a first media sample and a second media sample from a media sample set, wherein the noise content of the first media sample is less than the noise content of the second media sample.

[0106] The first cross-entropy loss module 520 is configured to construct a first cross-entropy loss of the first media sample according to the true label and the predicted label of the first media sample.

[0107] The regularization loss module 530 is configured to construct a regularization loss of the second media sample according to the prior label probability distribution and the predicted label probability distribution of the second media sample.

[0108] The sample total loss module 540 is configured to construct a sample total loss according to the first cross-entropy loss and the regularization loss.

[0109] The selective learning module 550 is configured to train the model based on the sample total loss, so that the model implements selective learning on the first media sample and the second media sample.

[0110] According to an embodiment of the present disclosure, the media sample set includes a plurality of media samples. The media sample determination module includes a second cross-entropy loss value calculation unit and a media sample determination unit.

[0111] The second cross-entropy loss value calculation unit is configured to calculate, for each media sample in the media sample set, a second cross-entropy loss value of the media sample according to a true label and a predicted label of the media sample, to obtain a plurality of second cross-entropy loss values determined for the plurality of media samples in the media sample set.

[0112] The media sample determination unit is configured to determine, according to the plurality of second cross-entropy loss values, a first media sample and a second media sample, the second cross-entropy loss value of the first media sample being smaller than the second cross-entropy loss value of the second media sample.

[0113] According to an embodiment of the present disclosure, the media sample determination unit includes a sorting subunit, a product calculation subunit, a difference calculation subunit, a first media sample determination subunit, and a second media sample determination subunit.

[0114] The sorting subunit is configured to sort, in ascending order, the plurality of media samples in the media sample set according to the plurality of second cross-entropy loss values, to obtain an ascending order sorting result.

[0115] The product calculation subunit is configured to determine a first numerical value according to a product of a predetermined ratio and a total number of samples of the plurality of media samples.

[0116] The difference calculation subunit is configured to determine a second numerical value according to a difference between the total number of samples and the first numerical value.

[0117] The first media sample determination subunit is configured to determine, as the first media sample, a media sample corresponding to a first second numerical value element in the ascending order sorting result.

[0118] The second media sample determination subunit is configured to determine, as the second media sample, a media sample corresponding to an element with a last value in the ascending order result.

[0119] According to an embodiment of the present disclosure, the media sample set comprises an imputed media sample obtained by imputing a missing feature of a media sample to be imputed, and the imputation accuracy of the first media sample is higher than that of the second media sample.

[0120] According to an embodiment of the present disclosure, the selective learning device of the media sample further comprises a training sample set acquisition module, a multi-dimensional weight acquisition module, a weighted distance calculation module, a target other media sample determination module, and an imputation module.

[0121] The training sample set acquisition module is configured to acquire a training sample set, and the training sample set comprises a media sample to be imputed, wherein the media sample to be imputed is represented by a multi-dimensional feature vector and a multi-dimensional mask vector corresponding to the multi-dimensional feature vector, the media sample to be imputed further has a true label, and each mask element in the multi-dimensional mask vector represents whether a corresponding feature vector in the multi-dimensional feature vector is missing or not missing by using a first value or a second value.

[0122] The multi-dimensional weight acquisition module is configured to acquire a multi-dimensional weight determined for the multi-dimensional feature vector.

[0123] The weighted distance calculation module is configured to calculate a weighted distance between the media sample to be imputed and other media samples based on the multi-dimensional weight, wherein the other media samples represent other media samples in the media sample set that have the same true label as the media sample to be imputed and are different from the media sample to be imputed.

[0124] The target other media sample determination module is configured to determine a preset number of target other media samples closest to the media sample to be imputed in terms of the weighted distance.

[0125] The imputation module is configured to impute a to-be-imputed feature vector of the media sample to be imputed according to a target feature vector of the target other media sample to obtain an imputed media sample, wherein the target feature vector has a corresponding relationship with the to-be-imputed feature vector.

[0126] According to an embodiment of the present disclosure, the target feature vector has a plurality of target feature vectors. The imputation module comprises a mean imputation unit and a mode imputation unit.

[0127] The mean imputation unit is configured to, in response to determining that the to-be-imputed feature is a continuous feature, perform imputation according to a mean of the plurality of target feature vectors.

[0128] The mode imputation unit is configured to, in response to determining that the to-be-imputed feature is a discrete feature, perform imputation according to a mode of the plurality of target feature vectors.

[0129] According to an embodiment of the present disclosure, the selective learning device of the media sample further comprises a third cross-entropy loss calculation module and a training module.

[0130] The third cross-entropy loss calculation module is configured to calculate a third cross-entropy loss of the media sample according to a true label and a predicted label of the media sample in the media sample set.

[0131] The training module is configured to train the model for a preset number of rounds based on the third cross-entropy loss.

[0132] Any one or more of the modules, units, sub-units according to the embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, units, sub-units according to the embodiments of the present disclosure can be split into multiple modules for implementation. Any one or more of the modules, units, sub-units according to the embodiments of the present disclosure can be implemented at least in part as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware through integration or packaging of circuits, or in any one of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, one or more of the modules, units, sub-units according to the embodiments of the present disclosure can be implemented at least in part as computer program modules that can perform corresponding functions when executed.

[0133] For example, any and more of the media sample determination module 510, the first cross-entropy loss module 520, the regularization loss module 530, the total sample loss module 540, and the selective learning module 550 can be combined into one module / unit / subunit, or any one of these modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least some of the functionality of one or more of these modules / units / subunits can be combined with at least some of the functionality of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of this disclosure, at least one of the media sample determination module 510, the first cross-entropy loss module 520, the regularization loss module 530, the total sample loss module 540, and the selective learning module 550 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable method of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three methods. Alternatively, at least one of the media sample determination module 510, the first cross-entropy loss module 520, the regularization loss module 530, the total sample loss module 540, and the selective learning module 550 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0134] Figure 6 A block diagram of an electronic device suitable for implementing the selective learning method for media samples according to embodiments of the present disclosure is illustrated. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0135] like Figure 6 As shown, an electronic device 600 according to an embodiment of this disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0136] In the RAM 603, various programs and data required for the operation of the electronic device 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other via the bus 604. The processor 601 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 602 and / or the RAM 603. It should be noted that the programs can also be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 can also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0137] According to an embodiment of the present disclosure, the electronic device 600 can further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 can further include one or more of the following components connected to the input / output (I / O) interface 605: an input part 606 including a keyboard, a mouse, etc.; an output part 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 608 including a hard disk, etc.; and a communication part 609 including a network interface card such as a LAN card, a modem, etc. The communication part 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as necessary. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 610 as necessary, so that a computer program read therefrom is installed in the storage part 608 as necessary.

[0138] According to an embodiment of the present disclosure, the method flow according to the embodiments of the present disclosure can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable storage medium, the computer program containing program codes for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network by the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0139] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or can exist independently without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, which, when executed, implement the selective learning method of media samples according to the embodiments of the present disclosure.

[0140] According to embodiments of the present disclosure, the computer readable storage medium can be a non-volatile computer readable storage medium. For example, it can include, but is not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in connection with an instruction execution system, apparatus, or device.

[0141] For example, according to embodiments of the present disclosure, the computer readable storage medium can include one or more memories of the ROM 602 and / or the RAM 603 described above and / or other than the ROM 602 and the RAM 603.

[0142] Embodiments of the present disclosure also include a computer program product, which includes a computer program containing program codes for executing the method provided by the embodiments of the present disclosure, and when the computer program product is run on an electronic device, the program codes are used to make the electronic device implement the selective learning method of media samples provided by the embodiments of the present disclosure.

[0143] When the computer program is executed by the processor 601, the above functions defined in the system / apparatus of the embodiments of the present disclosure are performed. According to embodiments of the present disclosure, the system, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0144] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, downloaded and installed in the form of signals on a network medium, and be downloaded and installed through the communication part 609 and / or installed from the detachable medium 611. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any suitable combination of the foregoing.

[0145] According to embodiments of the present disclosure, program code of a computer program provided by embodiments of the present disclosure can be written in any combination of one or more programming languages, and specifically, can be implemented using a high-level procedural and / or object-oriented programming language, and / or an assembly / machine language. Programming languages include, but are not limited to, Java, C++, python, "C" language, or similar programming languages. Program code can execute entirely on a user's computing device, partly on a user device, partly on a remote computing device, or entirely on a remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0146] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0147] Embodiments of the present disclosure have been described above. However, these embodiments are merely intended to illustrate the present disclosure, and are not intended to limit the scope of the present disclosure. Although each of the embodiments is described above separately, this does not mean that the measures in each of the embodiments cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. A method for selective learning of media samples, comprising: determining a first media sample and a second media sample from a media sample set, wherein a noise content of the first media sample is less than a noise content of the second media sample, and the media samples in the media sample set comprise any one of the following: text samples, image samples, audio samples; constructing a first cross-entropy loss of the first media sample according to a true label and a predicted label of the first media sample; constructing a regularization loss of the second media sample according to a prior label probability distribution and a predicted label probability distribution of the second media sample; constructing a total sample loss according to the first cross-entropy loss and the regularization loss; and training a model based on the total sample loss so as to implement selective learning of the first media sample and the second media sample by the model, the selective learning aiming to reduce negative effects of low-quality samples while effectively learning an imputed data set; wherein the media sample set comprises imputed media samples obtained by imputing missing features for to-be-imputed media samples with missing features, and an imputation accuracy of the first media sample is greater than an imputation accuracy of the second media sample; the method further comprises, before the determining of the first media sample and the second media sample from the media sample set, obtaining a training sample set comprising the to-be-imputed media sample, wherein the to-be-imputed media sample is represented by a multi-dimensional feature vector and a multi-dimensional mask vector corresponding to the multi-dimensional feature vector, the to-be-imputed media sample further has a true label, each mask element in the multi-dimensional mask vector represents whether a corresponding feature vector in the multi-dimensional feature vector is missing or not by using a first value or a second value, a multi-dimensional weight determined for the multi-dimensional feature vector is obtained, a weighted distance between the to-be-imputed media sample and other media samples is calculated based on the multi-dimensional weight, wherein the other media samples represent other media samples in the media sample set having the same true label as the to-be-imputed media sample and being different from the to-be-imputed media sample, a preset number of target other media samples closest to the to-be-imputed media sample in terms of the weighted distance are determined, and a to-be-imputed feature vector of the to-be-imputed media sample is imputed according to target feature vectors of the target other media samples to obtain the imputed media sample, wherein the target feature vectors have a corresponding relationship with the to-be-imputed feature vector.

2. The method of claim 1, wherein, the media sample set comprises a plurality of media samples; the determining of the first media sample and the second media sample from the media sample set comprises: calculating a second cross-entropy loss value of each media sample in the media sample set according to a true label and a predicted label of the media sample, to obtain a plurality of second cross-entropy loss values determined for the plurality of media samples in the media sample set; and ​ determine a first media sample and a second media sample according to the plurality of second cross-entropy loss values, the first media sample having a smaller second cross-entropy loss value than the second media sample.

3. The method of claim 2, wherein, The determining a first media sample and a second media sample according to the plurality of second cross-entropy loss values comprises: performing ascending order sorting on the plurality of media samples in the media sample set according to the plurality of second cross-entropy loss values to obtain an ascending order sorting result; determine a first numerical value according to a predetermined ratio and a product of a total number of the plurality of media samples; determine a second numerical value according to a difference between the total number of the plurality of media samples and the first numerical value; determine the first media sample as a media sample corresponding to a front second numerical value element in the ascending order sorting result; and determine the second media sample as a media sample corresponding to a rear first numerical value element in the ascending order sorting result.

4. The method of claim 1, wherein, The target feature vectors have a plurality of target feature vectors; the interpolating the to-be-interpolated feature vector of the to-be-interpolated media sample according to the target feature vectors of the target other media samples comprises: in response to determining that the to-be-interpolated feature is a continuous feature, interpolating according to a mean value of the plurality of target feature vectors; and in response to determining that the to-be-interpolated feature is a discrete feature, interpolating according to a mode value of the plurality of target feature vectors.

5. The method of any one of claims 1-4, further comprising: Before the determining a first media sample and a second media sample from a media sample set, calculate a third cross-entropy loss of the media sample according to a real label and a predicted label of the media sample in the media sample set; and train the model for a preset number of rounds based on the third cross-entropy loss.

6. A selective learning device for media samples, comprising: a media sample determination module configured to determine a first media sample and a second media sample from a media sample set, wherein a noise content of the first media sample is smaller than a noise content of the second media sample, and the media samples in the media sample set comprise any one of the following: text samples, image samples, audio samples; a first cross-entropy loss module configured to construct a first cross-entropy loss of the first media sample according to a real label and a predicted label of the first media sample; a regularization loss module configured to construct a regularization loss of the second media sample according to a prior label probability distribution and a predicted label probability distribution of the second media sample; a total sample loss module configured to construct a total sample loss according to the first cross-entropy loss and the regularization loss; and a selective learning module configured to train a model based on the total sample loss, so that the model implements selective learning on the first media sample and the second media sample, and the selective learning aims to effectively learn an interpolated data set while mitigating negative effects of low-quality samples; wherein the media sample set comprises an interpolated media sample obtained by interpolating a to-be-interpolated media sample having missing features, and an interpolation accuracy of the first media sample is greater than an interpolation accuracy of the second media sample. ​ The device further comprises a training sample set obtaining module, a multi-dimensional weight obtaining module, a weighted distance calculating module, a target other media sample determining module and an interpolation module. The training sample set obtaining module is configured to obtain a training sample set, the training sample set comprising the media sample to be interpolated, wherein the media sample to be interpolated is represented by a multi-dimensional feature vector and a multi-dimensional mask vector corresponding to the multi-dimensional feature vector, the media sample to be interpolated further having a true label, each mask element in the multi-dimensional mask vector representing whether a corresponding feature vector in the multi-dimensional feature vector is missing or not by using a first value or a second value; The multi-dimensional weight obtaining module is configured to obtain a multi-dimensional weight determined for the multi-dimensional feature vector; The weighted distance calculating module is configured to calculate a weighted distance between the media sample to be interpolated and other media samples based on the multi-dimensional weight, wherein the other media samples represent other media samples in the media sample set having the same true label as the media sample to be interpolated and being different from the media sample to be interpolated; The target other media sample determining module is configured to determine a preset number of target other media samples closest to the media sample to be interpolated in terms of the weighted distance; The interpolation module is configured to interpolate a feature vector to be interpolated of the media sample to be interpolated according to a target feature vector of the target other media sample to obtain an interpolated media sample, wherein the target feature vector has a corresponding relationship with the feature vector to be interpolated. 7.An electronic device, comprising: one or more processors; memory storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1 to 5. 8.A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 5.

Images