Classification correction method and computer program product

Abstract

The invention relates to a classification correction method and a computer program product. The classification correction method comprises the steps that according to a first intermediate result output by a first model in a plurality of models for first input data, the certainty of the first intermediate result is judged, and the first intermediate result comprises a plurality of intermediate categories and confidence coefficients thereof; when the first intermediate result is determined to be uncertain, determining whether statistics of features extracted by the plurality of models from an input data set including the first input data satisfy a correction condition; and when the statistical magnitude satisfies the correction condition, correcting the first prediction classification corresponding to the uncertain first intermediate result according to the context information as the corrected first prediction classification. According to the classification correction method disclosed by the invention, the prediction classification of the uncertain intermediate result is corrected by utilizing the context information, and the accuracy of the prediction classification corresponding to the uncertain intermediate result is improved, so that the accuracy of final classification is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence classification, and in particular to a classification correction method and a computer program product. Background Art

[0002] In order to achieve the goal of classification or scoring, it is often necessary to integrate multiple models for classification, where each model is often pre-trained separately. Since the data used for pre-training may be unbalanced and the task difficulty may be different, the performance of a certain model may be relatively low, resulting in a high error rate for the generated intermediate results when the confidence level is low, resulting in poor final classification performance of the integrated multiple models.

[0003] For example, Figure 1 An example of classifying people in a video as focused or not is shown. Figure 1 As shown in , a model analyzes the input video frames, and the analysis results are converted into different behavior categories and their confidence levels by the softmax function, as shown in Figure 1 As shown in the histogram in , each item in the histogram represents a behavior category and the column length represents the confidence. Then, the behavior category with the highest confidence is selected as the behavior category of the video frame through the Max (softmax) function. Figure 1 In the histograms corresponding to the video frames 102-106 shown, the behavior category with the highest confidence is "looking at the screen", so the behavior category of these video frames is determined to be "looking at the screen". Then, according to the behavior categories of multiple video frames of the video, the attention score is calculated according to the rules (if sleeping behavior occurs, the attention score is 0, if talking behavior occurs, the attention score is 0.5, if yawning behavior occurs, the attention score is 0.25, and if neither sleeping, talking, nor yawning occurs, the screen looking rate is the attention score). Figure 1 In the example, since the behavior category of the three video frames is "looking at the screen", the final attention score is the screen-looking rate equal to 1. Finally, whether the person in the video is focused is judged based on whether the attention score is greater than the threshold. Figure 1 The threshold shown is 0.75, resulting in misclassification as focus. Figure 1In video frames 102 and 104, although the person is not looking at the screen (indicated by a dotted box) and the confidence of the behavior category "looking at the screen" is low, since "looking at the screen" has the highest confidence among all behavior categories, these two video frames 102 and 104 are incorrectly identified as "looking at the screen". In video frame 106, the person is looking at the screen (indicated by a solid box) and the confidence of "looking at the screen" is very high, so this video frame 106 is correctly identified as "looking at the screen". Finally, since the screen-looking rate of the video frames in the video is 1, which is greater than the threshold value of 0.75, the person in this video is incorrectly identified as being focused. This shows that if the behavior category in the middle is wrong, it may eventually lead to an error in the judgment of whether the person is focused.

[0004] Currently, the conventional practice is to directly discard results with low confidence levels; however, a single intermediate result with low confidence may be correct or wrong, and directly discarding such an intermediate result will cause information loss, thus affecting the final classification performance. For example, if you discard Figure 1 If the two video frames 102 and 104 are displayed, the behavior result of video 106 in this video is "watching the video" and the screen viewing rate of the video frames in this video is still 1, so the person in the video is incorrectly identified as focused. In other words, if the intermediate result with low confidence is incorrect, directly discarding it may ultimately lead to incorrect judgment of whether the person is focused. Summary of the Invention

[0005] In view of the above problems, the present disclosure provides a classification correction method and a computer program product.

[0006] According to one aspect of the present disclosure, a classification correction method is provided, comprising:

[0007] Determining the certainty of a first intermediate result output by a first model among the multiple models for the first input data, wherein the first intermediate result includes a plurality of intermediate categories and their confidence levels;

[0008] When the first intermediate result is determined to be uncertain, determining whether statistics of features extracted by the plurality of models from an input data set including the first input data satisfy a correction condition; and

[0009] When the statistic satisfies the correction condition, the first predicted classification corresponding to the uncertain first intermediate result is corrected according to the context information as a corrected first predicted classification.

[0010] According to an example of the present disclosure, the context information includes a plurality of determined second intermediate results output by the plurality of models for the input data set, and

[0011] Correcting the first predicted classification corresponding to the uncertain first intermediate result according to the context information includes:

[0012] Calculating a second predicted classification of the input data set and a confidence level thereof according to the plurality of determined second intermediate results; and

[0013] The first predicted classification corresponding to the uncertain first intermediate result is corrected according to the second predicted classification and its confidence.

[0014] According to an example of the present disclosure, correcting the first predicted classification corresponding to the uncertain first intermediate result according to the second predicted classification and its confidence includes:

[0015] Determine top k intermediate categories in the uncertain first intermediate result whose sum of confidences is greater than a preset confidence threshold, where k is a positive integer;

[0016] Determining a predicted classification for each of the first k intermediate categories based on the confidences of the first k intermediate categories and the second predicted classification; and

[0017] When the second predicted classification is included in the corresponding predicted classifications of the first k intermediate categories, it is determined that the uncertain first intermediate result corresponds to the second predicted classification.

[0018] According to an example of the present disclosure, correcting the first predicted classification corresponding to the uncertain first intermediate result according to the second predicted classification and its confidence includes:

[0019] When the second predicted classification is not included in the corresponding predicted classifications of the top k intermediate categories, it is determined that the uncertain first intermediate result corresponds to a classification of the opposite type to the second predicted classification, the predicted classification with the highest sum of confidences among the top k intermediate categories, or the corresponding predicted classification of the intermediate category with the highest confidence among the top k intermediate categories.

[0020] According to an example of the present disclosure, correcting the first predicted classification corresponding to the uncertain first intermediate result according to the second predicted classification and its confidence includes:

[0021] Determine top k intermediate categories in the uncertain first intermediate result whose sum of confidences is greater than a preset confidence threshold, where k is a positive integer;

[0022] Determining a predicted classification for each of the first k intermediate categories based on the confidences of the first k intermediate categories and the second predicted classification;

[0023] When the second predicted classification is the same as the corresponding predicted classification of the intermediate category with the second highest confidence among the top k intermediate categories, determining that the uncertain first intermediate result corresponds to the second predicted classification; and

[0024] When the second predicted classification is different from the corresponding predicted classification of the intermediate category with the second highest confidence among the top k intermediate categories, it is determined that the uncertain first intermediate result corresponds to the corresponding predicted classification of the intermediate category with the highest confidence among the top k intermediate categories.

[0025] According to an example of the present disclosure, the classification correction method further includes: when the statistic does not meet the correction condition, not correcting the uncertain intermediate result.

[0026] According to another example of the present disclosure, the classification correction method further includes: when the statistic does not meet the correction condition, determining the classification corresponding to the intermediate category with the maximum confidence in the uncertain intermediate result as the corrected first predicted classification.

[0027] According to an example of the present disclosure, the classification correction method further includes: determining a final classification of the input data set based on the corrected first predicted classification.

[0028] According to an example of the present disclosure, the input data of the multiple models is one of the following data types: video, image, voice and text.

[0029] According to an example of the present disclosure, a rule set for the correction condition is self-learned based on multiple sets of labeled data, each set of labeled data including multiple statistics of features extracted by the multiple models from an input sample set and labels of whether the classification predicted by the multiple models based on the features extracted from the input sample set is correct.

[0030] According to an example of the present disclosure, the rule set for self-learning the correction condition based on the multiple sets of labeled data includes:

[0031] Discretizing a continuous attribute of each of the plurality of statistics to obtain a discretized labeled data set, wherein the number of the plurality of statistics is n, where n is a positive integer;

[0032] Generate all candidate rules from a single statistic satisfying the corresponding discrete conditions to n statistics satisfying the corresponding discrete conditions;

[0033] Traverse all candidate rules in the order from a single statistic satisfying the corresponding discrete condition to n statistics satisfying the corresponding discrete condition, and determine whether the discretized labeled data set includes labeled data that meets the candidate rule:

[0034] When the discretized labeled data set does not include labeled data that conforms to the candidate rule, the candidate rule is determined to be an invalid rule; and

[0035] When the discretized labeled data set includes labeled data that conforms to the candidate rule, determining whether the candidate rule only covers misclassified data:

[0036] When the candidate rule only covers misclassified data, adding the candidate rule to the rule set of the correction condition and removing the data covered by the candidate rule from the discretized labeled data set; and

[0037] When the candidate rule covers correctly classified data, the candidate rule is determined to be an invalid rule.

[0038] According to another aspect of the present disclosure, a device for classification correction is provided, comprising: a processor; a memory storing one or more computer program modules; wherein the one or more computer program modules are configured to perform the above-mentioned classification correction method when run by the processor.

[0039] According to yet another aspect of the present disclosure, a computer program product is provided, comprising computer instructions, wherein the computer instructions perform the above classification correction method when executed by a processor.

[0040] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which computer instructions are stored, wherein the computer instructions perform the above-mentioned classification correction method when executed by a processor.

[0041] In the above aspects of the present disclosure, whether to correct the classification is determined by the confidence of the intermediate results and the feature statistics, and the context information is used to correct the predicted classification of the uncertain intermediate results, thereby improving the accuracy of the predicted classification corresponding to the uncertain intermediate results, thereby improving the accuracy of the final classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and other purposes, features, and advantages of the present disclosure will become more apparent through a more detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0043] Figure 1 An example of classifying a person in a video as being focused or not in the prior art is shown;

[0044] Figure 2A flow chart of a classification correction method according to an embodiment of the present disclosure is shown;

[0045] Figure 3 A schematic diagram illustrating a process of calculating attention scores of people in a video using multiple models according to an example of an embodiment of the present disclosure;

[0046] Figure 4 A flow chart of a classification correction method according to an embodiment of the present disclosure is shown;

[0047] Figure 5 A schematic diagram showing a method for correcting uncertain results in a video attentive classification example;

[0048] Figure 6 A flowchart illustrating a process of correcting an ambiguous result based on contextual information according to an embodiment of the present disclosure is shown;

[0049] Figure 7 A flowchart illustrating a process of calculating a predicted classification of an input data set and its confidence level based on a plurality of determined intermediate results according to an example of an embodiment of the present disclosure is shown;

[0050] Figure 8 shows an example of two sets of video frames and their inattention coefficients according to an embodiment of the present disclosure;

[0051] Figure 9 A flowchart illustrating a process of correcting a predicted classification corresponding to an uncertain intermediate result based on the predicted classification of an input data set and its confidence according to an embodiment of the present disclosure is shown;

[0052] Figure 10 A schematic diagram illustrating determining predicted classifications of the top k intermediate categories according to a focused classification example of an embodiment of the present disclosure is shown;

[0053] Figure 11 A flowchart illustrating a process of correcting a predicted classification corresponding to an uncertain intermediate result based on the predicted classification of an input data set and its confidence according to another embodiment of the present disclosure is shown;

[0054] Figure 12 A flowchart illustrating a process of self-learning a rule set for correction conditions based on multiple sets of labeled data according to an embodiment of the present disclosure is shown;

[0055] Figure 13 A schematic diagram showing an example of generating multiple sets of labeled data for video focus prediction according to an embodiment of the present disclosure is shown;

[0056] Figure 14 A functional block diagram of a device for classification correction according to an embodiment of the present disclosure is shown; and

[0057] Figure 15 A schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0058] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0059] Flowcharts are used in this application to illustrate the steps of the methods according to the embodiments of the present application. It should be understood that the preceding or following steps do not necessarily need to be performed in precise order. Instead, the various steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more steps may be removed from these processes.

[0060] First, refer to Figure 2 1 is a flow chart of a classification error correction method 200 based on multiple models according to an embodiment of the present disclosure. Figure 2 As shown, the classification error correction method 200 based on multiple models includes the following steps S202-S206.

[0061] In step S202, the certainty of a first intermediate result output by a first model of the multiple models for the first input data is determined, wherein the first intermediate result includes multiple intermediate categories and their confidence levels. In embodiments of the present disclosure, the model may be any artificial intelligence model. In embodiments of the present disclosure, the first model may be a model with lower reliability, and thus the certainty of the output result is determined based on the confidence levels of each category in the output result. In some embodiments of the present disclosure, for models with higher reliability (e.g., the ratio of the model output result to the manually labeled result is greater than a threshold), the output result may be directly determined to be reliable. In other embodiments of the present disclosure, for models with higher reliability, the certainty of the output result is also determined based on the confidence levels of each category in the output result. In some embodiments of the present disclosure, the method for determining certainty may be, for example, comparing the confidence levels of each category in the output result with a threshold. When the highest confidence level is higher than the threshold, the intermediate result is determined to be certain. In some embodiments of the present disclosure, an intermediate result may be multiple categories and their confidence levels obtained by inputting sample data into a model. Such an intermediate result may be further processed for subsequent prediction and classification.

[0062] When the first intermediate result is determined to be uncertain, in step S204 , it is determined whether the statistics of the features extracted by the multiple models from the input data set including the first input data meet the correction condition.

[0063] When the statistic meets the correction condition, in step S206, the first predicted classification corresponding to the uncertain first intermediate result is corrected according to the context information as the corrected first predicted classification. In an embodiment of the present disclosure, the context information of an input sample (or a piece of input data) may refer to the information of other input samples associated with the input sample, and these other input samples belong to the same sample set as the input sample. In some embodiments of the present disclosure, these other input samples may reflect the overall information of the sample set together with the input sample. For example, in the field of image processing, context information may refer to the information contained in the pixels around a pixel; in the field of video processing, context information may refer to the information contained in the video frames around (before and after) a video frame; in the field of text processing, context information may refer to the context of a sentence; in the field of speech recognition, context information may refer to the speech before and after a speech; in the medical field, the context information of an examination result may refer to the examination results of other examinations performed by a patient at the same time as the examination to obtain the examination result and / or the examination results of the patient at different times.

[0064] Reference Figure 3 , taking the calculation of the attention score of students in class in the video as an example, the classification correction method 200 is further explained. Figure 3 FIG. 1 is a schematic diagram showing a process of calculating the attention score of a person in a video using multiple models according to an example of an embodiment of the present disclosure. Figure 3As shown, video frames 310a-310p are input as input data to the face key point detection model 302, the head posture model 304, the speaking / yawning recognition model 306 and the mask recognition model 308 respectively. The face key point detection model 302 generates a mark to indicate whether the video frame can be cropped out with a face, the head posture model generates a mark in the video frame whether the head is facing up, down, sideways or frontally, the speaking / yawning recognition model generates a mark to indicate whether the person in the video frame is speaking / yawning, and the mask recognition model generates a mark to indicate whether the person in the video frame is wearing a mask, thereby generating video frames 320a-320p with the above marks. In some examples, the facial landmark detection model 302 generates an intermediate result including the categories of whether a face can be cropped and whether a face cannot be cropped, and their confidence levels, and determines whether a face can be cropped based on the confidence levels of the two categories. The head posture model 304 generates an intermediate result including the categories of head facing up, down, sideways, and frontally, and their confidence levels, and determines whether the head is facing up, down, sideways, or frontally based on the confidence levels of these categories. The speaking / yawning recognition model 306 generates an intermediate result including the categories of speaking and yawning, and their confidence levels, and determines whether the face is speaking / yawning based on the confidence levels of these two categories. The mask recognition model 308 generates an intermediate result including the categories of wearing a mask and not wearing a mask, and their confidence levels, and determines whether the face is wearing a mask based on the confidence levels of these two categories. In an embodiment of the present disclosure, an intermediate result includes different categories identified by a model and their confidence levels. The video frames with the head posture facing the front with the above signs are input to the facial behavior recognition model, in which only the video frames where the face can be cut out and the head is facing the front (in Figure 3 The video frames (represented by unshaded boxes in the figure, while shaded boxes represent video frames in which the face cannot be cropped out or the face can be cropped out but the head is not facing the front) are input into the facial behavior recognition model 312 for calculation.

[0065] The facial behavior recognition model 312 calculates the confidence level of different behavior categories (also referred to herein as intermediate categories, such as looking at the screen, looking elsewhere, blinking, covering the face, talking, yawning, etc.) identified from the video frames. In some embodiments, the behavior category with the highest confidence level is determined as the recognized behavior category to obtain video frames 330a-330p labeled with the behavior category. Figure 3 In the example, the dotted line frame represents the behavior category of "looking at the screen", the dotted line frame represents the behavior category of "blinking", and the solid line frame represents the behavior category of "looking elsewhere". In some embodiments, the behavior category with the highest confidence exceeding a preset threshold (e.g., 70%) is determined as the determined behavior category result (referred to herein as the determined intermediate result, in Figure 3The behavior category with the highest confidence that does not exceed the preset threshold is determined as an uncertain behavior category result (referred to as an uncertain intermediate result in this article, Figure 3 ), corresponding to step S202, i.e., judging the certainty of the first intermediate result according to the first intermediate result output by the first model for the first input data. Figure 3 In the illustrated example, video frame 330a indicates that the behavior category is "blinking" and is certain, video frame 330c indicates that the behavior category is "looking at the screen" and is uncertain, video frame 330e indicates that the behavior category is "looking elsewhere" and is certain, and video frame 330f indicates that the behavior category is "blinking" and is uncertain. Figure 3 The example shows a case where models are connected in series, where data is input to one model, the model outputs an intermediate result as input to another model, and the other model outputs another intermediate result. Each intermediate result may be uncertain, thereby affecting subsequent prediction and classification. However, those skilled in the art will appreciate that some embodiments of the present disclosure may be applied to a case where models are connected in parallel. In other words, data is input to multiple models in parallel, and each of the multiple models generates an intermediate result. These intermediate results may be uncertain, thereby affecting subsequent prediction and classification.

[0066] For video frames with uncertain behavior category results, it is necessary to determine whether the video frame needs to be corrected, which corresponds to step S204, that is, when the first intermediate result is determined to be uncertain, determine whether the statistics of the features extracted by multiple models from the input data set including the first input data meet the correction conditions. When the video frame with uncertain behavior category results needs to be corrected, the predicted classification of the video frame with uncertain behavior category results is corrected according to the correction direction determined by the prediction results of the surrounding video frames, thereby obtaining video frames 340a-340p with corrected predicted classifications, which corresponds to step S206, that is, when the statistics meet the correction conditions, the first predicted classification corresponding to the uncertain first intermediate result is corrected according to the context information as the corrected first predicted classification. Figure 3 In the example, the context information refers to the surrounding video frames of the video frame with uncertain behavior category results, and the behavior category results generated by each surrounding video frame correspond to a predicted classification. According to the predicted classification corresponding to each surrounding video frame, the predicted classification corresponding to all surrounding video frames as a whole can be obtained as the predicted result of the surrounding video frames, that is, the correction direction of the predicted classification of the video frame with uncertain behavior category results.

[0067] In some embodiments, the corrected predicted classification can be converted into a behavior category result. For example, if the predicted classification of the video frame is corrected to the focus classification, the corrected predicted classification can be converted into screen looking and calculated into the screen looking rate. Otherwise, it will not be converted into screen looking and calculated into the screen looking rate.

[0068] In the absence of a video frame that needs correction, the attention score is calculated based on the behavior category results (referred to herein as intermediate results) of each video frame 340a-340p. Figure 3 In the example, the attention score is calculated based on the following formula (1):

[0069] Attention score = min(sleeping score, talking score, yawning score, screen viewing rate) (1)

[0070] In one example, if sleeping occurs, the sleeping score is 0, otherwise the sleeping score is 1; if speaking occurs, the speaking score is 0.5, otherwise the speaking score is 1; if yawning occurs, the yawning score is 0.25, otherwise the yawning score is 1. In other words, in this example, the sleeping score is lower when sleeping occurs, the speaking score is lower when speaking occurs, and the yawning score is lower when yawning occurs. The screen viewing rate is the proportion of screen viewing in the actual video. Formula (1) indicates that as long as there is a case of inattention, the attention score is low. The lower the attention score, the less focused, and the higher the attention score, the more focused. Therefore, the attention score can be used to determine whether the user is focused.

[0071] In the case where there is a video frame that needs to be corrected, if the original behavior category result of the video frame with an uncertain behavior category result is "side" (corresponding to the non-focused classification), and the video frame with an uncertain behavior category result is corrected to the focused classification after correction, then the corrected predicted classification is converted to the behavior category result of "looking at the screen", so that the screen-looking in formula (1) increases, which may change the final attention score.

[0072] Figure 4 FIG. 4 shows a flow chart of a classification correction method 400 according to an embodiment of the present disclosure. Figure 4 As shown, the classification correction method 400 may include steps S402 - S408 . Figure 4 Steps S402-S406 of the classification correction method 400 are the same as Figure 2 In step S408, the final classification of the input data set is determined based on the corrected first predicted classification. In some embodiments of the present disclosure, not only the final classification of the input data set is determined, but also the score of the final classification is obtained (for example, Figure 3In some embodiments, when it is determined in step S404 that the statistic does not satisfy the correction condition, the uncertain intermediate result may not be corrected. In some embodiments, when it is determined in step S404 that the statistic does not satisfy the correction condition, the predicted classification corresponding to the category with the maximum confidence in the uncertain intermediate result may be determined as the first predicted classification after correction.

[0073] Figure 5 Schematic diagram showing a method for correcting uncertain results in a video focus classification example. Figure 5 As shown, the classification correction method selects the first few behavior categories whose sum of confidence exceeds a preset threshold (for example, 70%) for determining the behavior category result. According to the mapping relationship between the behavior category and the classification of whether it is focused or not, the first few behavior categories can be mapped to the focused classification, neutral classification or unfocused classification respectively. Then, the percentage of the sum of the confidence of the focused classification, neutral classification and unfocused classification to the sum of the confidence of the first few behavior categories is calculated, so that the uncertain video frame can be judged as an unfocused classification or a focused classification based on the percentage of the focused classification. From Figure 5 It can be seen that video frame 504 identifies that the behavior category with the highest confidence is "looking at the screen", but the confidence of "looking at the screen" does not exceed 70%, which is an uncertain situation (indicated by the dotted box). The confidence of the first two behavior categories identified by video frame 504 exceeds 70%. These two behavior categories are "looking at the screen" and "blocking the face" and both correspond to the focus classification. Therefore, the percentage of the focus classification is 1, and the percentages of the neutral classification and the inattention classification are both 0. It can be judged that video frame 504 corresponds to the focus classification. In addition, video frame 502 is identified as a definite "tilted head" behavior category, which corresponds to the focus classification, and video frame 506 is identified as a definite "looking down" behavior category, which also corresponds to the focus classification. Finally, combined with the classification of whether the video frames 502-506 correspond to focus, it is concluded that the person in the video is focused. However, this conclusion is incorrect. The person in the video changes from looking up to the upper right in video frame 502, through the intermediate state in video frame 504, to looking down to the lower left in video frame 506. This state change clearly indicates that the person in the video is not focused. The present application notes that due to information limitations, the correction performance of a single frame is limited, which may lead to an erroneous final classification conclusion.

[0074] In view of this, the present application proposes a method for correcting the corresponding prediction classification of uncertain intermediate results by considering contextual information. Figure 6 FIG. 6 is a flow chart showing a process 600 for correcting an ambiguous result based on contextual information according to an embodiment of the present disclosure. Figure 6 As shown, process 600 may correspond to Figure 2Step S206 of the method 200 and Figure 4 Step S406 of the method 400 is shown. In some embodiments (e.g. Figure 6 In the exemplary embodiment), the context information may include multiple intermediate results determined by multiple models according to the input data set (for example, Figure 3 In the example, video frame 330a indicates that the behavior category result is "blinking" and is certain, and video frame 330e indicates that the behavior category result is "looking elsewhere" and is certain), the input data set includes multiple input data associated with the input data of the uncertain intermediate result. The multiple input data reflect the information within the field of a single input data of the uncertain intermediate result. In some embodiments, the above-mentioned multiple determined intermediate results include not only multiple determined intermediate results output by a model for different input data, but also multiple determined intermediate results output by different models for the same input data. In some embodiments, the above-mentioned multiple determined intermediate results include results that are directly determined to be reliable output by a model with higher reliability. Figure 6 In the process 600 shown, the predicted classification of the input data set and its confidence can be calculated based on multiple determined intermediate results (which may be referred to as multiple determined second intermediate results in this article), step S602. In some embodiments of the present disclosure, based on which predicted classification has the highest proportion among the multiple predicted classifications corresponding to the multiple determined intermediate results, the predicted classification with the highest proportion can be determined as the predicted classification of the multiple determined intermediate results and the proportion can be set as the confidence of the predicted classification of the multiple determined intermediate results. Those skilled in the art understand that the present disclosure can adopt other reasonable methods to calculate the predicted classification of the input data set and its confidence. Then, the predicted classification corresponding to the uncertain intermediate result can be corrected based on the predicted classification of the input data set and its confidence, step S604.

[0075] In the embodiments of the present disclosure, an intermediate result corresponds to a predicted classification. For example, different categories identified in an intermediate result correspond to a predicted classification respectively. The predicted classification of the intermediate result (referred to as the second predicted classification in this article) can be obtained based on the confidence level and predicted classification of each category. Figure 3In the example shown in , the “looking at the screen” category corresponds to the focused category, the “looking elsewhere” category corresponds to the unfocused category, the “blinking” category corresponds to the neutral category, the “speaking” category corresponds to the unfocused category, the “looking down” category corresponds to the focused category, and so on. In one example, the predicted category corresponding to the category with the highest confidence among the multiple categories in the intermediate result is taken as the predicted category corresponding to the intermediate result. In other examples, other rules can be used to obtain the predicted category corresponding to the intermediate result based on the confidence of each category of the intermediate result. For example, multiple categories whose sum of confidence is greater than a preset threshold can be selected, and the sum of the confidences of the predicted categories corresponding to the multiple categories is calculated, and the predicted category with the highest sum of confidences is selected as the predicted category corresponding to the intermediate result.

[0076] Figure 7 A flowchart of a process 700 of calculating a predicted classification of an input data set and its confidence level according to a plurality of determined intermediate results according to an example of an embodiment of the present disclosure is shown. Figure 7 The illustrated process 700 may correspond to step S602 , ie, calculating the predicted classification of the input data set and its confidence level according to the plurality of determined intermediate results. Figure 7 The example is an example of identifying whether a person in a video is focused on a scene, and describes the process of calculating the inattention coefficient as a process 700 of calculating the predicted classification and confidence of the input data set. In step S702, the number of categories Ncca of the determined behavior category results is counted. The more the number of categories of the determined behavior category results is, the more likely the student is to be inattentive, because he looks around and has diverse behaviors. The determined behavior category results can be mapped to the inattention classification or the focus classification. In step S704, the number Nne of the determined inattention behavior categories is counted. The higher the proportion of inattention behavior, the more likely the student is to be inattentive. In step S706, the number Ns of switches between the uncertain behavior category and the determined behavior category is counted. The higher the number of switches, the more likely the student is to be inattentive, because the confidence distribution of the predicted results varies greatly, indicating that the student's action state is unstable. Then, in step S708, the inattention coefficient, i.e., the confidence of inattention, can be calculated based on one or more combinations of the number Ncca of the determined behavior categories, the number Nne of the determined inattention behavior categories, and the number of switches Ns. In some embodiments, the inattention coefficient can be calculated according to the following formula (2):

[0077] Distraction coefficient = W1*Ncca+W2*[Nne / (number of frames - 1)]+W3*(Ns / number of frames) (2)

[0078] Wherein, W1, W2, and W3 are weight coefficients, and W1+W2+W3=1. In one example, W1=0.5, W2=0.25, and W3=0.25.

[0079] Figure 8 1 shows an example of two sets of video frames and their inattention coefficients according to an embodiment of the present disclosure. Figure 8 As shown, each group of video frames includes 10 video frames, and the highest confidence of the category of the predicted video frame is greater than or equal to the preset threshold (in Figure 8 ), the behavior category of the video frame is considered to be determined (in Figure 8 Otherwise, the behavior category of the video frame is considered uncertain (in Figure 8 (marked with a dotted box). Figure 8 The inattention coefficient of each group of video frames in is based on the reference Figure 7 The described method is used for calculation. Figure 8 The above set of video frames has a high inattention coefficient of 0.65, where the uncertain video frames are some intermediate transition states of inattention behaviors (for example, the process of side face or lowering the head, etc.). Figure 8 The following set of video frames has a low inattention coefficient of 0.17. The uncertain video frames are caused by, for example, a slightly tilted head or blurred image quality. In fact, the person in the video is focused.

[0080] Figure 9 9 is a flowchart showing a process 900 of correcting a predicted classification corresponding to an uncertain intermediate result (referred to herein as a first predicted classification) according to the predicted classification of an input data set and its confidence according to an embodiment of the present disclosure. Figure 9 As shown, in step S902, the first k intermediate categories whose sum of confidences in the uncertain intermediate results (referred to herein as the first intermediate results) is greater than a preset confidence threshold can be determined, where k is a positive integer. In step S904, based on the confidences of the first k intermediate categories and the predicted classification of the input data set, the predicted classification of each of the first k intermediate categories can be determined. In some embodiments, there is a certain mapping relationship between the confidence of the intermediate category and the predicted classification of the input data set (referred to herein as the second predicted classification) and the predicted classification, and step S904 is implemented based on such mapping relationship. Figure 10 FIG. 1 shows a schematic diagram of determining the predicted classification of each of the first k intermediate categories according to the mapping in accordance with an embodiment of the present disclosure, focusing on the classification example. Figure 10 As shown in , the mapping shows whether different categories are focused or not under different inattention confidence levels. Figure 10 In the example of , the two video frames of the “head tilt” class have different inattention confidences (in Figure 7 and Figure 8 In the example, it is called the inattention coefficient), the inattention confidence of the left video frame is greater than 0.4 and the inattention confidence of the right video frame is less than or equal to 0.4. Looking up the mapping table shows that the video frame on the left corresponds to the inattention classification, while the video frame on the right corresponds to the focus classification. Similarly, the two video frames of the "yawning" category are mapped to the inattention classification and the focus classification under different inattention confidence conditions. In addition, the situation is similar for the two video frames of the "occluded face" category and the two video frames of the "speaking" category. In addition, it can be seen from the mapping table that under the "looking at the screen" category, regardless of whether the inattention confidence is greater than 0.4, the video frame corresponds to the focus classification, and under the "side" and "looking elsewhere" categories, regardless of whether the inattention confidence is greater than 0.4, the video frame corresponds to the inattention classification.

[0081] In step S906, it can be determined whether the corresponding predicted classifications of the first k intermediate categories include a correction direction. In the embodiment of the present disclosure, the correction direction can be the predicted classification to which the uncertain intermediate result needs to be corrected when it meets the correction condition, and can be the predicted classification when the confidence of the predicted classification of the input data set is greater than a preset threshold, for example, Figure 7 and Figure 8 When the inattention coefficient is greater than 0.4, the correction direction is inattention classification, otherwise the correction direction is attention classification. Figure 9 , when the correction direction is included in the corresponding predicted classifications of the first k intermediate categories, it can be determined that the uncertain intermediate result corresponds to the correction direction, step S908. When the correction direction is not included in the corresponding predicted classifications of the first k intermediate categories, it can be determined that the uncertain intermediate result corresponds to a classification of the opposite type to the correction direction, step S910. In some embodiments, when the confidence of the predicted classification of the input data set (referred to as the second predicted classification in this article) is greater than a preset threshold, the predicted classification of the input data set can be identified as the correction direction. In other embodiments, when the confidence of the predicted classification of the input data set is not greater than a preset threshold, the classification opposite to the predicted classification of the input data set can be identified as the correction direction. In some embodiments, the correction direction can be identified as the final classification of the input data set.

[0082] Those skilled in the art will understand that when the correction direction is not included in the corresponding predicted classification of the top k intermediate categories, in step S910, it can be determined that the uncertain intermediate result corresponds to the predicted classification with the highest sum of confidences among the top k intermediate categories or the corresponding predicted classification of the intermediate category with the highest confidence among the top k features.

[0083] Figure 11FIG. 1 is a flowchart illustrating a process 1100 of correcting a predicted classification corresponding to an uncertain intermediate result according to the predicted classification of an input data set and its confidence according to another embodiment of the present disclosure. Figure 11 As shown, the process 1100 may include the following steps S1102-S1110: Figure 11 Steps S1102-S1104 of process 1100 are the same as Figure 9 Steps S902-S904 of process 900 are the same as those in step S1106 and are not further described here. In step S1106, it can be determined whether the correction direction is the same as the corresponding predicted classification of the intermediate category with the second highest confidence among the top k intermediate categories. If the correction direction is the same as the corresponding predicted classification of the intermediate category with the second highest confidence among the top k intermediate categories, in step S1108, it can be determined that the uncertain intermediate result corresponds to the correction direction. If the correction direction is not the same as the corresponding predicted classification of the intermediate category with the second highest confidence among the top k intermediate categories, in step S1110, it can be determined that the uncertain intermediate result corresponds to the corresponding predicted classification of the intermediate category with the highest confidence among the top k intermediate categories. As described above, in some embodiments, when the confidence of the predicted classification of the input dataset is greater than a preset threshold, the predicted classification of the input dataset can be identified as the correction direction. In other embodiments, when the confidence of the predicted classification of the input dataset is not greater than a preset threshold, the classification opposite to the predicted classification of the input dataset can be identified as the correction direction.

[0084] As described above, for uncertain intermediate results, their predicted classifications are corrected only when the uncertain intermediate results meet the correction conditions. Figure 12 FIG1 shows a flowchart of a process 1200 for self-learning a rule set for correcting conditions based on multiple sets of labeled data according to an embodiment of the present disclosure. Figure 12 In the process 1200 shown, in step S1202, multiple training models extract features from an input sample set and calculate multiple statistics of the features, and mark whether the classification of the input sample set by the multiple training models is correct to generate a set of labeled data. Multiple sets of labeled data can be obtained by labeling different input sample sets. In some embodiments, multiple training models output multiple statistics for an input data set, and each statistic is generated for features extracted from multiple input data in an input data set. In some embodiments, each set of labeled data includes multiple statistics output by multiple training models for an input data set and a labeling of whether the classification predicted based on the features extracted from the input sample set is correct. In one example, the classification of the context feature set adopts a hard discrimination method, and other suitable discrimination methods can be adopted in other examples.

[0085] In step S1204, the continuous attribute of each of the multiple statistics is discretized to obtain a discretized labeled data set, and the number of the multiple statistics is n, where n is a positive integer. In some embodiments of the present disclosure, the continuous attribute of each of the multiple statistics can be discretized according to a preset discretization condition to obtain a discretized labeled data set. In other embodiments of the present disclosure, other suitable discretization methods can be used, such as an equal-width discretization method, an equal-frequency discretization method, etc.

[0086] In step S1206, all candidate rules are generated, ranging from a single statistic satisfying the corresponding discrete condition to n statistics satisfying the corresponding discrete condition. In step S1208, all candidate rules are traversed in order from a single statistic satisfying the corresponding discrete condition to n statistics satisfying the corresponding discrete condition. For each candidate rule, in step S1210, it is determined whether the discretized labeled dataset includes annotated data that meets the candidate rule. If the discretized labeled dataset does not include annotated data that meets the candidate rule, in step S1212, the candidate rule is invalidated. If the discretized labeled dataset includes annotated data that meets the candidate rule, in step S1214, it is determined whether the candidate rule only covers misclassified data. In step S1214, rules for misclassified samples are screened. If the candidate rule covers correctly classified data, the process proceeds to step S1212, where the candidate rule is invalidated. If the candidate rule only covers misclassified data, in step S1216, the candidate rule is added to the rule set for the correction condition and the data covered by the candidate rule is removed from the discretized labeled dataset.

[0087] Figure 13 A schematic diagram illustrating generating multiple sets of labeled data according to an example of video focus prediction according to an embodiment of the present disclosure is shown. Figure 13The process of generating multiple sets of annotated data in step S1202 is further explained using video focus prediction as an example. A video frame is first input into the facial landmark detection model 1302, which generates a marker indicating that a face cannot be cropped. Then, the video frame with markers indicating that a face can be cropped is input into the head pose model 1304, the speech / yawn recognition model 1306, and the mask recognition model 1308. The head pose model 1304 generates markers indicating that the head is facing up, down, sideways, or frontally. The speech / yawn recognition model 1306 generates a speech / yawn marker, and the mask recognition model 1308 generates a mask marker. Video frames with a frontal head pose are input into the facial action recognition model to generate facial action categories and their confidence levels. Based on the hard discrimination results, a screen-looking marker is then derived for the video frame. Furthermore, the screen-looking marker, mask marker, and speech / yawn marker are used to generate an action percentage. Finally, a focus score is predicted based on the action percentage. In addition, the concentration scores of people in the video are manually annotated to mark whether the concentration classification is correct (indicated by 1) or wrong (indicated by 0).

[0088] On the other hand, statistics f1 to fn of context features are generated based on the facial key point detection model 1302, the head posture model 1304, the speech / yawn recognition model 1306, and the mask recognition model 1308. Figure 13 As shown in , f1 is the “ratio of faces not being cropped”, and fn is the uncertainty rate of the video frame. Each video segment can correspond to a context feature set and a label indicating whether the focus classification of the video segment is correct, i.e., annotated structured data is generated. Further, as described in step S1204, by discretizing the continuous values of the annotated structured data, a discretized annotated data set can be obtained. For example, Figure 13 As shown, the discretization condition of the statistic f1 is whether the continuous value of f1 is greater than 0.8. When the continuous value of f1 is less than or equal to 0.8, the discrete value of the statistic f1 is A1 (for example, 0). When the continuous value of f1 is greater than 0.8, the discrete value of the statistic f1 is A2 (for example, 1). Similarly, the discretization condition of the statistic fn is whether the continuous value of fn is greater than 5. When the continuous value of f1 is less than or equal to 5, the discrete value of the statistic fn is C1 (for example, 0). When the continuous value of fn is greater than 5, the discrete value of the statistic fn is C2 (for example, 1). Those skilled in the art should understand that Figure 13The discrete segmentation forms shown are only examples and are not limiting. The discretization conditions of each statistic can discretize continuous values into more discrete values, for example, more than two discrete values. In some embodiments of the present disclosure, the discretization conditions can be set based on the meaning represented by the statistic of the feature or a general method (such as an equal-width discretization method, an equal-frequency discretization method, etc.) can be used, and the discretization conditions or methods can be adjusted by comparing the classification performance under the automatically generated rule set.

[0089] It should be understood by those skilled in the art that although the present disclosure Figure 3 、 Figure 5 、 Figure 7-Figure 8 、 Figure 10 and Figure 13 The technical solution of the present invention is described by taking the video-focused classification scenario as an example, but the technical solution of the present invention can be extended to other application scenarios. For example, the input data of the multiple models of the input integration can be videos, pictures, texts or voices. In the application scenario of text or voice processing, there are different types of models such as word segmentation models and semantic classification models to process texts. The word segmentation model outputs different word segmentation categories and their confidences for the input text or voice (referred to in this article as intermediate results including multiple intermediate categories and their confidences). For the word segmentation category results with lower confidence, when the statistics of the word segmentation features proposed from the input text meet the correction conditions, the predicted word meaning classification corresponding to the word segmentation (referred to in this article as predicted classification) is corrected according to the preceding and following texts of the word segmentation (referred to in this article as context information).

[0090] In addition, in the disease diagnosis application scenario, the input data can be different medical modalities, such as MRI data, FD-PET data, CSF data or genetic data, etc. At the same time, different analysis models (equivalent to the multiple models referred to in this article) are used for different input data types, such as MRI data analysis model, FD-PET data analysis model, CSF data analysis model and genetic data analysis model, etc. Each of the MRI data, FD-PET data, CSF data and genetic data of a patient can be regarded as an input sample, and the MRI analysis results obtained by the MRI data analysis model (referred to as intermediate results in this article) can include multiple MRI diagnostic categories (referred to as intermediate categories in this article) and their confidence levels. For MRI analysis results with lower confidence levels, when the statistics of the features proposed from the input sample set composed of the multiple data of the patient meet the correction conditions, the predicted disease classification corresponding to the MRI analysis result (referred to as predicted classification in this article) is corrected based on the information of the FD-PET data, CSF data and genetic data of the same patient (referred to as context information in this article).

[0091] The classification correction method based on multiple models disclosed in the present invention does not simply discard input data with uncertain intermediate results, but preserves the contribution of all information. In addition, the classification correction method based on multiple models disclosed in the present invention determines whether to correct the classification based on the confidence level of the intermediate results and the feature statistics, and corrects the predicted classification of input data with uncertain intermediate results based on contextual information, thereby improving the overall classification accuracy. In addition, the self-learning integrated model generates a rule set of correction conditions based on the characteristic regularities that are easily misclassified, so that corrections are made when the intermediate results are uncertain, thereby improving the accuracy of the final classification.

[0092] Below, refer to Figure 14 The following describes a device 1400 for classification correction according to an embodiment of the present disclosure. Figure 14 Schematic diagram of a device for classification correction according to an embodiment of the present disclosure. Figure 2 and Figure 4 The details of the described methods are the same, so for the sake of simplicity, a detailed description of the same contents is omitted here.

[0093] The device for classification correction disclosed in the present invention includes a processor 1402; and a memory 1401, in which computer-readable instructions are stored, wherein a classification correction method is executed when the computer-readable instructions are executed by the processor, including: judging the certainty of a first intermediate result output by a first model among multiple models for first input data, wherein the first intermediate result includes multiple intermediate categories and their confidence levels; when the first intermediate result is judged to be uncertain, judging whether the statistics of the features extracted by the multiple models from the input data set including the first input data meet a correction condition; and when the statistics meet the correction condition, correcting the first predicted classification corresponding to the uncertain first intermediate result as a corrected first predicted classification according to context information.

[0094] Regarding the technical effects of the device 1400 for classification based on multiple models in different embodiments, reference can be made to the technical effects of the classification correction method based on multiple models provided in the embodiments of the present disclosure, which will not be repeated here.

[0095] The device 1400 for performing classification based on multiple models can be used in various suitable electronic devices.

[0096] Figure 15 is a schematic diagram of a computer-readable storage medium 1500 according to an embodiment of the present disclosure.

[0097] like Figure 15As shown, the present disclosure also includes a computer-readable storage medium 1500, on which computer-readable instructions 1501 are stored. When the computer-readable instructions are executed by a computer, the computer executes a classification correction method based on multiple models, including: judging the certainty of a first intermediate result output by a first model among multiple models for first input data, wherein the first intermediate result includes multiple intermediate categories and their confidence levels; when the first intermediate result is judged to be uncertain, judging whether the statistics of the features extracted by the multiple models from the input data set including the first input data meet a correction condition; and when the statistics meet the correction condition, correcting the first predicted classification corresponding to the uncertain first intermediate result as a corrected first predicted classification according to context information.

[0098] The various methods / implementations described in this specification may be used individually or in combination, and may be switched during execution. Furthermore, the processing steps, sequences, flow charts, and the like of the various methods / implementations described in this specification may be reordered as long as there is no conflict. For example, various step units described in this specification are presented in an exemplary order and are not limited to the specific order presented.

[0099] The phrase "based on" used in this specification does not mean "only based on" unless otherwise specified. In other words, the phrase "based on" means both "only based on" and "at least based on."

[0100] Any reference to a unit using a designation such as "first," "second," etc., in this specification does not necessarily limit the number or order of these units. These designations may be used in this specification as a convenient method of distinguishing two or more units. Thus, a reference to a first unit and a second unit does not mean that only two units may be used or that the first unit must precede the second unit in some manner.

[0101] When the terms "include," "comprising," and their variations are used in this specification or claims, these terms are open ended, just like the term "having." Furthermore, the term "or" used in this specification or claims does not mean exclusive or.

[0102] It will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as "data blocks," "modules," "engines," "units," "components," or "systems." In addition, various aspects of the present application may be represented by a computer product located in one or more computer-readable media, the product including computer-readable program code.

[0103] This application uses specific terms to describe the embodiments of this application. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a certain feature, structure, or characteristic associated with at least one embodiment of this application. Therefore, it should be emphasized and noted that "one embodiment," "an embodiment," or "an alternative embodiment" mentioned twice or multiple times in different locations in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application may be appropriately combined.

[0104] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and should not be interpreted in an idealized or highly formal sense, unless explicitly defined as such herein.

[0105] While the present disclosure has been described in detail above, it will be apparent to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure may be implemented in various modifications and variations without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the descriptions in this specification are for illustrative purposes only and are not intended to be limiting of the present disclosure.

Claims

1. A classification correction method, comprising: Determining the certainty of a first intermediate result output by a first model among the multiple models for the first input data, wherein the first intermediate result includes a plurality of intermediate categories and their confidence levels; When the first intermediate result is determined to be uncertain, determining whether statistics of features extracted by the plurality of models from an input data set including the first input data satisfy a correction condition; as well as When the statistic satisfies the correction condition, the first predicted classification corresponding to the uncertain first intermediate result is corrected according to the context information as a corrected first predicted classification.

2. The classification correction method according to claim 1, wherein: The context information includes a plurality of determined second intermediate results output by the plurality of models for the input data set, and Correcting the first predicted classification corresponding to the uncertain first intermediate result according to the context information includes: Calculating a second predicted classification of the input data set and a confidence level thereof according to the plurality of determined second intermediate results; and The first predicted classification corresponding to the uncertain first intermediate result is corrected according to the second predicted classification and its confidence.

3. The classification correction method according to claim 2, wherein: Correcting the first predicted classification corresponding to the uncertain first intermediate result according to the second predicted classification and its confidence includes: Determine top k intermediate categories in the uncertain first intermediate result whose sum of confidences is greater than a preset confidence threshold, where k is a positive integer; Determining a predicted classification for each of the first k intermediate categories based on the confidences of the first k intermediate categories and the second predicted classification; and When the second predicted classification is included in the corresponding predicted classifications of the first k intermediate categories, it is determined that the uncertain first intermediate result corresponds to the second predicted classification.

4. The classification correction method according to claim 3, wherein: Correcting the first predicted classification corresponding to the uncertain first intermediate result according to the second predicted classification and its confidence includes: When the second predicted classification is not included in the corresponding predicted classifications of the top k intermediate categories, it is determined that the uncertain first intermediate result corresponds to a classification of the opposite type to the second predicted classification, the predicted classification with the highest sum of confidences among the top k intermediate categories, or the corresponding predicted classification of the intermediate category with the highest confidence among the top k intermediate categories.

5. The classification correction method according to claim 2, wherein: Correcting the first predicted classification corresponding to the uncertain first intermediate result according to the second predicted classification and its confidence includes: Determine top k intermediate categories in the uncertain first intermediate result whose sum of confidences is greater than a preset confidence threshold, where k is a positive integer; Determining a predicted classification for each of the first k intermediate categories based on the confidences of the first k intermediate categories and the second predicted classification; When the second predicted classification is the same as the corresponding predicted classification of the intermediate category with the second highest confidence among the top k intermediate categories, determining that the uncertain first intermediate result corresponds to the second predicted classification; and When the second predicted classification is different from the corresponding predicted classification of the intermediate category with the second highest confidence among the top k intermediate categories, it is determined that the uncertain first intermediate result corresponds to the corresponding predicted classification of the intermediate category with the highest confidence among the top k intermediate categories.

6. The classification correction method according to any one of claims 1 to 5, further comprising: A final classification of the input data set is determined based on the corrected first predicted classification.

7. The classification correction method according to any one of claims 1 to 5, wherein: The input data of the multiple models is one of the following data types: video, image, voice and text.

8. The classification correction method according to any one of claims 1 to 5, wherein: The rule set of the correction conditions is self-learned based on multiple sets of labeled data, each set of labeled data in the multiple sets of labeled data includes multiple statistics of features extracted by the multiple models from an input sample set and labels of whether the classification predicted by the multiple models based on the features extracted from the input sample set is correct.

9. The classification correction method according to claim 8, wherein: The rule set for self-learning the correction condition based on the multiple sets of labeled data includes: Discretizing a continuous attribute of each of the plurality of statistics to obtain a discretized labeled data set, wherein the number of the plurality of statistics is n, where n is a positive integer; Generate all candidate rules from a single statistic satisfying the corresponding discrete conditions to n statistics satisfying the corresponding discrete conditions; Traverse all candidate rules in the order from a single statistic satisfying the corresponding discrete condition to n statistics satisfying the corresponding discrete condition. For each candidate rule, determine whether the discretized labeled data set includes labeled data that meets the candidate rule: When the discretized labeled data set does not include labeled data that conforms to the candidate rule, the candidate rule is determined to be an invalid rule; and When the discretized labeled data set includes labeled data that conforms to the candidate rule, determining whether the candidate rule only covers misclassified data: When the candidate rule only covers misclassified data, adding the candidate rule to the rule set of the correction condition and removing the data covered by the candidate rule from the discretized labeled data set; and When the candidate rule covers correctly classified data, the candidate rule is determined to be an invalid rule.

10. A computer program product comprising computer instructions, wherein: When the computer instructions are executed by a processor, the classification correction method according to any one of claims 1 to 9 is performed.

Images