Classification method and electronic equipment

Abstract

The invention discloses a classification method and electronic equipment. The electronic equipment determines a first confidence degree that target data belongs to a first classification set; if the first confidence does not meet the first confidence threshold condition, determining a classification result of the target data based on the second confidence; wherein the second confidence coefficient is the confidence coefficient of the target data belonging to the second classification set, and if the first confidence coefficient meets a first confidence coefficient threshold condition or the second confidence coefficient does not meet a second confidence coefficient threshold condition, the target data belong to the first classification set.

Description

TECHNICAL FIELD

[0001] The present application relates to a classification method and an electronic device. BACKGROUND

[0002] Classification technology is widely used in mobile phone album management, content recommendation and other fields, and its core goal is to improve the data management efficiency and user experience through automatic identification and classification of content. However, the current classification method lacks adaptability to the feature complexity difference between different categories, resulting in low accuracy of the classification result. SUMMARY

[0003] The embodiments of the present application provide a classification method and an electronic device.

[0004] The technical scheme of the embodiments of the present application is implemented as follows:

[0005] In a first aspect, the embodiments of the present application provide a classification method, comprising:

[0006] determining a first confidence that the target data belongs to a first classification set;

[0007] if the first confidence does not satisfy a first confidence threshold condition, determining a classification result of the target data based on a second confidence;

[0008] wherein the second confidence is a confidence that the target data belongs to a second classification set, and if the first confidence satisfies the first confidence threshold condition, or the second confidence does not satisfy a second confidence threshold condition, the target data belongs to the first classification set.

[0009] In the above method, the first confidence that the target data belongs to the first classification set is determined, comprising:

[0010] obtaining a target feature vector of the target data;

[0011] determining the first confidence that the target data belongs to the first classification set based on the target feature vector.

[0012] In the above method, the method further comprises:

[0013] determining the second confidence that the target data belongs to the second classification set based on the target feature vector.

[0014] In the above method, the first classification set comprises a first sub-classification and a second sub-classification, and the first confidence comprises a first sub-confidence corresponding to the first sub-classification and a second sub-confidence corresponding to the second sub-classification.

[0015] if the first confidence does not satisfy the first confidence threshold condition, determining the classification result of the target data based on the second confidence, comprising:

[0016] If the first sub-confidence is not higher than the first sub-confidence threshold and the second sub-confidence is not higher than the second sub-confidence threshold, a classification result of the target data is determined based on the second confidence.

[0017] In the method, the second classification set includes a third sub-classification and a fourth sub-classification, the first sub-classification corresponds to the third sub-classification, and a feature vector corresponding to first data belonging to the first sub-classification satisfies a similarity condition with a feature vector corresponding to second data belonging to the third sub-classification.

[0018] The second sub-classification corresponds to the fourth sub-classification, and a feature vector corresponding to second data belonging to the second sub-classification satisfies a similarity condition with a feature vector corresponding to fourth data belonging to the fourth sub-classification.

[0019] In the method, the second classification set includes a first general classification and a second general classification, the first general classification includes a first classification goal and a second classification goal, and the second general classification includes a third classification goal and a fourth classification goal.

[0020] The classification result of the target data is determined based on the second confidence, including:

[0021] If a first classification confidence belonging to the first classification goal is higher than a first classification confidence threshold, or a second classification confidence belonging to the second classification goal is higher than a second classification confidence threshold, a classification result that the target data belongs to the first general classification is obtained.

[0022] If a third classification confidence belonging to the third classification goal is higher than a third classification confidence threshold, or a fourth classification confidence belonging to the fourth classification goal is higher than a fourth classification confidence threshold, a classification result that the target data belongs to the second general classification is obtained.

[0023] In the method, the first sub-confidence threshold and the second sub-confidence threshold are different.

[0024] The first classification confidence threshold and the second classification confidence threshold are different.

[0025] The third classification confidence threshold and the fourth classification confidence threshold are different.

[0026] In the method, a first similarity of each of a plurality of sub-classifications belonging to the same classification satisfies a preset approximation condition, and the first similarity represents a similarity between data in each sub-classification.

[0027] A second similarity between data belonging to different classifications does not satisfy the preset approximation condition.

[0028] In the method, the target data is image data, and the target data is image data. The first confidence that the target data belongs to the first classification set includes:

[0029] In response to obtaining the target data, a first confidence degree that the target data belongs to the first classification set is determined;

[0030] The method further includes:

[0031] Based on the classification result, label information of the target data is determined;

[0032] The label information is used for indexing to the target data when searching.

[0033] In a second aspect, an electronic device is provided, including a processor, and a memory storing processor-executable instructions; when the instructions are executed by the processor, the following steps are implemented:

[0034] A first confidence degree that the target data belongs to the first classification set is determined;

[0035] If the first confidence degree does not satisfy a first confidence degree threshold condition, a classification result of the target data is determined based on a second confidence degree;

[0036] The second confidence degree is a confidence degree that the target data belongs to a second classification set, and if the first confidence degree satisfies the first confidence degree threshold condition, or the second confidence degree does not satisfy a second confidence degree threshold condition, the target data belongs to the first classification set. BRIEF DESCRIPTION OF DRAWINGS

[0037] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the present application.

[0038] Figure 1 An implementation flowchart of the classification method proposed by the embodiments of the present application is shown in the figure;

[0039] Figure 2 An implementation architecture diagram of the classification method proposed by the embodiments of the present application is shown in the figure;

[0040] Figure 3 A diagram of the probability space proposed by the embodiments of the present application is shown in the figure;

[0041] Figure 4 A diagram of the negative example filtering proposed by the embodiments of the present application is shown in the figure;

[0042] Figure 5 A diagram of the sample distribution proposed by the embodiments of the present application is shown in the figure Figure 1 ;

[0043] Figure 6 A diagram of the sample distribution proposed by the embodiments of the present application is shown in the figure Figure 2 ;

[0044] Figure 7 An implementation schematic diagram of class distinction based on a local confidence threshold value proposed by an embodiment of the present application;

[0045] Figure 8 An assembly structure schematic diagram of an electronic device proposed by an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that, for the convenience of description, only the parts related to the application are shown in the drawings.

[0047] At present, there are some problems in the classification of mobile phone albums. For example, the pictures in the album are intelligently classified by using limited 50 to 100 categories, but the pictures in the user album are wide in range and mixed, and there are many disordered pictures, so it is difficult for the limited categories to cover all use cases. The classification result of the samples that are not covered is unpredictable. In addition, the classification accuracy of each category is easily disturbed by similar negative sample pictures, such as messy garbage bags similar to clothes and ground fallen leaves similar to food. In addition, the categories of the album are limited, and it is impossible to subdivide all categories to only contain one similar feature. There are inevitably some categories that contain more styles, and the number far exceeds that of another category, that is, the data volume of each category is unevenly distributed, for example, the category style of fruits far exceeds that of cats. Or although the sample quantity of each category is balanced, some categories have complex features, and some categories have simple features, causing the classification ability of the classifier for each category to be different. In addition, the current album classification method does not set negative samples, only labels the positive samples corresponding to the to-be-detected categories, trains the classifier to classify, and sets a global confidence threshold value. If the confidence of a category of a picture is higher than the threshold value, the picture is classified into the category. If the picture has no category, it will be classified into the negative sample. This approach will result in insufficient sample coverage and interference from similar feature negative sample pictures. A higher threshold value is likely to miss correctly classified pictures, and a lower threshold value is likely to misclassify pictures with similar features. At the same time, due to the imbalance of training samples between categories, the detection rate of the category with less samples is not high due to excessive training, that is, overfitting, and the category with too many samples has insufficient training and low accuracy, that is, underfitting. The use of a global confidence value also easily causes the slightly high missing rate of the category with strong discrimination ability and the high misclassification rate of the category with weak discrimination ability.

[0048] To solve the above problems, the embodiment of the present application provides a classification method and an electronic device, by introducing key technical points such as multi-negative sample class filtering, positive sample class equal division and local confidence adjustment, effectively solving the problems of low classification accuracy, poor detection rate and uneven sample distribution in the current classification method, so as to realize more efficient and more accurate classification.

[0049] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0050] An embodiment of the present application provides a classification method, as shown in the figure, Figure 1 The classification method of the electronic device can include the following steps:

[0051] Step 101, determining a first confidence that the target data belongs to a first classification set.

[0052] In the embodiments of the present application, the electronic device can determine the first confidence that the target data belongs to the first classification set.

[0053] In the embodiments of the present application, the electronic device can be any device with communication, storage and computing functions, such as tablet computers, notebook computers, mobile phones, displays, all-in-one machines, game consoles and other electronic devices.

[0054] In the embodiments of the present application, the data type of the target data is not limited by the present application, for example, the target data can be image data, audio data and video data, etc.

[0055] In the embodiments of the present application, the confidence refers to the probability or confidence level of predicting that a certain data belongs to a certain category; the confidence usually takes a value range of [0, 1], and the higher the value, the more likely the data belongs to the category; for example, a picture containing a cat may be judged by the model as a cat category with a confidence of 0.85, while another picture with complex background may be judged as a cat category with a confidence of only 0.1.

[0056] In the embodiments of the present application, the classification set is a set of classification targets with clear semantics; the classification set can include a first classification set and a second classification set, wherein the first classification set represents a main category set of negative samples, and the second classification set represents a main category set of positive samples; the first classification set and the second classification set can each include a plurality of sub-classifications.

[0057] In the embodiments of the present application, the negative example category can be understood as a category that has no semantic association such as inclusion, intersection, etc. with the positive example category, i.e. data that does not belong to any to-be-recognized positive category, such as a chaotic background, a non-target object, etc. The existence of the negative example is likely to cause misjudgment in the positive classification process, and therefore the present application filters through the specially designed negative example category, filters through the negative classification before the positive classification, and reduces the misidentification rate of the positive classification result.

[0058] It can be understood that in the embodiments of the present application, the first confidence degree refers to the probability or confidence degree of predicting that the target data belongs to the first classification set of the negative example category; the first confidence degree can include a sub-confidence degree of predicting that the target data belongs to each sub-category in the first classification set.

[0059] In the embodiments of the present application, each sub-category in the first classification set has a certain feature complexity and sample distribution; some categories have weak recognition ability due to small sample quantity or unclear features; and some categories have strong recognition ability due to rich samples and clear features; therefore, a global threshold cannot be simply used to determine whether a picture belongs to a category in the first classification set. By calculating the first confidence degree, the matching degree of the target data and each sub-category in the first classification set can be preliminarily evaluated, which provides a basis for subsequent classification decision and helps to improve the accuracy and robustness of classification.

[0060] In step 102, if the first confidence degree does not satisfy the first confidence degree threshold condition, the classification result of the target data is determined based on the second confidence degree; wherein the second confidence degree is a confidence degree of the target data belonging to the second classification set, and if the first confidence degree satisfies the first confidence degree threshold condition, or the second confidence degree does not satisfy the second confidence degree threshold condition, the target data belongs to the first classification set.

[0061] It can be understood that the first classification set is a bottom classification of the second classification set (data not belonging to the second classification set), and if the second confidence degree does not satisfy the second confidence degree threshold condition, even if the first confidence degree corresponding to the target data does not satisfy the first confidence degree threshold condition, the target data will belong to the first classification set, i.e. the target data will become a "negative classification".

[0062] In the embodiments of the present application, after the electronic device determines the first confidence degree of the target data belonging to the first classification set, if the first confidence degree does not satisfy the first confidence degree threshold condition, the classification result of the target data is determined based on the second confidence degree.

[0063] In some embodiments of the present application, the confidence threshold condition can be a critical value for determining whether the target data belongs to a certain category; the confidence threshold condition can include a first confidence threshold condition and a second confidence threshold condition, wherein the first confidence threshold condition represents a critical value for determining whether the target data belongs to the first classification set, and the second confidence threshold condition represents a critical value for determining whether the target data belongs to the second classification set.

[0064] In some embodiments of the present application, the first confidence not satisfying the first confidence threshold condition can be understood as the first confidence being lower than the first confidence threshold condition; at this time, it indicates that the classification model is not confident enough that the target data belongs to the first classification set, and needs to further refer to the second confidence, i.e., the probability value of the target data belonging to the second classification set.

[0065] In some embodiments of the present application, the order of obtaining the first confidence and the second confidence is not limited in the present application; for example, the first confidence and the second confidence can be obtained simultaneously before being determined by the confidence threshold condition, and then the classification result can be determined by combining the first confidence threshold condition and the second confidence threshold condition; or the first confidence can be obtained first, and then the first confidence threshold condition is used to determine whether it belongs to the first classification set, if not, the second confidence is obtained to further determine by the second confidence threshold condition and the second confidence, and the final classification result is obtained.

[0066] As can be seen, the embodiments of the present application can significantly improve the classification accuracy and detection rate while ensuring the classification efficiency by setting a judgment mechanism of multiple confidences, including two judgment levels of the first confidence and the second confidence, and combining the classification logic of the first classification set and the second classification set about the negative example category and the positive example category; and are especially suitable for scenes such as mobile phone album intelligent classification systems, where image samples are diverse, category distribution is uneven, and there are a large amount of noise interference.

[0067] In some embodiments of the present application, the method for the electronic device to determine the first confidence that the target data belongs to the first classification set can include the following steps:

[0068] Step 101a, obtaining a target feature vector of the target data.

[0069] In the embodiments of the present application, when the electronic device determines the first confidence that the target data belongs to the first classification set, the target feature vector of the target data can be obtained first.

[0070] In the embodiments of the present application, the target feature vector is a numerical representation of a set of key features extracted from the target data, which can include color histograms, texture features, edge information, object positions, etc., depending on the content of the target data and the application scenario; the target feature vector can convert the original target data into structured data that can be used as input for the classifier or classification model; by introducing the target feature vector as an intermediate variable, the classification efficiency can be improved and the misjudgment rate can be reduced while maintaining the classification accuracy, especially when facing diversified image data, the extraction of the target feature vector can help the classification model better capture the essential features of the image and avoid false classification caused by image background interference or noise.

[0071] In the embodiments of the present application, the extraction method of the target feature vector is not limited by the present application, for example, it can be extracted by a pre-trained neural network model.

[0072] For example, in the album classification scenario, if the target data is an image, the target feature vector can be composed of a high-dimensional vector extracted by a convolutional neural network, which contains a highly abstract representation of the image content. This representation not only reduces the computational complexity, but also enhances the understanding ability of the classifier for the image content.

[0073] Step 101b, determining a first confidence degree that the target data belongs to the first classification set based on the target feature vector.

[0074] In the embodiments of the present application, the classification model can compare the target feature vector of the target data with the learned feature distribution of each category during training, so as to calculate the possibility that the target data belongs to the first classification set, i.e. the first confidence degree; the structure of the classification model is not limited by the present application, for example, the classification model can be implemented based on various classification algorithms, such as Support Vector Machine (SVM), Random Forest or Deep Neural Networks (DNN), etc.

[0075] As can be seen, in the embodiments of the present application, it is necessary to extract the feature vector and compare it using the target feature vector to determine whether the target data belongs to the negative example category, i.e. the first classification set, compared with the two-classification method of automatically classifying as negative classification when it is determined that it does not belong to the positive classification in the related art, which can significantly improve the accuracy of classification.

[0076] In some embodiments of the present application, the classification method of the electronic device can further include the following steps:

[0077] Step 103, determining a second confidence that the target data belongs to the second classification set based on the target feature vector.

[0078] In embodiments of the present application, the second confidence refers to a probability or a degree of confidence that the target data belongs to the second classification set of positive example categories; the second confidence can include sub-confidence that the target data belongs to each sub-category in the second classification set.

[0079] In embodiments of the present application, the classification of positive example categories and negative example categories can be implemented based on the same target feature vector; or different feature vectors can be used for the classification of positive example categories and the classification of negative example categories, respectively.

[0080] In some embodiments of the present application, the target data can be subjected to feature extraction by a first feature extractor to obtain a first feature vector, so as to determine a first confidence that the target data belongs to a first classification set based on the first feature vector; the target data can be subjected to feature extraction by a second feature extractor to obtain a second feature vector, so as to determine a second confidence that the target data belongs to a second classification set based on the second feature vector.

[0081] In embodiments of the present application, the first feature extractor and the second feature extractor can be different feature extractors; for example, the first feature extractor is a feature extractor constructed based on a convolutional neural network (ResNet), and the second feature extractor is a feature extractor constructed based on a self-attention model (Transformer).

[0082] As can be seen, the present application can further calculate the second confidence on the basis of the first confidence, forming a judgment mechanism of multiple confidences, which helps to enhance the recognition ability of positive examples, so that based on the recognition of negative examples and positive examples respectively, the wrong classification of the target data can be effectively avoided, and the accuracy of classification can be improved.

[0083] In some embodiments of the present application, the first classification set can include a first sub-category and a second sub-category; the first confidence can include a first sub-confidence corresponding to the first sub-category and a second sub-confidence corresponding to the second sub-category; if the first confidence does not satisfy a first confidence threshold condition, the method for determining a classification result of the target data based on the second confidence by the electronic device can include the following steps:

[0084] Step 102a, if the first sub-confidence is not higher than the first sub-confidence threshold and the second sub-confidence is not higher than the second sub-confidence threshold, determining the classification result of the target data based on the second confidence.

[0085] In embodiments of the present application, the second sub-category is a sub-category different from the first sub-category, and the number of the first sub-category and the second sub-category is not limited in the present application.

[0086] It should be noted that by introducing the first sub-classification and the second sub-classification and setting independent confidence threshold values, the reliability of each sub-classification can be more accurately evaluated. In this way, the problem of misjudgment caused by uneven sample distribution or similar features can be avoided, thereby improving the accuracy and recall rate of classification and enhancing the stability and practicality of the classification model.

[0087] In the embodiments of the present application, the first sub-confidence refers to the probability or reliability of the target data belonging to the first sub-classification.

[0088] In the embodiments of the present application, the first sub-confidence threshold value can be understood as a judgment standard corresponding to the first sub-classification, for measuring whether the judgment of the target data belonging to the first sub-classification is reliable; and the second sub-confidence threshold value can be understood as a judgment standard corresponding to the second sub-classification, for measuring whether the judgment of the target data belonging to the second sub-classification is reliable.

[0089] In the embodiments of the present application, the first sub-confidence threshold value and the second sub-confidence threshold value can be different, and the specific values of the first sub-confidence threshold value and the second sub-confidence threshold value are not limited in the present application.

[0090] For example, the first sub-confidence threshold value is 0.7, the first sub-confidence is 0.6, the second sub-confidence threshold value is 0.6, and the second sub-confidence is 0.5. Since the first sub-confidence is not higher than the first sub-confidence threshold value and the second sub-confidence is not higher than the second sub-confidence threshold value, the classification result of the target data can be determined based on the second confidence.

[0091] In the embodiments of the present application, the first sub-confidence threshold value and the second sub-confidence threshold value can be used to distinguish the confidence judgment standards of two different sub-classes. Since the feature distribution and the number of samples between different sub-classes can be different, setting the first sub-confidence threshold value and the second sub-confidence threshold value for each sub-class separately can better adapt to its distinguishing ability. For example, if the features of a certain sub-class are more obvious and the number of samples is larger, a lower first sub-confidence threshold value can be set to improve the detection rate; while another sub-class with ambiguous features or fewer samples needs to set a higher second sub-confidence threshold value to reduce the false detection rate. In this way, the accuracy and recall rate of the classification result can be more finely controlled, thereby improving the overall classification performance. That is, the present application can better adapt to the differences in distinguishing ability between different classes through the "local confidence" and the corresponding "local confidence threshold", thereby optimizing the classification effect.

[0092] In some embodiments of the present application, if the first sub-confidence is higher than the first sub-confidence threshold value, or the second sub-confidence is higher than the second sub-confidence threshold value, the classification result of the target data belonging to the first classification set is obtained.

[0093] Therefore, the embodiment of the present application can more flexibly adapt to the distribution characteristics of different sub-categories in the negative example category by subdividing the first classification set into multiple sub-categories and setting an independent confidence threshold for each sub-category, thereby improving the generalization ability and robustness of the classification model.

[0094] In some embodiments of the present application, the second classification set includes a third sub-category and a fourth sub-category; the first sub-category corresponds to the third sub-category; the feature vector corresponding to the first data belonging to the first sub-category satisfies the similarity condition with the feature vector corresponding to the second data belonging to the third sub-category.

[0095] In some embodiments of the present application, the second sub-category corresponds to the fourth sub-category; the feature vector corresponding to the second data belonging to the second sub-category satisfies the similarity condition with the feature vector corresponding to the fourth data belonging to the fourth sub-category.

[0096] It can be understood that in the embodiments of the present application, the fourth sub-category can be understood as the classification of the positive example corresponding to the second sub-category.

[0097] It should be noted that the third sub-category can be understood as the classification of the positive example corresponding to the first sub-category; the first sub-category can be used to filter data that does not belong to the third sub-category but has certain similar features to the third sub-category; similarly, the second sub-category can be used to filter data that does not belong to the fourth sub-category but has certain similar features to the fourth sub-category.

[0098] For example, the first sub-category is a fine classification under the negative classification system, and its core function is to form a precise correspondence with the sub-category of a specific positive classification, thereby filtering the "false positive" negative examples of the positive example; assuming that the third sub-category is "cat", the first sub-category can be an image that coincidentally presents the shape of a "cat" due to visual features, such as the approximation of contour, posture, and local texture, which belongs to the category of "other classification", "no classification" or "negative classification"; for example, the first sub-category can be other categories of fallen leaves stacked on the ground to form the contour of a cat, or other categories of clothes randomly stacked to visually present the overall contour of a cat.

[0099] Therefore, the first classification set and the second classification set can have a certain correspondence between each sub-category, and the similarity condition can effectively identify negative class samples that are highly similar to positive class features, thereby improving the anti-interference ability of the classification model.

[0100] In some embodiments of the present application, the second classification set can include a first general classification, a second general classification; the first general classification can include a first classification category, a second classification category; the second general classification can include a third classification category, a fourth classification category; the method for determining the classification result of the target data based on the second confidence can include the following steps:

[0101] Step 201, if the first classification confidence belonging to the first classification category is higher than the first classification confidence threshold, or the second classification confidence belonging to the second classification category is higher than the second classification confidence threshold, the classification result of the target data belonging to the first general classification is obtained.

[0102] Step 202, if the third classification confidence belonging to the third classification category is higher than the third classification confidence threshold, or the fourth classification confidence belonging to the fourth classification category is higher than the fourth classification confidence threshold, the classification result of the target data belonging to the second general classification is obtained.

[0103] In embodiments of the present application, the second classification set of the positive example category can be divided into a first general classification and a second general classification, and each general classification can be further divided into more classification categories; such a multi-layer classification structure can improve the recognition ability of the classification model for complex categories, and avoid the influence of too many features or unbalanced samples on the overall classification effect of a single category. For example, in the album picture classification scene, the person can be taken as the first general classification, and the first general classification includes two classification categories of adults and children; and the scenery is taken as the second general classification, and the second general classification can be divided into specific classification categories such as natural landscape and urban building; in this way, the classification model can make judgments on the basis of finer granularity classification, thereby improving the accuracy and robustness of the overall classification.

[0104] In some embodiments of the present application, the first classification confidence threshold is different from the second classification confidence threshold; the third classification confidence threshold is different from the fourth classification confidence threshold; the specific values of the first classification confidence threshold, the second classification confidence threshold, the third classification confidence threshold and the fourth classification confidence threshold are not limited in the present application.

[0105] It should be noted that, in embodiments of the present application, unlike the unified global confidence threshold used in the related methods, the present application sets an independent confidence threshold for each classification category, thereby adapting to the feature complexity of different classification categories. For example, the confidence threshold of the first classification category can be lower, because the model has strong recognition ability for this category, and the confidence threshold of the second classification category can be higher to reduce the risk of misjudgment. Such a design enables the model to maintain high detection rate while reducing the occurrence of misjudgment when facing the first classification category, the second classification category, the third classification category and the fourth classification category, thereby realizing higher classification precision and robustness.

[0106] Therefore, the embodiment of the present application can improve the support capability for complex classification tasks and enhance the stability of positive sample class classification by subdividing the second classification set into multiple classification categories and setting different confidence thresholds while keeping the classification architecture clear.

[0107] In some embodiments of the present application, the first similarity of each of the plurality of sub-classifications belonging to the same classification meets a preset approximation condition; the first similarity represents the similarity between data in each sub-classification; and the second similarity between data belonging to different classifications does not meet the preset approximation condition.

[0108] In the embodiments of the present application, the first similarity can be used to represent the similarity between the data contained in each sub-classification under the same classification; for example, under the classification of animals, the sub-classifications can be subdivided into cats, dogs, rabbits, etc., and the data in the sub-classifications, such as multiple cat pictures, should have highly consistent features, thereby ensuring that the model has stronger learning ability for the sub-classification.

[0109] In the embodiments of the present application, the preset approximation condition can be a set threshold or range for judging whether the data samples belong to the same sub-classification; the preset approximation condition can be dynamically adjusted according to the actual training data distribution to ensure the consistency of the data in the sub-classification and at the same time avoid confusion between different sub-classifications.

[0110] In the embodiments of the present application, the second similarity can be used to represent the similarity between data of different classifications; for example, although the cat in the animal category and the natural scenery in the scenery category can have similar colors in the background, their semantic contents are completely different, so the similarity between these data samples is low.

[0111] In the embodiments of the present application, when two data respectively belong to different main classifications, if the similarity between them meets the preset approximation condition, it can lead to model misjudgment, i.e., incorrectly classifying data not belonging to a certain category into the category, therefore, the embodiment sets the second similarity not to meet the preset approximation condition to prevent cross-classification error matching, thereby strengthening the classification model's ability to distinguish the boundaries between different classifications and helping to improve the robustness and accuracy of classification.

[0112] It should be noted that in the image classification scenario, if a large category contains a large number of images and the internal difference is significant, such as the "animal" category covering cats, dogs, birds and other subcategories with different shapes, or the "cat" category containing cats in different breeds, poses and scenes, directly training a model with a single category will face two major problems. One is that the model is difficult to capture the core common features within the category, such as the typical shape of a cat, and is instead disturbed by a large number of differentiated features, such as the color difference between black and white cats and the pose difference between standing and lying down, resulting in an increased misjudgment rate for edge samples. The second is that when the internal distribution is loose, such as the "landscape" category containing snow mountains, beaches and forests, the decision boundary generated by the model will be too broad, easily overlapping with other large categories such as "buildings", reducing classification accuracy. Therefore, by hierarchically dividing subcategories, the features of images within the same subcategory are more consistent (the first similarity meets the preset approximate condition), and the features of different large categories are significantly different (the second similarity does not meet the preset approximate condition), which can achieve accurate recognition from coarse to fine.

[0113] In some embodiments of the present application, to achieve the above-mentioned accurate recognition from coarse to fine, when training the classification model, the samples can be divided according to the differences between a large number of samples for training and the data volume. For example, the first-level large categories can be determined according to the core semantics (such as "animal", "plant", "object"). For each large category, if the average similarity of the internal images is lower than the preset threshold, i.e. the difference is significant, then the subcategories are automatically divided. For example, in the "landscape" large category, the feature differences of snow mountain, beach and forest images are calculated. If the similarity between the beach and the snow mountain is lower than the threshold, then the subcategories "beach landscape", "mountain landscape" and the like are divided. If there are still significant differences within the subcategories (such as "beach landscape" containing "sand beach", "rocky coast" and "island"), they can be further recursively divided into more detailed subcategories. When the similarity of the internal images of a category (including large categories and subcategories) reaches the preset threshold, i.e. the features are sufficiently uniform, the division is stopped.

[0114] In some embodiments of the present application, regardless of whether the final division result is a large category or a subcategory, the number of samples of each category in the training set needs to be balanced (such as 1000-2000 images for each category), to avoid the problem of underfitting due to too many samples in a category (such as the sample quantity of "pet cat" being 10 times that of "wild animals"), or the problem of overfitting due to too few samples.

[0115] It can be seen that, in the embodiments of the present application, a complex classification is split into multiple sub-classifications, and it is ensured that the data within each sub-classification has high similarity, and the similarity between the data of different classifications is low, which helps to improve the recognition accuracy of the classification model for samples of each category. This method can effectively solve the overfitting or underfitting problem caused by unbalanced category samples in related methods. In addition, when training the classification model, the positive example category samples and the negative example category samples in the sample data can be evenly divided, so that the number of training samples of all categories is approximately the same, so that the classification model can uniformly learn the features of each category, neither overfitting nor underfitting.

[0116] In some embodiments of the present application, the target data can be image data; when determining the first confidence that the target data belongs to the first classification set, the electronic device can include the following steps:

[0117] Step 301, in response to obtaining the target data, determining the first confidence that the target data belongs to the first classification set.

[0118] It can be understood that the electronic device can extract visual feature information from the image data, and determine the first confidence using the visual feature information.

[0119] In some embodiments of the present application, for the application scenario where the target data is image data, the classification method of the electronic device can further include the following steps:

[0120] Step 302, determining label information of the target data based on the classification result; wherein the label information is used for indexing to the target data when searching.

[0121] For example, a certain picture (target data) is classified as a "landscape" label, and when searching for the picture, the picture can be searched based on the "landscape" label.

[0122] It can be seen that, by combining image feature extraction and a classification model, the embodiments of the present application achieve efficient classification of image data, which is particularly suitable for image-intensive application scenarios such as photo albums, and improves the practicality and expandability of the classification method.

[0123] The embodiment of the application provides a classification method, an electronic device determines a first confidence degree that target data belongs to a first classification set; if the first confidence degree does not satisfy a first confidence degree threshold condition, a classification result of the target data is determined based on a second confidence degree; wherein the second confidence degree is a confidence degree that the target data belongs to a second classification set, if the first confidence degree satisfies the first confidence degree threshold condition, or the second confidence degree does not satisfy a second confidence degree threshold condition, the target data belongs to the first classification set. As can be seen, by setting the first classification set and the second classification set, and the first confidence degree threshold condition corresponding to the first classification set and the second confidence degree threshold condition corresponding to the second classification set, the first confidence degree that the target data belongs to the first classification set and the second confidence degree that the target data belongs to the second classification set can be determined when the target data is classified, so as to compare the first confidence degree with the first confidence degree threshold condition, and compare the second confidence degree with the second confidence degree threshold condition, thereby the classification of the target data can be accurately determined, the accuracy of the classification is ensured, the misjudgment and the omission are reduced, the problem that the classification effect is poor due to that the global confidence degree is difficult to adapt to different classification recognition abilities in the prior art is effectively solved, and the classification accuracy of the target data is greatly improved.

[0124] Based on the above embodiment, in another embodiment of the application, to adapt to the classification of the diversified pictures of the album, the interference of the negative example pictures is reduced by setting a negative example category filter, the positive example category samples are evenly divided to balance the samples of each category, and the local confidence degree is adapted to the recognition ability of each category to improve the effect of the classification of the album pictures. The negative example category filter can be that the negative examples corresponding to each category are marked as negative example categories, and the pictures detected as the negative example categories are filtered out during classification, so that the negative example pictures with similar features and real categories can be filtered out by using the negative example category. The evenly divided positive example category samples means that the categories with more feature styles can be evenly divided into multiple subcategories, so that the number of samples of each category is approximately the same as that of other categories, the data of each category in the classification samples is balanced, and the training of all categories is not under-fitted or over-fitted. The local confidence degree can be that a dedicated confidence threshold is set for each category according to the high and low of the recognition ability of each category. On the premise of not affecting the accuracy, the category with high recognition ability uses a lower confidence degree to improve the detection rate, that is, to reduce the missing detection rate. On the premise of not affecting the detection rate, the category with low recognition ability uses a higher confidence degree to improve the accuracy, that is, to reduce the false detection rate. By increasing the negative example category, evenly dividing the positive example category, and using the local confidence degree, the negative example pictures can be greatly filtered out without changing the complexity and the amount of calculation, the problem of unbalanced categories is solved, and the accuracy and the detection rate of the classification are improved. Moreover, the classification method provided by the embodiment of the application is suitable for the classification of the album pictures, and is also suitable for the classification of pictures with negative samples, unbalanced samples of each category, and different complexity of category features.

[0125] In some embodiments of the present application, data division can be performed according to the data amount of positive example data and negative example data to evenly divide the samples of each category, so as to obtain divided sample data, and then the classification model is trained by using the divided sample data to improve the classification accuracy of the classification model.

[0126] For example, as shown in Figure 2 The classification model can filter the input image according to the negative example category. The negative example category can include multiple categories. If it is determined that the image does not belong to the negative example category, the image is further identified and detected according to the positive example category. The positive example category can also include multiple categories, and the positive example category can further include different subcategories.

[0127] It should be noted that the general image classification only has a positive example category, and the classifier outputs the confidence (conf) of the picture belonging to the positive example category. A threshold (thresh) is set for the confidence. If the confidence of the picture for a certain category is greater than the threshold, the picture is divided into a certain category. However, in the user album, the pictures are diverse, and the negative examples with similar features are easily fall into the probability space of [thresh, 1]. A high threshold value is easy to miss the positive example pictures, and a low threshold value is easy to misdetect the negative example pictures with similar features. As shown in Figure 3 The closer to the center, the higher the confidence, and the farther from the center, the lower the confidence. Based on the threshold filtering, some negative examples fall into the probability space of the positive examples, and some positive examples are filtered out as negative samples because they are outside the probability space. If the threshold is increased, too many positive examples will be filtered out, and if the threshold is reduced, more negative examples will fall into the probability space of the positive examples. Based on this, the negative examples corresponding to each category are marked as negative example categories, and the disordered pictures can be divided into negative example categories. As shown in Figure 4 By adding the negative example category, the disordered pictures are more likely to fall into the probability space of the negative example category during classification, and are less likely to fall into the probability space of the positive example category with similar features. At the same time, the negative example pictures falling into the probability space of the positive example category can also be classified into the probability space of the negative example category. Finally, during the prediction of the classification model, the pictures not falling into the probability space of the positive example category are all filtered out as negative examples. At the same time, by using a suitable confidence threshold, the negative examples with similar features can be distinguished from the positive examples, so as to improve the accuracy of the model without affecting the detection ability of the model.

[0128] It should be noted that, due to the large number of negative sample patterns, using only one negative sample category can easily cause category bloating, which is not conducive to the iterative training of the model, and the detection ability of the negative sample category is not strong enough. If too many negative sample categories are set, it will also lead to excessive occupation of computing power. Based on this, the embodiments of the present application combine similar negative samples into one category. Assuming that the number of positive sample categories is N, some similar negative sample categories are combined, and the number of negative sample categories is controlled to be less than 1 / 3N, so that the complexity and calculation amount of the classification model are basically unchanged.

[0129] In some embodiments of the present application, when the data is divided according to the data amount of the positive sample data and the negative sample data to obtain the divided sample data, the data in the negative sample data that meets the preset similar condition can be processed to obtain the combined negative sample data; the positive sample data is divided according to the data amount of the positive sample data to obtain the divided positive sample data; and the combined negative sample data and the divided positive sample data are determined as the divided sample data.

[0130] As shown in Figure 5 , the sample data amount of each category is different, and in the case that the training rounds of each category are the same, if the categories with more data samples are fully trained, it will lead to low detection rate of the categories with less sample due to excessive training, that is, overfitting. If the training rounds of the categories with less sample are moderate, it will lead to low accuracy of the categories with more sample due to insufficient training, that is, underfitting. Assuming that the categories of persons, landscapes, and cats need to be detected, only the sample data of these categories is collected to train the classifier, such as collecting a certain amount of data according to a certain feature. The sample amount is approximately: cat < person < landscape. In order to fully train the landscape, it is easy to cause overfitting of the cat and person. If the sample amounts of the cat and the person are the same as the landscape, a large amount of repeated feature data will appear in the cat and the person. In order to fully train the landscape, it is easy to cause overfitting of the cat and the person. Based on this, the embodiments of the present application divide the categories with more feature styles into multiple subcategories, so that the sample amount of each category is approximately the same, as shown in Figure 6 , so that all categories can be fully trained under the same training rounds, without underfitting or overfitting. When the classification result of the model is obtained, the subcategories are combined to achieve the same classification effect. Thus, the training sample can be balanced by splitting the subcategories under the same data amount, and the accuracy and detection rate of the model can be further improved without increasing the calculation amount of the model.

[0131] Exemplarily, taking the categories of detected persons, landscapes, and cats as examples, the persons can be divided into three sub-categories: adults, children, and crowds, and the landscapes can be divided into many different scenes: natural landscapes, skies, beaches, mountains, night scenes, and the like. When dividing, the number of samples in the sub-categories of persons and landscapes is ensured to be approximately the same as the number of samples of cats. When training, the same number of training rounds can be used to fully train the sub-categories of cats, persons, and landscapes. When the classification model classifies, the sub-categories of persons are classified as persons, and the sub-categories of landscapes are classified as landscapes.

[0132] It should be noted that, as shown in Figure 7 , the probability space of each category is different, i.e., the discrimination ability is different, because the category feature complexity is different. If a global confidence threshold is used, the category with strong discrimination ability may have a slightly high missed detection rate, and the category with weak discrimination ability may have a high false detection rate. Based on this, the application sets a special confidence threshold for each category, so that the category with strong discrimination ability can improve the detection rate without affecting the accuracy, i.e., reduce the missed detection rate, and the category with weak discrimination ability can improve the accuracy without affecting the detection rate, i.e., reduce the false detection rate.

[0133] Exemplarily, the detection rate represents the proportion of sample pictures correctly recognized by the classification model as belonging to a category, and the detection rate can be represented as TP / TP+FN, where TP is a true positive, which represents a picture that is actually a positive class and is predicted by the model as a positive class, and FN is a false negative, which represents a picture that is actually a positive class but is predicted by the model as a negative class. The missed detection rate represents the proportion of sample pictures that are not correctly recognized by the classification model as belonging to a category, and the missed detection rate can be represented as FN / TP+FN. The accuracy rate represents the proportion of sample numbers correctly predicted by the model to the total number of samples, and the accuracy rate can be represented as TP / TP+FP, where FP is a false positive, which represents a sample that is actually a negative class but is incorrectly predicted by the model as a positive class. The false detection rate represents the proportion of samples that are incorrectly predicted by the classification model as positive classes, and the false detection rate can be represented as FP / TP+FP.

[0134] In the embodiments of the application, further as shown in Figure 8 , the electronic device proposed in the embodiments of the application can include a processor 11, a memory 12 storing instructions executable by the processor 11; further, the electronic device 1 can also include a communication interface 13, and a bus 14 for connecting the processor 11, the memory 12, and the communication interface 13.

[0135] In an embodiment of the present application, the processor 11 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the above-mentioned processor function may also be other, and the embodiment of the present application is not specifically limited. The electronic device 1 may further include a memory 12, which may be connected to the processor 11, wherein the memory 12 is used to store executable program code, the program code including computer operating instructions, and the memory 12 may include a high-speed RAM memory, and may also include a non-volatile memory, for example, at least two disk memories.

[0136] In the embodiment of the present application, the bus 14 is used to connect the communication interface 13, the processor 11 and the memory 12, as well as to facilitate mutual communication between these devices.

[0137] In the embodiment of the present application, the memory 12 is used to store instructions and data.

[0138] Furthermore, in an embodiment of the present application, the processor 11 may be configured to determine a first confidence level that the target data belongs to the first classification set;

[0139] If the first confidence level does not satisfy a first confidence level threshold condition, determining a classification result of the target data based on a second confidence level;

[0140] The second confidence level is the confidence level that the target data belongs to the second classification set. If the first confidence level satisfies the first confidence level threshold condition, or the second confidence level does not satisfy the second confidence level threshold condition, the target data belongs to the first classification set.

[0141] In practical applications, the memory 12 can be a volatile memory, such as a Random-Access Memory (RAM), or a non-volatile memory, such as a Read-Only Memory (ROM), a flash memory, a Hard Disk Drive (HDD) or a Solid-State Drive (SSD), or a combination of the above types of memories, and provides instructions and data to the processor 11.

[0142] Specifically, the program instructions corresponding to the classification method in the embodiment can be stored on a storage medium such as an optical disc, a hard disk, a U disk, etc. When the program instructions corresponding to the classification method in the storage medium are read by the processor or executed, the following steps are included:

[0143] determining a first confidence that the target data belongs to the first classification set;

[0144] if the first confidence does not satisfy a first confidence threshold condition, determining a classification result of the target data based on a second confidence;

[0145] wherein the second confidence is a confidence that the target data belongs to the second classification set, and if the first confidence satisfies the first confidence threshold condition, or the second confidence does not satisfy a second confidence threshold condition, the target data belongs to the first classification set.

[0146] When the program instructions corresponding to the classification method in the storage medium are read by the processor or executed, the following steps can also be included:

[0147] determining a first confidence that the target data belongs to the first classification set;

[0148] if the first confidence does not satisfy a first confidence threshold condition, determining a classification result of the target data based on a second confidence;

[0149] wherein the second confidence is a confidence that the target data belongs to the second classification set, and if the first confidence satisfies the first confidence threshold condition, or the second confidence does not satisfy a second confidence threshold condition, the target data belongs to the first classification set.

[0150] In summary, by setting the first classification set and the second classification set, and the first confidence threshold condition corresponding to the first classification set and the second confidence threshold condition corresponding to the second classification set, the electronic device can determine the first confidence of the target data belonging to the first classification set and the second confidence of the target data belonging to the second classification set when classifying the target data, respectively, compare the first confidence with the first confidence threshold condition and compare the second confidence with the second confidence threshold condition, respectively, so as to accurately determine the classification of the target data, reduce misjudgment and omission while ensuring classification accuracy, effectively solve the problem of poor classification effect in the prior art due to the difficulty of global confidence to adapt to different classification recognition capabilities, and greatly improve the classification accuracy of the target data.

[0151] In addition, each functional module in the embodiment can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional module.

[0152] The integrated unit, if realized in the form of a software functional module and not sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiment can be embodied in the form of a software product, the computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the embodiment method. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0153] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer usable program codes.

[0154] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure 1 one or more functions specified in the flowchart or flowsheet block or blocks. Figure 1 one or more functions specified in the flowchart or flowsheet block or blocks.

[0155] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart or flowsheet block or blocks. Figure 1 one or more functions specified in the flowchart or flowsheet block or blocks. Figure 1 one or more functions specified in the flowchart or flowsheet block or blocks.

[0156] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheet block or blocks. Figure 1 one or more functions specified in the flowchart or flowsheet block or blocks. Figure 1 one or more functions specified in the flowchart or flowsheet block or blocks.

[0157] The above embodiments are only preferred embodiments of the present application, but the protection scope of the present application is not limited to this. Any equivalent substitution or transformation made by those skilled in the art based on the present application should be within the protection scope of the present application.

Claims

1. A classification method comprising: Determining a first confidence level that the target data belongs to the first classification set; If the first confidence level does not satisfy a first confidence level threshold condition, determining a classification result of the target data based on a second confidence level; Among them, the second confidence is the confidence that the target data belongs to the second classification set. If the first confidence satisfies the first confidence threshold condition, or the second confidence does not satisfy the second confidence threshold condition, the target data belongs to the first classification set.

2. The classification method according to claim 1, wherein determining the first confidence level that the target data belongs to the first classification set comprises: Obtaining a target feature vector of the target data; Based on the target feature vector, a first confidence level that the target data belongs to a first classification set is determined.

3. The classification method according to claim 2, further comprising: Based on the target feature vector, a second confidence level that the target data belongs to a second classification set is determined.

4. The classification method according to claim 1, wherein the first classification set includes a first sub-classification and a second sub-classification, and the first confidence level includes a first sub-confidence level corresponding to the first sub-classification and a second sub-confidence level corresponding to the second sub-classification; If the first confidence level does not satisfy a first confidence level threshold condition, determining a classification result of the target data based on a second confidence level includes: If the first sub-confidence is not higher than a first sub-confidence threshold and the second sub-confidence is not higher than a second sub-confidence threshold, a classification result of the target data is determined based on the second confidence.

5. The classification method according to claim 4, The second classification set includes a third subclassification and a fourth subclassification; the first subclassification corresponds to the third subclassification; a feature vector corresponding to first data belonging to the first subclassification and a feature vector corresponding to second data belonging to the third subclassification satisfy a similarity condition; The second subclassification corresponds to the fourth subclassification; a feature vector corresponding to the second data belonging to the second subclassification and a feature vector corresponding to the fourth data belonging to the fourth subclassification satisfy the similarity condition.

6. The classification method according to claim 1, wherein the second classification set includes a first general classification and a second general classification, the first general classification includes a first category and a second category, and the second general classification includes a third category and a fourth category; Determining the classification result of the target data based on the second confidence level includes: If the first classification confidence level of the first classification item is higher than the first classification confidence threshold, or the second classification confidence level of the second classification item is higher than the second classification confidence threshold, a classification result is obtained indicating that the target data belongs to the first general classification; If the third classification confidence belonging to the third classification purpose is higher than the third classification confidence threshold, or the fourth classification confidence belonging to the fourth classification purpose is higher than the fourth classification confidence threshold, the classification result is obtained that the target data belongs to the second general classification.

7. The classification method according to claim 4 or 6, The first sub-confidence threshold is different from the second sub-confidence threshold; The first classification confidence threshold is different from the second classification confidence threshold; The third classification confidence threshold is different from the fourth classification confidence threshold.

8. The classification method according to claim 1, The first similarities of the plurality of subcategories belonging to the same category each satisfy a preset approximation condition; the first similarities represent the similarity between the data in each subcategory; The second similarity between the data belonging to different categories does not satisfy the preset approximation condition.

9. The classification method according to any one of claims 1 to 6, wherein the target data is image data; Determining a first confidence level that the target data belongs to the first classification set includes: In response to acquiring the target data, determining a first confidence level that the target data belongs to the first classification set; The method further comprises: Determining label information of the target data based on the classification result; The tag information is used to index the target data during retrieval.

10. An electronic device comprising a processor and a memory storing instructions executable by the processor; when the instructions are executed by the processor, the following steps are implemented: Determining a first confidence level that the target data belongs to the first classification set; If the first confidence level does not satisfy a first confidence level threshold condition, determining a classification result of the target data based on a second confidence level; in, The second confidence level is the confidence level that the target data belongs to the second classification set. If the first confidence level satisfies the first confidence level threshold condition, or the second confidence level does not satisfy the second confidence level threshold condition, the target data belongs to the first classification set.

Images