Data set granularity classification method and apparatus
By adjusting images to different resolutions and training and testing neural network models, the granularity of the dataset is automatically determined, solving the problem of users lacking judgment skills and realizing convenient and efficient dataset granularity determination.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- THE FOURTH PARADIGM BEIJING TECH CO LTD
- Filing Date
- 2021-12-31
- Publication Date
- 2026-05-29
AI Technical Summary
In existing technologies, users of image classification systems lack the skills to determine the coarseness or fineness of datasets, making human judgment methods inconvenient, time-consuming, laborious, and lacking in universality.
By adjusting the images in the dataset to be classified to two different resolutions, and training and testing them separately using the same neural network model, the classification model accuracy at the two resolutions is obtained, and the dataset granularity is automatically determined based on the difference in accuracy and a preset threshold.
A method for automatically determining the granularity of datasets has been implemented, which is universal and saves time and computing resources.
Smart Images
Figure CN116433949B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image processing, and in particular to a dataset granularity classification method and apparatus. Background Technology
[0002] In deep learning image classification, a distinction is made between classification tasks on coarse-grained and fine-grained datasets. Related technologies often rely on human judgment to determine the granularity of the dataset, followed by targeted modeling based on the judgment results to perform image classification. However, in practice, not all users of image classification systems possess the skill to determine the granularity of a dataset, making this human-judgment-dependent method inconvenient, time-consuming, labor-intensive, and lacking in universality. Summary of the Invention
[0003] This disclosure provides a dataset granularity classification method and apparatus to at least solve the problems in the aforementioned related technologies, or it may not solve any of the aforementioned problems. The technical solution of this disclosure is as follows:
[0004] According to a first aspect of the present disclosure, a dataset granularity classification method is provided, comprising: acquiring a dataset to be classified, wherein the dataset to be classified includes at least one image; obtaining a first dataset and a second dataset based on the dataset to be classified, wherein the first dataset is obtained by adjusting all images in the dataset to a first resolution, and the second dataset is obtained by adjusting all images in the dataset to a second resolution; training and testing a neural network model using the first dataset to obtain a first classification model for image classification and a classification accuracy of the first classification model, and training and testing the neural network model using the second dataset to obtain a second classification model for image classification and a classification accuracy of the second classification model; and obtaining a granularity classification result of the dataset to be classified based on the classification accuracy of the first classification model, the classification accuracy of the second classification model, and a preset threshold.
[0005] Optionally, the dataset to be classified is pre-split into a training dataset and a test dataset; the first dataset includes a first training dataset obtained based on the training dataset and a first test dataset obtained based on the test dataset, and the second dataset includes a second training dataset obtained based on the training dataset and a second test dataset obtained based on the test dataset.
[0006] Optionally, the step of training and testing the neural network model using the first dataset to obtain a first classification model for image classification and the classification accuracy of the first classification model includes: training the neural network model based on a first training dataset to obtain the first classification model; testing the first classification model based on a first test dataset to obtain a first test result; and obtaining the classification accuracy of the first classification model based on the first test result. The step of training and testing the neural network model using the second dataset to obtain a second classification model for image classification and the classification accuracy of the second classification model includes: training the neural network model based on a second training dataset to obtain the second classification model; testing the second classification model based on a second test dataset to obtain a second test result; and obtaining the classification accuracy of the second classification model based on the second test result.
[0007] Optionally, the first test result includes the classification results of the images in the first test dataset; wherein, obtaining the classification accuracy of the first classification model based on the first test result includes: obtaining a first accuracy rate for classifying the images in the first test dataset according to the first test result and the category labels carried by the images in the first test dataset, and using the first accuracy rate as the classification accuracy of the first classification model; the second test result includes the classification results of the images in the second test dataset; wherein, obtaining the classification accuracy of the second classification model based on the second test result includes: obtaining a second accuracy rate for classifying the images in the second test dataset according to the second test result and the category labels carried by the images in the second test dataset, and using the second accuracy rate as the classification accuracy of the second classification model.
[0008] Optionally, the first classification model and the second classification model are trained using the same training configuration and training hyperparameters.
[0009] Optionally, obtaining the granular classification result of the dataset to be classified based on the classification accuracy of the first classification model, the classification accuracy of the second classification model, and a preset threshold includes: obtaining the absolute value of the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model; and obtaining the granular classification result of the dataset to be classified based on the absolute value of the difference and the preset threshold.
[0010] Optionally, obtaining the granular classification result of the dataset to be classified based on the absolute value of the difference and the preset threshold includes: determining that the granular classification result of the dataset to be classified is a fine-grained dataset when the absolute value of the difference is greater than the preset threshold; and determining that the granular classification result of the dataset to be classified is a coarse-grained dataset when the absolute value of the difference is not greater than the preset threshold.
[0011] Optionally, at least one round of training and testing is performed on the neural network model to obtain the first classification model and the second classification model; wherein, when the absolute value of the difference is greater than the preset threshold, determining that the granular classification result of the dataset to be classified is a fine-grained dataset includes: obtaining the absolute value of the difference after each round of training and testing, and comparing the absolute value of the difference with the preset threshold; once it is determined that the absolute value of the difference obtained after a certain round of training and testing is greater than the preset threshold, determining that the granular classification result of the dataset to be classified is a fine-grained dataset, and stopping training and testing.
[0012] According to a second aspect of the present disclosure, a dataset granularity classification apparatus is provided, comprising: an acquisition unit configured to: acquire a dataset to be classified, wherein the dataset to be classified includes at least one image; an adjustment unit configured to: obtain a first dataset and a second dataset based on the dataset to be classified, wherein the first dataset is obtained by adjusting all images in the dataset to be classified to a first resolution, and the second dataset is obtained by adjusting all images in the dataset to be classified to a second resolution; a training unit configured to: train and test a neural network model using the first dataset to obtain a first classification model for image classification and a classification accuracy of the first classification model, and train and test the neural network model using the second dataset to obtain a second classification model for image classification and a classification accuracy of the second classification model; and a classification unit configured to: obtain a granularity classification result of the dataset to be classified based on the classification accuracy of the first classification model, the classification accuracy of the second classification model, and a preset threshold.
[0013] Optionally, the dataset to be classified is pre-split into a training dataset and a test dataset; the first dataset includes a first training dataset obtained based on the training dataset and a first test dataset obtained based on the test dataset, and the second dataset includes a second training dataset obtained based on the training dataset and a second test dataset obtained based on the test dataset.
[0014] Optionally, the training unit is configured to: train the neural network model based on a first training dataset to obtain a first classification model; test the first classification model based on a first test dataset to obtain a first test result; obtain the classification accuracy of the first classification model based on the first test result; train the neural network model based on a second training dataset to obtain a second classification model; test the second classification model based on a second test dataset to obtain a second test result; and obtain the classification accuracy of the second classification model based on the second test result.
[0015] Optionally, the first test result includes the classification result of the images in the first test dataset; the second test result includes the classification result of the images in the second test dataset; wherein, the training unit is configured to: obtain a first accuracy rate for classifying the images in the first test dataset based on the first test result and the class labels carried by the images in the first test dataset, and use the first accuracy rate as the classification accuracy of the first classification model; obtain a second accuracy rate for classifying the images in the second test dataset based on the second test result and the class labels carried by the images in the second test dataset, and use the second accuracy rate as the classification accuracy of the second classification model.
[0016] Optionally, the first classification model and the second classification model are trained using the same training configuration and training hyperparameters.
[0017] Optionally, the classification unit is configured to: obtain the absolute value of the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model based on the classification accuracy of the first classification model and the classification accuracy of the second classification model; and obtain the granular classification result of the dataset to be classified based on the absolute value of the difference and the preset threshold.
[0018] Optionally, the classification unit is configured to: determine that the granularity classification result of the dataset to be classified is a fine-grained dataset when the absolute value of the difference is greater than the preset threshold; and determine that the granularity classification result of the dataset to be classified is a coarse-grained dataset when the absolute value of the difference is not greater than the preset threshold.
[0019] Optionally, at least one round of training and testing is performed on the neural network model to obtain the first classification model and the second classification model; wherein, the classification unit is configured to: obtain the absolute value of the difference once after each round of training and testing, and compare the absolute value of the difference with the preset threshold; once it is determined that the absolute value of the difference obtained after a certain round of training and testing is greater than the preset threshold, the granular classification result of the dataset to be classified is determined to be a fine-grained dataset, and training and testing are stopped.
[0020] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform a dataset granularity classification method according to the present disclosure.
[0021] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by at least one processor, causes the at least one processor to perform a dataset granularity classification method according to the present disclosure.
[0022] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:
[0023] According to the dataset granularity classification method and apparatus disclosed herein, images in the dataset to be classified are adjusted to two different resolutions. Then, the same neural network model is used to train and test on both resolution datasets. This allows for obtaining the classification accuracy of the model trained on the two resolution datasets. These two classification accuracies can then be compared with a preset threshold for granularity to obtain the granularity classification result of the dataset to be classified. Based on this process, automatic determination of dataset granularity can be achieved, demonstrating universality.
[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.
[0026] Figure 1 This is a flowchart illustrating a dataset granularity classification method according to an exemplary embodiment.
[0027] Figure 2 This is an overall flowchart illustrating another dataset granularity classification method according to an exemplary embodiment.
[0028] Figure 3 This is a block diagram illustrating a dataset granularity classification device according to an exemplary embodiment.
[0029] Figure 4 This is a block diagram of an electronic device 400 according to an exemplary embodiment. Detailed Implementation
[0030] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0031] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0032] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.
[0033] In deep learning image classification, there is a distinction between classification tasks on coarse-grained and fine-grained datasets. These datasets can be image datasets. Images in coarse-grained datasets can be distinguished by features such as outlines that occupy most of the image area. Classification tasks on coarse-grained datasets involve classifying images from datasets with coarse-grained attributes, such as outlines that occupy a large portion of the image area. Images in fine-grained datasets can be distinguished by detailed features that occupy a small portion of the image area. Classification tasks on fine-grained datasets involve classifying images from datasets with fine-grained attributes, such as detailed features that occupy a small portion of the image area. For example, a coarse-grained dataset might contain images of different animal breeds like cats, dogs, and horses, while a fine-grained dataset might contain images of different cat breeds, such as breed A, breed B, and breed C. Because most features between image categories in fine-grained datasets are similar, and the effective features used to distinguish between categories are subtle and not prominent, classification tasks on fine-grained datasets are more difficult than those on coarse-grained datasets. Therefore, image classification models for fine-grained datasets cannot be built by modeling features that occupy most of the image area, as is done for coarse-grained datasets. Instead, they need to model the effective, subtle features. Based on this, current methods involve separate modeling for coarse-grained and fine-grained datasets. However, modeling for fine-grained datasets requires significantly more computation than modeling for coarse-grained datasets.
[0034] In related technologies, the granularity of datasets is determined by human judgment (e.g., by AI professionals), and then targeted modeling is performed based on the determination results to classify images. However, in actual image classification system development, such as for users of automated machine learning (AutoML) systems, not all users possess the skill to determine the granularity of datasets. This makes this method, which requires human judgment, inconvenient, time-consuming, labor-intensive, and lacking in universality.
[0035] To address the problems existing in the aforementioned related technologies, this disclosure proposes a dataset granularity classification method and apparatus. The method involves adjusting the images in the dataset to be classified to two different resolutions, and then training and testing the same neural network model on both resolution datasets separately. This allows for obtaining the classification accuracy of the model trained on the two resolution datasets. These two classification accuracies can then be compared with a preset granularity threshold to obtain the granularity classification result of the dataset to be classified. Based on this process, automatic determination of dataset granularity can be achieved, demonstrating universality.
[0036] Below, we will refer to Figures 1 to 4 This document describes in detail the dataset granularity classification method and apparatus according to this disclosure.
[0037] Figure 1 This is a flowchart illustrating a dataset granularity classification method according to an exemplary embodiment. (Refer to...) Figure 1 In step 101, a dataset to be classified can be obtained, wherein the dataset to be classified includes at least one image.
[0038] According to exemplary embodiments of this disclosure, the dataset to be classified can be an image dataset to be classified into a coarse-grained dataset or a fine-grained dataset. Images in the dataset to be classified can be used for image classification. Images in the dataset to be classified can carry category labels, which can be labels used to distinguish categories when performing image classification.
[0039] Existing modeling for classification tasks on fine-grained datasets can be achieved by increasing the resolution of images in the dataset. This is because images with lower resolutions may lose fine-grained features. Based on this, an exemplary embodiment of this disclosure can adjust the resolution of images in the dataset to be classified to two different values, and then train and test the model. Specifically, in step 102, a first dataset and a second dataset can be obtained based on the dataset to be classified. The first dataset is obtained by adjusting all images in the dataset to be classified to a first resolution, and the second dataset is obtained by adjusting all images in the dataset to be classified to a second resolution.
[0040] According to exemplary embodiments of this disclosure, the first resolution and the second resolution can be a higher resolution and a lower resolution that form a distinct contrast. For example, the first resolution can be set to 448 (448×448) and the second resolution to 224 (224×224), or the first resolution can be 224 (224×224) and the second resolution can be 448 (448×448).
[0041] According to an exemplary embodiment of this disclosure, to better train and test the neural network model, the dataset to be classified obtained in step 101 may be pre-split into a training dataset and a test dataset, for example, using an 8:2 or 7:3 ratio. Based on this, the first dataset obtained in step 102 may include a first training dataset based on the training dataset and a first test dataset based on the test dataset; the second dataset may include a second training dataset based on the training dataset and a second test dataset based on the test dataset.
[0042] After obtaining the first and second datasets, the neural network model can be trained using step 103 below. When training and testing with the first and second datasets respectively, the same training configuration and hyperparameters can be used; that is, the first classification model and the second classification model can be trained using the same training configuration and hyperparameters. It should be noted that the training configuration may include, but is not limited to, the neural network model used and the graphics card (GPU) used for training; the training hyperparameters may include, but are not limited to, the learning rate, the batch size selected in a single training epoch, the number of training epochs, data augmentation strategies, the optimizer, the learning rate adjustment strategy, and the loss function used. For example, the neural network model used may be, but is not limited to, ResNet50 or ResNet-101, and the number of training epochs may be, but is not limited to, 20.
[0043] In step 103, the neural network model can be trained and tested using the first dataset to obtain a first classification model for image classification and the classification accuracy of the first classification model. Then, the neural network model can be trained and tested using the second dataset to obtain a second classification model for image classification and the classification accuracy of the second classification model.
[0044] According to an exemplary embodiment of this disclosure, in order to obtain the first classification model and the classification accuracy of the first classification model, the neural network model can first be trained based on the first training dataset to obtain the first classification model.
[0045] Then, the first classification model can be tested based on the first test dataset to obtain the first test result.
[0046] As can be seen here, the first classification model is used for image classification. Therefore, the images in the first test dataset can be predicted using the first classification model to obtain the first test result, which may include the classification results of the images in the first test dataset.
[0047] Finally, the classification accuracy of the first classification model can be obtained based on the first test results.
[0048] Here, the first accuracy rate for classifying the images in the first test dataset can be obtained based on the first test results and the category labels carried by the images in the first test dataset, and the first accuracy rate can be used as the classification accuracy of the first classification model.
[0049] For example, the number of images contained in the first test dataset can be used as the denominator, the number of images with the same class label as the first test result can be used as the numerator, and the score obtained based on the numerator and denominator can be used as the first accuracy.
[0050] According to an exemplary embodiment of this disclosure, in order to obtain the classification accuracy of the second classification model and the second classification model, the neural network model can first be trained based on the second training dataset to obtain the second classification model.
[0051] Here, the same training configuration and hyperparameters as those used in training the neural network model based on the first training dataset can be used for training.
[0052] Then, the second classification model can be tested based on the second test dataset to obtain the second test results.
[0053] As we know here, the second classification model is used for image classification. Therefore, the second classification model can be used to predict the images in the second test dataset to obtain the second test result, which can include the classification results of the images in the second test dataset.
[0054] Finally, the classification accuracy of the second classification model can be obtained based on the results of the second test.
[0055] Here, based on the category labels carried by the images in the second test dataset and the second test results, the second accuracy of classifying the images in the second test dataset can be obtained, and the second accuracy can be used as the classification accuracy of the second classification model.
[0056] For example, the number of images in the second test dataset can be used as the denominator, the number of images in the second test result that have the same class label as the numerator can be used as the numerator, and the score obtained based on the numerator and denominator can be used as the second accuracy.
[0057] It should be noted that the exemplary embodiments of this disclosure are described in terms of obtaining the first classification model and the classification accuracy of the first classification model first, and then obtaining the second classification model and the classification accuracy of the second classification model. However, this disclosure is not limited to this, and also protects the cases in which the classification accuracy of the second classification model and the second classification model is obtained first, and then the classification accuracy of the first classification model and the first classification model is obtained, as well as the cases in which two classification models and their corresponding classification accuracies are obtained simultaneously.
[0058] In step 104, the granular classification result of the dataset to be classified can be obtained based on the classification accuracy of the first classification model, the classification accuracy of the second classification model, and a preset threshold. The preset threshold can be set to be greater than 0.
[0059] According to an exemplary embodiment of this disclosure, the classification accuracy corresponding to a higher resolution dataset and the classification accuracy corresponding to a lower resolution dataset can be subtracted. The difference is then compared with a preset threshold, and the granular classification result is obtained based on the relationship between the two. If the difference is greater than the preset threshold, the granular classification result of the dataset to be classified is determined to be a fine-grained dataset. If the difference is not greater than the preset threshold, the granular classification result of the dataset to be classified is determined to be a coarse-grained dataset. For an exemplary embodiment of this disclosure, the first resolution may be higher or lower than the second resolution. Based on this, the absolute value of the difference between the two classification accuracies can be calculated. Specifically, firstly, the absolute value of the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model can be obtained based on the classification accuracy of the first classification model and the classification accuracy of the second classification model; then, the granular classification result of the dataset to be classified can be obtained based on the absolute value of the difference and the preset threshold.
[0060] The relationship between the absolute value of the difference and the preset threshold can be divided into two cases: the absolute value of the difference is greater than the preset threshold, and the absolute value of the difference is not greater than the preset threshold. In both cases, if the absolute value of the difference is greater than the preset threshold, the granularity of the dataset to be classified is determined to be a fine-grained dataset. If the absolute value of the difference is not greater than the preset threshold, the granularity of the dataset to be classified is determined to be a coarse-grained dataset.
[0061] According to the exemplary embodiments of this disclosure, as described above, the first classification model and the second classification model can be trained using the same training configuration and training hyperparameters. The training hyperparameters may include the number of training rounds, which can be at least one round. The first and second classification models each obtain a classification accuracy in each training round, that is, they each obtain a corresponding absolute value of the difference in each training round. This allows for the determination of the relationship between the absolute value of the difference in that training round and a preset threshold. Based on this, the exemplary embodiments of this disclosure can first complete all the preset number of training rounds to obtain the absolute values of the corresponding number of differences. Then, the absolute values of the obtained differences are compared with a preset threshold. If an absolute value of a difference is found to be greater than the preset threshold, the granularity classification result of the dataset to be classified is determined to be a fine-grained dataset; otherwise, it is a coarse-grained dataset.
[0062] However, in actual training and testing, there may be situations where the absolute value of the difference corresponding to previous training rounds is greater than a preset threshold. To save time and computational resources, an exemplary embodiment of this disclosure can stop training and testing after a training round in which the granularity classification result can be determined. Specifically, it can be as follows: First, at least one round of training and testing can be performed on the neural network model to obtain a first classification model and a second classification model. Then, after each round of training and testing, the absolute value of the difference can be obtained and compared with a preset threshold. Once it is determined that the absolute value of the difference obtained after a certain round of training and testing is greater than the preset threshold, the granularity classification result of the dataset to be classified is determined to be a fine-grained dataset, and training and testing are stopped. According to the dataset granularity classification method of this disclosure, stopping training and testing after a training round in which the granularity classification result can be determined can save an average of 50% of the time, based on experimental results on multiple existing datasets.
[0063] It should be noted here that when the first resolution is greater than the second resolution, the absolute value of the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model in the above process can be replaced with the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model to execute the dataset granularity classification method in the exemplary embodiment of this disclosure.
[0064] According to exemplary embodiments of this disclosure, as described above, the first classification model and the second classification model can be trained using the same training configuration and training hyperparameters, including the learning rate. However, in the field of machine learning, the learning rate may vary for each training session depending on the training data, the model used, etc. Therefore, exemplary embodiments of this disclosure can also automatically search for the optimal learning rates used during training for the first classification model and the second classification model. Specifically, this can be configured to pre-train the neural network models using the first dataset and the second dataset respectively, obtaining the first learning rate used by the first classification model and the second learning rate used by the second classification model during training.
[0065] Below, based on the existing coarse-grained dataset MINC2500 and fine-grained dataset cub_200_2011, a dataset granularity classification method according to an exemplary embodiment of this disclosure will be described in detail through specific examples. Figure 2 This is an overall flowchart illustrating another dataset granularity classification method according to an exemplary embodiment.
[0066] refer to Figure 2First, the training configuration and training hyperparameters can be determined. The training configuration may include, but is not limited to, the neural network model used, the GPU card used for training, the split ratio of the dataset to be classified, and the preset threshold. The training hyperparameters may include, but are not limited to, the learning rate, batch size, number of training rounds, data augmentation strategy, optimizer, learning rate adjustment strategy, and loss function used.
[0067] Then, the dataset to be classified can be obtained, and based on the splitting ratio of the dataset to be classified, it can be split into a training dataset and a test dataset.
[0068] Next, all images in the dataset to be classified can be adjusted to a resolution of 448 (448×448) to obtain the first dataset; then, all images in the dataset to be classified can be adjusted to a resolution of 224 (224×224) to obtain the second dataset. It should be noted that the first dataset includes a first training dataset based on the training dataset and a first test dataset based on the test dataset, and the second dataset includes a second training dataset based on the training dataset and a second test dataset based on the test dataset.
[0069] Next, the neural network model can be trained on the first training dataset and the second training dataset using the determined training configuration and training hyperparameters to obtain the first classification model and the second classification model.
[0070] Then, the first classification model can be tested based on the first test dataset to obtain the first test result, and the second classification model can be tested based on the second test dataset to obtain the second test result.
[0071] Then, based on the first test results, the classification accuracy of the first classification model can be obtained, and based on the second test results, the classification accuracy of the second classification model can be obtained.
[0072] Finally, the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model can be obtained. Based on the difference and the preset threshold, the granular classification result of the dataset to be classified can be obtained.
[0073] Here, the first and second classification models can be trained iteratively. Based on a pre-defined number of training rounds, the first training and testing are performed on the first dataset, followed by the first training and testing on the second dataset. The difference between the first training rounds is obtained and compared to a preset threshold. If the difference is greater than the preset threshold, the dataset to be classified is determined to be a fine-grained dataset, and training and testing stop. Otherwise, the next training round is executed in the order of first dataset, then second dataset, using the same judgment rule, until all the pre-defined number of training rounds has been completed. If the difference for all training rounds is not greater than the preset threshold, the dataset to be classified is determined to be a coarse-grained dataset.
[0074] It should be noted that the above process involves training and testing in the order of the first dataset first and then the second dataset. However, this disclosure is not limited to this. It also protects the situation where training and testing are performed in the order of the second dataset first and then the first dataset, as well as the situation where the first dataset and the second dataset are trained and tested in parallel. The execution process can be referred to the above process, and will not be repeated here.
[0075] based on Figure 2 The illustrated process can be used to experiment with the coarse-grained dataset MINC2500 and the fine-grained dataset cub_200_2011. Specifically, it can be used as the coarse-grained dataset MINC2500 and the fine-grained dataset cub_200_2011, respectively, to perform granular classification. It should be noted that the specific granular classification process can be implemented as follows... Figure 2 The process will not be elaborated here.
[0076] Table 1 shows the coarse-grained dataset used by MINC2500. Figure 2 The table below shows the classification accuracy of the first and second classification models used in the experiment with the illustrated process. Table 2 shows the classification accuracy of the fine-grained dataset cub_200_2011. Figure 2 The table shows the classification accuracy of the first and second classification models used in the experiment, as illustrated. The training run consisted of 8 rounds.
[0077] As shown in Tables 1 and 2, in all training epochs of the coarse-grained dataset MINC2500, the classification accuracy of the first classification model is worse than that of the second classification model. However, in training epoch 2 of the fine-grained dataset cub_200_2011, the classification accuracy of the first classification model is significantly better than that of the second classification model.
[0078]
[0079] Table 1
[0080]
[0081] Table 2
[0082] Figure 3 This is a block diagram illustrating a dataset granularity classification apparatus according to an exemplary embodiment. (Reference) Figure 3 The dataset granularity classification device 300 includes an acquisition unit 301, an adjustment unit 302, a training unit 303, and a classification unit 304.
[0083] The acquisition unit 301 can acquire the dataset to be classified, wherein the dataset to be classified includes at least one image.
[0084] According to exemplary embodiments of this disclosure, the dataset to be classified can be an image dataset to be classified into a coarse-grained dataset or a fine-grained dataset. Images in the dataset to be classified can be used for image classification. Images in the dataset to be classified can carry category labels, which can be labels used to distinguish categories when performing image classification.
[0085] The adjustment unit 302 can obtain a first dataset and a second dataset based on the dataset to be classified. The first dataset is obtained by adjusting all images in the dataset to be classified to a first resolution, and the second dataset is obtained by adjusting all images in the dataset to be classified to a second resolution.
[0086] According to an exemplary embodiment of this disclosure, the first resolution and the second resolution can be a higher resolution and a lower resolution that form a distinct contrast. For example, the adjustment unit 302 can set the first resolution to 448 (448×448) and the second resolution to 224 (224×224), or the first resolution to 224 (224×224) and the second resolution to 448 (448×448).
[0087] According to an exemplary embodiment of this disclosure, in order to better train and test the neural network model, the dataset to be classified acquired by the acquisition unit 301 may be pre-split into a training dataset and a test dataset, for example, using an 8:2 or 7:3 ratio to split the training dataset and the test dataset. Based on this, the first dataset obtained by the adjustment unit 302 may include a first training dataset obtained based on the training dataset and a first test dataset obtained based on the test dataset; the second dataset may include a second training dataset obtained based on the training dataset and a second test dataset obtained based on the test dataset.
[0088] After obtaining the first and second datasets, the neural network model can be trained using training unit 303. When training and testing with the first and second datasets respectively, the same training configuration and hyperparameters can be used; that is, the first classification model and the second classification model can be trained using the same training configuration and hyperparameters. It should be noted that the training configuration may include, but is not limited to, the neural network model used and the graphics card (GPU) used for training; the training hyperparameters may include, but are not limited to, the learning rate, the batch size selected in a single training epoch, the number of training epochs, data augmentation strategies, the optimizer, the learning rate adjustment strategy, and the loss function used. For example, the neural network model used may be, but is not limited to, ResNet50 or ResNet-101, and the number of training epochs may be, but is not limited to, 20.
[0089] Training unit 303 can use the first dataset to train and test the neural network model to obtain a first classification model for image classification and the classification accuracy of the first classification model. It can also use the second dataset to train and test the neural network model to obtain a second classification model for image classification and the classification accuracy of the second classification model.
[0090] According to an exemplary embodiment of this disclosure, in order to obtain the first classification model and the classification accuracy of the first classification model, the training unit 303 can first train the neural network model based on the first training dataset to obtain the first classification model.
[0091] Then, training unit 303 can test the first classification model based on the first test dataset to obtain the first test result.
[0092] As can be seen here, the first classification model is used for image classification. Therefore, the images in the first test dataset can be predicted using the first classification model to obtain the first test result, which may include the classification results of the images in the first test dataset.
[0093] Finally, training unit 303 can obtain the classification accuracy of the first classification model based on the first test results.
[0094] Here, the training unit 303 can obtain the first accuracy of classifying the images in the first test dataset based on the first test result and the category labels carried by the images in the first test dataset, and use the first accuracy as the classification accuracy of the first classification model.
[0095] For example, the training unit 303 can use the number of images contained in the first test dataset as the denominator, the number of images with the same class label as the first test result as the numerator, and the score obtained based on the numerator and denominator as the first accuracy.
[0096] According to an exemplary embodiment of this disclosure, in order to obtain the classification accuracy of the second classification model and the second classification model, the training unit 303 can first train the neural network model based on the second training dataset to obtain the second classification model.
[0097] Here, the training unit 303 can be trained using the same training configuration and training hyperparameters as in the training of the neural network model based on the first training dataset described above.
[0098] Then, training unit 303 can test the second classification model based on the second test dataset to obtain the second test result.
[0099] As we know here, the second classification model is used for image classification. Therefore, the second classification model can be used to predict the images in the second test dataset to obtain the second test result, which can include the classification results of the images in the second test dataset.
[0100] Finally, training unit 303 can obtain the classification accuracy of the second classification model based on the second test results.
[0101] Here, the training unit 303 can obtain the second accuracy of classifying the images in the second test dataset based on the second test results and the category labels carried by the images in the second test dataset, and use the second accuracy as the classification accuracy of the second classification model.
[0102] For example, training unit 303 can use the number of images contained in the second test dataset as the denominator, the number of images with the same class label as the second test result as the numerator, and the score obtained based on the numerator and denominator as the second accuracy.
[0103] Classification unit 304 can obtain the granular classification result of the dataset to be classified based on the classification accuracy of the first classification model, the classification accuracy of the second classification model, and a preset threshold.
[0104] According to an exemplary embodiment of this disclosure, classification unit 304 can subtract the classification accuracy corresponding to a higher resolution dataset from the classification accuracy corresponding to a lower resolution dataset, compare the obtained difference with a preset threshold, and obtain a granular classification result based on the relationship between the two. If the difference is greater than the preset threshold, classification unit 304 determines that the granular classification result of the dataset to be classified is a fine-grained dataset. If the difference is not greater than the preset threshold, classification unit 304 determines that the granular classification result of the dataset to be classified is a coarse-grained dataset. For an exemplary embodiment of this disclosure, the first resolution may be higher or lower than the second resolution. Based on this, classification unit 304 can calculate the absolute value of the difference between the two classification accuracies. Specifically, firstly, classification unit 304 can obtain the absolute value of the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model based on the classification accuracy of the first classification model and the classification accuracy of the second classification model; then, classification unit 304 can obtain the granular classification result of the dataset to be classified based on the absolute value of the difference and the preset threshold.
[0105] Regarding the relationship between the absolute value of the difference and the preset threshold, it can be divided into two cases: the absolute value of the difference is greater than the preset threshold, and the absolute value of the difference is not greater than the preset threshold. In both cases, if the absolute value of the difference is greater than the preset threshold, classification unit 304 determines that the granularity classification result of the dataset to be classified is a fine-grained dataset. If the absolute value of the difference is not greater than the preset threshold, classification unit 304 determines that the granularity classification result of the dataset to be classified is a coarse-grained dataset.
[0106] According to the exemplary embodiments of this disclosure, as described above, the first classification model and the second classification model can be trained by the training unit 303 using the same training configuration and training hyperparameters. The training hyperparameters may include the number of training rounds, which may be at least one round. The first classification model and the second classification model each obtain a classification accuracy in each training round, meaning that the training unit 303 obtains a corresponding absolute value of the difference in each training round. Subsequently, the classification unit 304 can determine the relationship between the absolute value of the difference corresponding to that training round and a preset threshold. Based on this, the exemplary embodiments of this disclosure can first have the training unit 303 complete all the preset number of training rounds to obtain the absolute values of the corresponding number of differences. Then, the classification unit 304 compares the obtained absolute values of the differences with a preset threshold. If it finds that the absolute value of a difference is greater than the preset threshold, the classification unit 304 determines that the granularity classification result of the dataset to be classified is a fine-grained dataset; otherwise, it is a coarse-grained dataset.
[0107] However, in actual training and testing, there may be situations where the absolute value of the difference corresponding to previous training rounds is greater than a preset threshold. To save time and computational resources, the exemplary embodiment of this disclosure can stop training and testing after the training round in which the granularity classification result can be determined. Specifically, the training unit 303 can be set to perform at least one round of training and testing on the neural network model to obtain a first classification model and a second classification model. Then, the classification unit 304 can obtain the absolute value of the difference after each round of training and testing and compare the absolute value of the difference with a preset threshold. Once it is determined that the absolute value of the difference obtained after a certain round of training and testing is greater than the preset threshold, the granularity classification result of the dataset to be classified is determined to be a fine-grained dataset, and training and testing are stopped. According to the dataset granularity classification device of this disclosure, stopping training and testing after the training round in which the granularity classification result can be determined can save an average of 50% of the time, based on experimental results on multiple existing datasets.
[0108] It should be noted here that when the first resolution is greater than the second resolution, the absolute value of the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model in the above-mentioned device unit can be replaced with the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model to execute the dataset granularity classification device in the exemplary embodiment of this disclosure.
[0109] According to the exemplary embodiments of this disclosure, as described above, the first classification model and the second classification model can be trained by the training unit 303 using the same training configuration and training hyperparameters, which may include the learning rate. However, in the field of machine learning, the learning rate may vary for each training session depending on the training data, the model used, etc. Therefore, the exemplary embodiments of this disclosure can also automatically search for the optimal learning rate used by the first classification model and the second classification model during training. Specifically, this can be configured to include a pre-training unit, which can use the first dataset and the second dataset to pre-train the neural network models respectively, obtaining the first learning rate used by the first classification model and the second learning rate used by the second classification model during training.
[0110] Figure 4 This is a block diagram of an electronic device 400 according to an exemplary embodiment.
[0111] Reference Figure 4The electronic device 400 includes at least one memory 401 and at least one processor 402. The at least one memory 401 stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by the at least one processor 402, a dataset granularity classification method according to an exemplary embodiment of the present disclosure is performed.
[0112] As an example, electronic device 400 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, electronic device 400 is not necessarily a single electronic device, but may be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 400 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.
[0113] In electronic device 400, processor 402 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.
[0114] The processor 402 can execute instructions or code stored in the memory 401, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transmission protocol.
[0115] The memory 401 may be integrated with the processor 402, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 401 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory 401 and the processor 402 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 402 to read files stored in the memory.
[0116] In addition, the electronic device 400 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device 400 can be interconnected via a bus and / or network.
[0117] According to exemplary embodiments of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are executed by at least one processor, they cause at least one processor to perform a dataset granularity classification method according to exemplary embodiments of the present disclosure. Examples of computer-readable storage media herein include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.
[0118] According to the dataset granularity classification method and apparatus disclosed herein, images in the dataset to be classified are adjusted to two different resolutions. Then, the same neural network model is used to train and test on both resolution datasets. This allows for obtaining the classification accuracy of the model trained on the two resolution datasets. These two classification accuracies can then be compared with a preset threshold for granularity to obtain the granularity classification result of the dataset to be classified. Based on this process, automatic determination of dataset granularity can be achieved, demonstrating universality.
[0119] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0120] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A dataset granularity classification method, characterized in that, include: Obtain a dataset to be classified, wherein the dataset includes at least one image; Based on the dataset to be classified, a first dataset and a second dataset are obtained, wherein the first dataset is obtained by adjusting all images in the dataset to be classified to a first resolution, and the second dataset is obtained by adjusting all images in the dataset to be classified to a second resolution; The neural network model is trained and tested using the first dataset to obtain a first classification model for image classification and the classification accuracy of the first classification model. The neural network model is then trained and tested using the second dataset to obtain a second classification model for image classification and the classification accuracy of the second classification model. Based on the classification accuracy of the first classification model, the classification accuracy of the second classification model, and a preset threshold, the granularity classification result of the dataset to be classified is obtained, wherein the granularity classification result indicates whether the dataset to be classified is a fine-grained dataset or a coarse-grained dataset. The classification accuracy of the first classification model is obtained as follows: based on the first test result and the category labels carried by the images in the first test dataset, the first accuracy of classifying the images in the first test dataset is obtained, and the first accuracy is used as the classification accuracy of the first classification model. The first test result is obtained by testing the first classification model based on the first test dataset, and the first test result includes the classification results of the images in the first test dataset. The classification accuracy of the second classification model is obtained as follows: based on the category labels carried by the images in the second test dataset and the second test results, the second accuracy of classifying the images in the second test dataset is obtained, and the second accuracy is used as the classification accuracy of the second classification model. The second test results are obtained by testing the second classification model on the second test dataset, and the second test results include the classification results of the images in the second test dataset.
2. The dataset granularity classification method as described in claim 1, characterized in that, The dataset to be classified was pre-split into a training dataset and a test dataset; The first dataset includes a first training dataset obtained based on the training dataset and a first test dataset obtained based on the test dataset, and the second dataset includes a second training dataset obtained based on the training dataset and a second test dataset obtained based on the test dataset.
3. The dataset granularity classification method as described in claim 2, characterized in that, The step of training and testing the neural network model using the first dataset to obtain a first classification model for image classification and the classification accuracy of the first classification model includes: The neural network model is trained based on the first training dataset to obtain the first classification model; The first classification model is tested based on the first test dataset to obtain the first test result; Based on the first test results, the classification accuracy of the first classification model is obtained; The step of training and testing the neural network model using the second dataset to obtain a second classification model for image classification and the classification accuracy of the second classification model includes: The neural network model is trained based on the second training dataset to obtain the second classification model; The second classification model was tested based on the second test dataset to obtain the second test results; Based on the results of the second test, the classification accuracy of the second classification model is obtained.
4. The dataset granularity classification method as described in claim 1, characterized in that, The first classification model and the second classification model were trained using the same training configuration and training hyperparameters.
5. The dataset granularity classification method as described in claim 1, characterized in that, The step of obtaining the granular classification result of the dataset to be classified based on the classification accuracy of the first classification model, the classification accuracy of the second classification model, and a preset threshold includes: Based on the classification accuracy of the first classification model and the classification accuracy of the second classification model, the absolute value of the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model is obtained; The granular classification result of the dataset to be classified is obtained based on the absolute value of the difference and the preset threshold.
6. The dataset granularity classification method as described in claim 5, characterized in that, The step of obtaining the granular classification result of the dataset to be classified based on the absolute value of the difference and the preset threshold includes: If the absolute value of the difference is greater than the preset threshold, the granularity classification result of the dataset to be classified is determined to be a fine-grained dataset. If the absolute value of the difference is not greater than the preset threshold, the granularity classification result of the dataset to be classified is determined to be a coarse-grained dataset.
7. The dataset granularity classification method as described in claim 6, characterized in that, At least one round of training and testing is performed on the neural network model to obtain the first classification model and the second classification model; Wherein, when the absolute value of the difference is greater than the preset threshold, determining the granularity classification result of the dataset to be classified as a fine-grained dataset includes: After each round of training and testing, the absolute value of the difference is obtained once, and the absolute value of the difference is compared with the preset threshold. Once it is determined that the absolute value of the difference obtained after a certain round of training and testing is greater than the preset threshold, the granular classification result of the dataset to be classified is determined to be a fine-grained dataset, and training and testing are stopped.
8. A dataset granularity classification device, characterized in that, include: The acquisition unit is configured to: acquire a dataset to be classified, wherein the dataset to be classified includes at least one image; The adjustment unit is configured to: obtain a first dataset and a second dataset based on the dataset to be classified, wherein the first dataset is obtained by adjusting all images in the dataset to be classified to a first resolution, and the second dataset is obtained by adjusting all images in the dataset to be classified to a second resolution; The training unit is configured to: train and test the neural network model using the first dataset to obtain a first classification model for image classification and the classification accuracy of the first classification model; and train and test the neural network model using the second dataset to obtain a second classification model for image classification and the classification accuracy of the second classification model. The classification unit is configured to: obtain the granular classification result of the dataset to be classified based on the classification accuracy of the first classification model, the classification accuracy of the second classification model, and a preset threshold, wherein the granular classification result indicates whether the dataset to be classified is a fine-grained dataset or a coarse-grained dataset; The training unit is configured to obtain the classification accuracy of the first classification model in the following manner: based on the first test result and the category labels carried by the images in the first test dataset, a first accuracy rate for classifying the images in the first test dataset is obtained, and the first accuracy rate is used as the classification accuracy of the first classification model. The first test result is obtained by testing the first classification model based on the first test dataset, and the first test result includes the classification results of the images in the first test dataset. The training unit is configured to obtain the classification accuracy of the second classification model in the following manner: based on the second test result and the category labels carried by the images in the second test dataset, a second accuracy for classifying the images in the second test dataset is obtained, and the second accuracy is used as the classification accuracy of the second classification model, wherein the second test result is obtained by testing the second classification model based on the second test dataset, and the second test result includes the classification results of the images in the second test dataset.
9. The dataset granularity classification device as described in claim 8, characterized in that, The dataset to be classified was pre-split into a training dataset and a test dataset; The first dataset includes a first training dataset obtained based on the training dataset and a first test dataset obtained based on the test dataset, and the second dataset includes a second training dataset obtained based on the training dataset and a second test dataset obtained based on the test dataset.
10. The dataset granularity classification device as described in claim 9, characterized in that, The training unit is configured as follows: The neural network model is trained based on the first training dataset to obtain the first classification model; The first classification model is tested based on the first test dataset to obtain the first test result; Based on the first test results, the classification accuracy of the first classification model is obtained; The neural network model is trained based on the second training dataset to obtain the second classification model; The second classification model was tested based on the second test dataset to obtain the second test results; Based on the results of the second test, the classification accuracy of the second classification model is obtained.
11. The dataset granularity classification device as described in claim 8, characterized in that, The first classification model and the second classification model were trained using the same training configuration and training hyperparameters.
12. The dataset granularity classification device as described in claim 8, characterized in that, The classification unit is configured as follows: Based on the classification accuracy of the first classification model and the classification accuracy of the second classification model, the absolute value of the difference between the classification accuracy of the first classification model and the classification accuracy of the second classification model is obtained; The granular classification result of the dataset to be classified is obtained based on the absolute value of the difference and the preset threshold.
13. The dataset granularity classification device as described in claim 12, characterized in that, The classification unit is configured as follows: If the absolute value of the difference is greater than the preset threshold, the granularity classification result of the dataset to be classified is determined to be a fine-grained dataset. If the absolute value of the difference is not greater than the preset threshold, the granularity classification result of the dataset to be classified is determined to be a coarse-grained dataset.
14. The dataset granularity classification device as described in claim 13, characterized in that, At least one round of training and testing is performed on the neural network model to obtain the first classification model and the second classification model; The classification unit is configured as follows: After each round of training and testing, the absolute value of the difference is obtained once, and the absolute value of the difference is compared with the preset threshold. Once it is determined that the absolute value of the difference obtained after a certain round of training and testing is greater than the preset threshold, the granular classification result of the dataset to be classified is determined to be a fine-grained dataset, and training and testing are stopped.
15. An electronic device, characterized in that, include: At least one processor; At least one memory that stores computer-executable instructions. The computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the dataset granularity classification method as described in any one of claims 1 to 7.
16. A computer-readable storage medium for storing instructions, characterized in that, When the instruction is executed by at least one processor, it causes the at least one processor to perform the dataset granularity classification method as described in any one of claims 1 to 7.