A method for constructing a high information quality image dataset

Abstract

The application discloses a method for constructing a high-information-quality image dataset. The method comprises the following steps: (1) dividing a dataset into a pool dataset and a test set, selecting 10% of the pool dataset as an initial labeled dataset, and selecting the remaining data as a candidate dataset; inputting the initial labeled dataset and the candidate dataset into a feature extraction network respectively, and performing feature extraction; (2) calculating the intra-class information richness of the initial labeled dataset; (3) calculating the inter-class information overlap of the candidate dataset; and (4) using the images with high inter-class information overlap and the initial labeled dataset to construct a high-information-quality image dataset. The method for constructing the high-information-quality image dataset can analyze high-dimensional feature characteristics of images, and provides an important reference for a subsequent inter-class information overlap measurement method for to-be-evaluated images.

Description

TECHNICAL FIELD

[0001] The present application relates to a method for constructing a high information quality image dataset. BACKGROUND

[0002] A large amount of labeled data enables intelligent perception algorithms represented by deep learning to play an important role in wide application. However, there are still many fields where data acquisition is difficult and labeling costs are high, and data has become an important bottleneck hindering the development of deep learning in this field. Blindly expanding data will introduce a large amount of redundant data, and the gain of intelligent perception tasks is limited, but the demand for hardware devices is actually considerable. Therefore, to break the bottleneck of image intelligent perception algorithms in fields where data acquisition is difficult and labeling costs are high, image augmentation cannot be blindly performed, and research on image information quality evaluation methods must be strengthened to provide scientific guidance for dataset construction.

[0003] Traditional image quality evaluation is from the perspective of human senses, including distortion discrimination and aesthetic evaluation, and the development of these technologies has provided help in the process of image processing and human interaction, but has not provided guidance from the perspective of intelligent perception algorithms. Image information quality evaluation methods related to intelligent perception algorithms mostly rely on active learning algorithms. The core idea of active learning is that the model actively selects images for labeling and training, which can obtain similar or even better results than labeling and training with a large number of randomly selected images.

[0004] Existing active learning algorithms can be roughly divided into uncertainty-based and model parameter change-based methods. The core idea of the uncertainty-based method is to select images that the network cannot easily distinguish for training; the core idea of the model parameter change-based method is to select images that have a greater impact on the model, such as reducing the loss value more. These technologies do not analyze the image information quality in essence, so the evaluation of image information quality is often inaccurate and cannot provide strong guidance for image dataset construction.

[0005] Therefore, the present application proposes a new method for constructing a high information quality image dataset, which can well guide the construction of intelligent perception datasets. SUMMARY

[0006] The present application aims to provide a method for constructing a high information quality image dataset, which analyzes the high-dimensional feature characteristics of images, and the method includes image intra-class richness measurement and image inter-class overlap measurement. The measurement of image intra-class richness provides an important reference for the subsequent inter-class information overlap measurement method of the image to be evaluated.

[0007] In order to achieve the above-mentioned purpose, the technical scheme adopted is:

[0008] A method for constructing a high-information-quality image dataset, comprising the following steps:

[0009] (1) extracting image features:

[0010] The dataset is divided into a pool dataset and a test set, 10% of the pool dataset is selected as an initial labeled dataset, and the remaining data is selected as a candidate dataset;

[0011] The initial labeled dataset and the candidate dataset are input into a feature extraction network, respectively, for feature extraction;

[0012] (2) calculating the intra-class information richness of the initial labeled dataset;

[0013] (3) calculating the inter-class information overlap of the candidate dataset;

[0014] (4) using the images with high inter-class information overlap and the initial labeled dataset to construct a high-information-quality image dataset.

[0015] Further, in step (1), the dataset is divided into a pool dataset and a test set in a ratio of 5:1.

[0016] Further, in step (1), the feature extraction network, the feature extraction backbone network selects ResNet-18, and the initial labeled dataset and the test set are used to optimize the backbone network to obtain the feature extraction network.

[0017] Further, in step (1), 512-dimensional feature extraction is performed.

[0018] Further, in step (2), the method for calculating the intra-class information richness of the initial labeled dataset is as follows: first, calculate the prototype of each class of images in the initial labeled dataset, then calculate the mean distance of the image features to the prototype of the class, and finally obtain the intra-class information richness of the dataset images.

[0019] Further, in step (2), the prototype of each class of images in the initial labeled dataset is calculated using the following formula:

[0020]

[0021] where p i is the prototype of the i-th class of images, is the k-th image feature of the i-th class, and n i is the number of images in the i-th class.

[0022] Further, in step (2), the mean distance of the image features to the prototype of the class is calculated as follows: The following formula is used:

[0023]

[0024] In the step (2), the intra-class information richness I of the data set image is calculated according to the following formula: I The following formula is used:

[0025]

[0026] Further, in the step (3), the method for calculating the inter-class information overlap of the image in the candidate data set is as follows: first, the distance of the image in the candidate data set to each class prototype is calculated, and then the inter-class overlap of the image in the candidate data set is determined to obtain the inter-class information overlap of the image in the candidate data set.

[0027] Further, in the step (3), the distance of the image in the candidate data set to each class prototype is calculated using the following formula:

[0028]

[0029] wherein, represents the qth image of the jth class, represents its high-dimensional feature, represents the distance of the image to the prototype of the ith class image.

[0030] Further, in the step (3), the inter-class overlap of the image in the candidate data set is determined The following formula is used:

[0031]

[0032] The inter-class information overlap of the image in the candidate data set is obtained The following formula is used:

[0033]

[0034] wherein, n l is the number of image classes in the data set.

[0035] Compared with the prior art, the present application has the following beneficial effects:

[0036] 1. The technical scheme of the present application is directly aimed at the neural network intelligent perception algorithm, and uses the high-dimensional feature vector in the neural network intelligent perception algorithm. The high information quality image evaluated is more consistent with the neural network intelligent perception rule, and better results can be achieved.

[0037] 2、The technical scheme of the present application is to analyze the essential law of the high-dimensional feature vector of an image in a high-dimensional feature space, directly analyze the decision boundary information of intelligent perception algorithm, evaluate the image near the decision boundary which is more conducive to neural network intelligent perception as a high information quality image, and thus better results can be achieved. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 A flowchart of the method for constructing a high information quality image data set. DETAILED DESCRIPTION

[0039] In order to further illustrate the method for constructing a high information quality image data set according to the present application and achieve the intended purpose of the present application, the method for constructing a high information quality image data set according to the present application, its specific implementation, structure, features and effects will be described in detail below in combination with preferred embodiments. In the following description, different "an embodiment" or "embodiments" do not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0040] The method for constructing a high information quality image data set according to the present application will be further described in detail below in combination with specific embodiments:

[0041] Since the existing image information quality evaluation method for neural network intelligent perception is lacking and does not analyze the image information quality from the essence, the present application designs a high information quality image data set construction method based on inter-class and intra-class characteristics for neural network intelligent perception. The method analyzes the high-dimensional feature characteristics of an image and includes image intra-class richness measurement and image inter-class overlap measurement. The measurement of the intra-class richness of an image provides important parameters for the subsequent inter-class information overlap measurement method of the image to be evaluated. If the image to be evaluated overlaps in more classes, the information quality of the image is higher. If such an image is labeled, the information quality of the image data set as a whole can be improved better.

[0042] The technical scheme of the present application is:

[0043] A method for constructing a high information quality image data set, comprising the following steps:

[0044] (1) Extracting image features:

[0045] Divide the data set into a pool data set and a test set, select 10% of the pool data set as an initial labeled data set, and the remaining data as a candidate data set;

[0046] Input the initial labeled data set and the candidate data set into a feature extraction network respectively for feature extraction;

[0047] (2) calculating the intra-class information richness of images in the initial labeled dataset;

[0048] (3) calculating the inter-class information overlap of images in the candidate dataset;

[0049] (4) using images with high inter-class information overlap and the initial labeled dataset to construct a high information quality image dataset.

[0050] Preferably, in the step (1), the dataset is divided into a pool dataset and a test set according to a ratio of 5:1.

[0051] Preferably, in the step (1), the feature extraction network is obtained by optimizing a main network of the feature extraction using the initial labeled dataset and the test set, and the main network of the feature extraction is ResNet-18.

[0052] Preferably, in the step (1), 512-dimensional feature extraction is performed.

[0053] Preferably, in the step (2), the method for calculating the intra-class information richness of images in the initial labeled dataset is as follows: first, calculating prototypes of each class of images in the initial labeled dataset, and then calculating the mean of the distance from the image features to the prototype of the class of images to obtain the intra-class information richness of each class of images in the dataset.

[0054] Further preferably, in the step (2), the prototypes of each class of images in the initial labeled dataset are calculated using the following formula:

[0055]

[0056] wherein, p i is the prototype of the i-th class of images, is the k-th image feature of the i-th class, and n i is the number of images of the i-th class.

[0057] Further preferably, in the step (2), the mean of the distance from the image features to the prototype of the class of images is calculated using the following formula:

[0058]

[0059] In the step (2), the intra-class information richness I I of each class of images in the dataset is calculated using the following formula:

[0060]

[0061] ​Preferably, in step (3), the method for calculating the inter-class information overlap of images in the candidate dataset is as follows: first, calculate the distance of images in the candidate dataset to each class prototype, then determine the overlap of images in the candidate dataset between classes, and obtain the inter-class information overlap of images in the candidate dataset.

[0062] Further preferably, in step (3), the distance of images in the candidate dataset to each class prototype is calculated using the following formula:

[0063]

[0064] wherein, represents the qth image of the jth class, represents its high-dimensional features, represents the distance of the image to the prototype of the ith class image.

[0065] Further preferably, in step (3), the overlap of images in the candidate dataset between classes is determined using the following formula:

[0066]

[0067] to obtain the inter-class information overlap of images in the candidate dataset

[0068] wherein, n l is the number of image classes in the dataset.

[0069] Embodiment 1.

[0070] In combination with

[0071] , the specific operation steps are as follows: Figure 1 Step 1: Extract image features.

[0072] (1) Divide the dataset:

[0073] According to the ratio of 5:1, divide the dataset into a pool dataset and a test set, select 10% of the pool dataset as the initial labeled dataset, and the remaining data in the pool dataset as the candidate dataset.

[0074] (2) Train the feature extraction network.

[0075] The feature extraction backbone network is selected as ResNet-18, and the initial labeled dataset and the test set are used to optimize the backbone network to obtain the feature extraction network.

[0076] (3) Obtain image features.

[0077] ​​

[0078] The images in the initial labeled data set are input into the trained feature extraction network, and the network is used to extract 512-dimensional features from the initial labeled data. The images in the candidate data set are input into the trained feature extraction network, and the network is used to extract 512-dimensional features from the candidate images.

[0079] Step 2: Calculate the intra-class information richness of the initial labeled data set images.

[0080] (1) Calculate the prototype of each class of images in the initial labeled data set. The following formula is used:

[0081]

[0082] where p i is the prototype of the i-th class of images, is the k-th image feature of the i-th class, n i is the number of images in the i-th class.

[0083] (2) Calculate the mean of the distance from the image feature to the prototype of the class The following formula is used:

[0084]

[0085] (3) Get the intra-class information richness I I richness of the data set images. The following formula is used:

[0086]

[0087] Step 3, calculate the inter-class information overlap of the image to be evaluated (the image to be evaluated is the image in the candidate data set).

[0088] (1) Calculate the distance from the image to be evaluated to the prototype of each class using the following formula:

[0089]

[0090] where represents the q-th image of the j-th class, represents its high-dimensional feature, represents the distance from the image to the prototype of the i-th class of images.

[0091] (2) Determine the overlap of the image to be evaluated among each class The following formula is used:

[0092]

[0093] (3) Get the inter-class information overlap of the image to be evaluated The following formula is used:

[0094]

[0095] wherein n l is the number of image classes within the dataset.

[0096] Images with high inter-class information overlap are high information quality images.

[0097] Fourthly, a high-quality intelligent perception dataset is constructed.

[0098] (1) According to the data set collection and labeling budget, high information quality images are selected from the candidate dataset in batches, and are added to the initial labeled dataset after labeling.

[0099] The images in the candidate dataset all have an inter-class information overlap, which is sorted from high to low, and the images are selected for labeling from high to low in combination with the labeling budget.

[0100] (2) After the previous batch of data is selected, the selection of the next batch of data is repeated from the first step to the fourth step until the labeling budget is exhausted.

[0101] Fifthly, the construction effect of the high-quality intelligent perception dataset is tested.

[0102] To demonstrate the effectiveness of the image information quality evaluation method proposed in the present application, two public datasets, CIFAR10 and mini-ImageNet, are used for experiments and tests. In the test, the data in the test set are input into the classifier model trained with different datasets, and the corresponding class prediction results are compared with the real labels in the test set to calculate the prediction accuracy of the model.

[0103] The test results of the classification models trained by the high-quality dataset constructed according to the method of the present application and the randomly constructed dataset are compared. Each dataset is trained three times, and the test results are the average of the three test results. The experimental results show that under the same budget, the model training accuracy of the high-quality dataset constructed by the method of the present application is higher than that of the randomly constructed dataset. This shows that the image information quality evaluation method proposed in the present application can well guide the construction of intelligent perception datasets, and the image information quality evaluation method proposed in the present application is effective. The experimental results are shown in Table 1.

[0104] Table 1 Comparison of experimental results of high-quality dataset construction and random dataset construction of CIFAR10 dataset

[0105] Number of samples added on initial label Random dataset construction Dataset constructed by the invention 5000 84.79％ 87.58％ 10000 88.04％ 91.65％ 15000 89.15％ 91.92％ 20000 90.25％ 92.15％

[0106] Mini-ImageNet dataset

[0107]

[0108]

[0109] The above is only the preferred embodiment of the present application, and does not limit the present application in any form. Any simple modification, equivalent change and modification of the above embodiment according to the technical essence of the present application are still within the scope of the technical solution of the present application.

Claims

1. A method for constructing a high-information-quality image dataset, characterized in that, Includes the following steps: (1) Extracting image features: The dataset is divided into a pool dataset and a test set. 10% of the pool dataset is selected as the initial labeled dataset, and the remaining data is used as the candidate dataset. The initial labeled dataset and the candidate dataset are respectively input into the feature extraction network for feature extraction; (2) Calculate the intra-class information richness of images in the initial labeled dataset; The method for calculating the intra-class information richness of images in the initial labeled dataset is as follows: first, calculate the prototype of each class of images in the initial labeled dataset, then calculate the mean distance from the image features to the prototype of that class of images, and finally obtain the intra-class information richness of the dataset images. In the calculation of the initial labeled dataset, the various image prototypes are calculated using the following formula: Where, p i For the i-th type of image prototype, For the k-th image feature of the i-th class, n i Let i be the number of images of the i-th class; The average distance from the calculated image features to the prototype image of that type is... Use the following formula: The dataset image in-class information richness (INF) I Use the following formula: Where n is the number of image categories in the dataset; (3) Calculate the inter-class information overlap of the images in the candidate dataset; The method for calculating the inter-class information overlap of images in the candidate dataset is as follows: first, calculate the distance from the images in the candidate dataset to each class prototype, then determine the overlap between the images in the candidate dataset and each class, and finally obtain the inter-class information overlap of the images in the candidate dataset. The distance from each prototype class to an image in the candidate dataset is calculated using the following formula: in, This represents the q-th image of the j-th class. This indicates its high-dimensional features. p represents the distance from the image to the prototype of the i-th class of images. i This is the prototype of the i-th type of image; The method described above determines the overlap between different classes of images in the candidate dataset. Use the following formula: Among them, the aforementioned This represents the mean distance from image features to the prototype images of this type. Obtain the inter-class information overlap of images in the candidate dataset. Use the following formula: Where n is the number of image categories in the dataset; (4) Use the images with high inter-class information overlap and the initial labeled dataset to construct a high information quality image dataset.

2. The construction method according to claim 1, characterized in that, In step (1), the dataset is divided into a pool dataset and a test set in a 5:1 ratio.

3. The construction method according to claim 1, characterized in that, In step (1), the feature extraction network is selected as ResNet-18 as the backbone network. The backbone network is optimized using the initial labeled dataset and the test set to obtain the feature extraction network.

4. The construction method according to claim 1, characterized in that, In step (1), 512-dimensional feature extraction is performed.

Images