A method and device for classifying unbalanced image data based on dynamic sampling

Abstract

The present application relates to a method and apparatus for classifying unbalanced image data based on dynamic sampling, the method comprising S1: manually creating unbalanced image data; S2: normalizing and one-hot encoding the created unbalanced image data to obtain training samples; S3: constructing a basic model for image classification and freezing the model; S4: dynamically sampling training samples using the f1-score algorithm; S5: unfreezing the model and starting training using the data dynamically sampled by f1-score, and then outputting the loss results and the accuracy of each type of image recognition based on the confusion matrix. The present application generates a new dataset before the training process to implement offline data enhancement. The generated dataset is large enough to ensure that there are enough samples to resample classes with low F1 scores during the training phase. Compared with online data enhancement, offline data enhancement has the advantages of being easy to read, convenient for later implementation and adjustment, and generating visual images that help control the enhanced images.

Description

Technical Field

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a method and device for classifying unbalanced image data based on dynamic sampling. Background Art

[0002] The recognition and classification of imbalanced image data is a mainstream technology in current artificial intelligence. Numerous approaches exist for handling imbalanced samples, and the main technologies fall into three broad areas. At the data level, this involves processing the training dataset to mitigate the impact of class imbalance. At the algorithmic level, this involves improving existing deep learning algorithms by modifying loss functions or learning methods to mitigate the effects of class imbalance. Furthermore, hybrid approaches based on data and algorithms are also explored.

[0003] CNNs undoubtedly dominate the field of image recognition and classification today. However, because neural networks are highly data-driven, they are highly sensitive to class imbalance. Mainstream methods for addressing this imbalance, such as SMOTE and cost-sensitive matrix extraction, have their own drawbacks. The most significant drawback is that they increase the likelihood of overlap between classes, generating samples that lack meaningful information. Summary of the Invention

[0004] In response to the above technical problems, this application proposes a method and device for classifying unbalanced image data based on dynamic sampling.

[0005] First, this application proposes a method for classifying unbalanced image data based on dynamic sampling, including the following:

[0006] step:

[0007] S1: Manually create unbalanced image data;

[0008] S2: Normalize and one-hot encode the created unbalanced image data to obtain training samples;

[0009] S3: Build a basic image classification model and freeze the model;

[0010] S4: Use the f1-score algorithm to dynamically sample training samples;

[0011] S5: Unfreeze the model and start training using the data dynamically sampled by f1-score, and then output the loss results and the accuracy of each type of image recognition based on the confusion matrix;

[0012] S6: Use the trained model to classify unbalanced image data.

[0013] Preferably, the S1 specifically includes:

[0014] Read the CIFAR-10 dataset and manually convert each type of data into unbalanced data according to the ratio of [250, 750, 1250, 1750, 2250, 2750, 3250, 3750, 4250, 5000] as an example.

[0015] Preferably, S3 specifically includes:

[0016] S31: Establish the basic model structure according to the residual network resnet-50, remove the top model, and set the input_shape according to the parameter shape = (32, 32, 3);

[0017] S32: Change the new top layer to a flattened fully connected layer:

[0018] S33: Freeze the base model.

[0019] Preferably, the S4 includes:

[0020] S41: Dynamically update the data capacity of the current training sample according to the f1-score;

[0021] S42: Check whether the sample size updated by f1-score is larger than the imbalance manually set at the beginning. If so, perform image enhancement; otherwise, perform random sampling.

[0022] Preferably, in S4, the dynamic sampling algorithm is:

[0023] UpdateSampleSize=(F1 i , c j )

[0024]

[0025]

[0026] Preferably, in S4, a softmax activation function is used for activation, and parameters are tuned by SGD stochastic gradient descent.

[0027] Preferably, in S4, three sets of unbalanced data and one set of balanced data are used to perform parameter tuning.

[0028] Preferably, the three groups of unbalanced data are:

[0029] A=[250, 750, 1250, 1750, 2250, 2750, 3250, 3750, 4370, 5000];

[0030] B=[250, 250, 250, 500, 500, 500, 2500, 2500, 2500, 5000];

[0031] C=[500, 500, 500, 500, 500, 5000, 5000, 5000, 5000, 5000];

[0032] The set of balance data is:

[0033] D=[5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000].

[0034] Preferably, in S5, the number of images of different categories is manually changed to obtain the current F1 score after every 5 epochs, and resampling and model training are repeated in groups of 5 epochs.

[0035] In a second aspect, the present application also proposes a device for classifying unbalanced image data based on dynamic sampling, characterized in that the device comprises:

[0036] an unbalanced image data creation module configured to manually create unbalanced image data;

[0037] A preprocessing module is configured to normalize and one-hot encode the created unbalanced image data to obtain training samples;

[0038] The basic model building module is configured to build a basic image classification model and freeze the model;

[0039] Dynamic sampling module, configured to dynamically sample training samples using the f1-score algorithm;

[0040] The model training module is configured to unfreeze the model and start training using data dynamically sampled by the f1-score, and then output the loss results and the accuracy of each type of image recognition based on the confusion matrix;

[0041] The imbalanced image data classification module is configured to classify imbalanced image data using the trained model.

[0042] In summary, this application has at least the following beneficial technical effects:

[0043] 1. This application designs a set of algorithms designed for class-imbalanced image data based on a residual network with artificial dynamic sampling as the core. The algorithm dynamically updates the samples for each model training through the F1-score, and incorporates a method for sample image data enhancement to effectively classify class-imbalanced image data.

[0044] 2. This application generates a new dataset before training to implement offline data augmentation. The generated dataset is large enough to ensure that there are enough samples to resample classes with low F1 scores during training. Compared with online data augmentation, offline data augmentation has the advantages of being easier to read, convenient for later adjustments, and generating visual images that help control the augmented images.

[0045] 3. In order to increase the feature learning depth of minority groups, the resampling algorithm based on F1-score and augmentation in this application calculates the F1-Scores on the validation dataset of the class every five training epochs. These F1-level scores will be used as weights to determine the target number of samples for the next class training iteration. The number of target training samples is equal in all iterations. The F1 score of a class with a lower number of samples means that the class has learned less in the previous iteration. Therefore, more samples will be selected from this class in the next iteration. The proposed method adopts a new weight-based algorithm adjustment method to correct the class distribution bias in the imbalanced data classification task. In the method of this application, the classification performance of the minority class is improved by incrementally increasing the depth of the feature extraction training process of the minority group. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and, together with the description, serve to explain the principles of the present application. Other embodiments and many of the expected advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with respect to each other. Like reference numerals designate corresponding similar parts.

[0047] Figure 1 This is a flowchart of a method for classifying unbalanced image data based on dynamic sampling in this application.

[0048] Figure 2 This is a schematic diagram of a specific embodiment of the unbalanced image data classification method based on dynamic sampling that can be applied to the present application.

[0049] Figure 3 Schematic diagram of the CIFAR-10 dataset in one embodiment of the present application.

[0050] Figure 4 This is a schematic diagram of the results obtained using a traditional CNN-based model in one embodiment of the present application.

[0051] Figure 5 This is a schematic diagram of the results obtained after dynamic sampling of the F1-score in one embodiment of the present application.

[0052] Figure 6 This is a schematic diagram of the module structure of an unbalanced image data classification device based on dynamic sampling in one embodiment of the present application.

[0053] Figure 7 It is a structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0054] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0055] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0056] Figure 1 A flow chart of a method for classifying unbalanced image data based on dynamic sampling is shown in the present application. Figure 1 , the method specifically comprises the following steps:

[0057] S1: Manually create unbalanced image data;

[0058] Wherein, the S1 specifically includes:

[0059] Read the CIFAR-10 dataset and manually convert each type of data into unbalanced data according to the ratio of [250, 750, 1250, 1750, 2250, 2750, 3250, 3750, 4250, 5000] as an example.

[0060] S2: Normalize and one-hot encode the created unbalanced image data to obtain training samples;

[0061] S3: Build a basic image classification model and freeze the model;

[0062] S3 specifically includes:

[0063] S31: Establish the basic model structure according to the residual network resnet-50, remove the top model, and set the input_shape according to the parameter shape = (32, 32, 3);

[0064] S32: Change the new top layer to a flattened fully connected layer;

[0065] S33: Freeze the base model

[0066] S4: Use the f1-score algorithm to dynamically sample training samples;

[0067] Wherein, the S4 includes:

[0068] S41: Dynamically update the data capacity of the current training sample according to the f1-score;

[0069] S42: Check whether the sample size updated by f1-score is larger than the imbalance manually set at the beginning. If so, perform image enhancement, otherwise perform random sampling;

[0070] In S4, the dynamic sampling algorithm is:

[0071] UpdateSampleSize=(F1 i , c j )

[0072]

[0073]

[0074] In S4, a softmax activation function is used for activation, and parameters are tuned by SGD stochastic gradient descent.

[0075] In S4, parameter tuning is performed using three sets of unbalanced data and one set of balanced data.

[0076] The three sets of unbalanced data are:

[0077] A=[250, 750, 1250, 1750, 2250, 2750, 3250, 3750, 4370, 5000];

[0078] B=[250, 250, 250, 500, 500, 500, 2500, 2500, 2500, 5000];

[0079] C=[500, 500, 500, 500, 500, 5000, 5000, 5000, 5000, 5000];

[0080] A set of balanced data is:

[0081] D=[5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000].

[0082] S5: Unfreeze the model and start training using the data dynamically sampled by f1-score, and then output the loss results and the accuracy of each type of image recognition based on the confusion matrix;

[0083] In S5, the number of images of different categories is manually changed to obtain the current F1 score after the results of every 5 epochs, and the resampling and training model are repeated in groups of 5 epochs.

[0084] S6: Use the trained model to classify unbalanced image data.

[0085] In a specific embodiment, referring to Figure 2 The following is a detailed description of the imbalanced image data classification method based on dynamic sampling in this application:

[0086] This application mainly samples and models image data with image class imbalance to maximize the elimination of classification losses caused by class imbalance.

[0087] Reference Figure 3 The data used in this application is the CIFAR-10 dataset, which consists of 60,000 32x32 color images in 10 categories, with each category containing 6,000 images. There are 50,000 training images and 10,000 test images. Figure 3 The words on the left represent a total of ten types of image data. The image content of each type of data mainly contains all or part of the characteristics of the object or organism.

[0088] In a specific embodiment, the algorithm implementation steps are:

[0089] (1) Read the data and manually convert each type of data into unbalanced data according to the ratio of [250, 750, 1250, 1750, 2250, 2750, 3250, 3750, 4250, 5000] as an example.

[0090] (2) Normalize and uniquely encode the data to ensure that the data dimensions are in the same range and convert the class labels of the classification problem into a matrix for easy calculation. study Model building.

[0091] (3) Start building the model structure:

[0092] (3.1) Establish the basic model structure based on the residual network resnet-50, remove the top model, and set the input_shape according to the parameter shape = (32, 32, 3);

[0093] (3.2) Change the new top layer to a flattened fully connected layer;

[0094] (3.3) Freeze the model;

[0095] (4) Dynamically sample training samples based on the f1-score algorithm and iterate new training samples.

[0096] (4.1) Select softmax as the activation function and tune the parameters through SGD stochastic gradient descent

[0097] (4.2) Tuning parameters: To make the results more intuitive, we name the three sets of unbalanced data A, B, and C in the order shown above. The balanced data is represented by D.

[0098] A=[250, 750, 1250, 1750, 2250, 2750, 3250, 3750, 4370, 5000]

[0099] B=[250, 250, 250, 500, 500, 500, 2500, 2500, 2500, 5000]

[0100] C=[500, 500, 500, 500, 500, 5000, 5000, 5000, 5000, 5000]

[0101] D=[5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000]

[0102] (5) Determine whether to perform image enhancement to supplement the data based on the result of (4);

[0103] (6) Return to (3) to unfreeze the model and start training. The model will output the loss result and the recognition accuracy of each type of image based on the confusion matrix.

[0104] In a specific embodiment, the pseudo code of this algorithm is disclosed below:

[0105]

[0106] The core sampling method of this algorithm dynamically ensures the imbalanced class sample formula is:

[0107] UpdateSampleSize=(F1 i , c j )

[0108]

[0109] In the process of dynamic sampling, the F1-score method is used to calculate the sample weight and determine whether data enhancement is needed. Dynamic sampling is an algorithm that continuously updates the parameters. The number of samples of the current input image depends on the F 1i-1 The result of the last iteration. This dynamic process can be expressed by the following formula. From |Xtrain| trained Xi in sampling. The number of images of the current input sample depends on F 1i-1 The result of the last iteration. After training the current model, input a new image, that is, the label verification set, and calculate the current F 1i，j . j represents the image category in the jth i iteration, so we can also summarize F 1i Calculation of and j:

[0110]

[0111] Pre i，j and Rec i，j Is the iteration C j The class's precision and recall metrics. If a core class is present, it is more likely to be replaced by the model. Therefore, we should prefer samples with ground-level F1-scores. However, the total number of training images remains the same, meaning that the only changes in the above formula are the weight parameters.

[0112] Dynamic sampling is the most important module and the core sampling method of this application. At the beginning, the pre-trained ResNet can be considered as the initial model M s . Second, it uses an unbalanced training dataset to train the model M s and using the reference dataset X ref Next, offline data augmentation is applied to the target domain-imbalanced Cifar-10 dataset to generate more images, which are later selected as new training data X train Then, based on the initial model M s The F1 score is calculated by modifying the number of images for each class. The new training set is obtained from the data augmentation described above. The model is trained for 5 epochs to obtain the new model M1. Finally, the process is repeated for the next 5 epochs, with resampling and training. It is worth noting that in this application's method, the number of images for each class must be manually changed to obtain the current F1 score after every 5 epochs, which is why it is called manual dynamic sampling.

[0113] The following discloses the results of the performance verification of the unbalanced image data classification method based on dynamic sampling of the present application.

[0114] Figure 4 This is the result obtained based on the traditional CNN model. Figure 5 This is the result obtained after dynamic sampling of F1-score. Figure 4 , where loss: training set loss value; accuracy: training set accuracy; val_loss: test set loss value; val_accruacy: test set accuracy. In the original CNN-based model, we can see that the accuracy of the training set is a very stable straight line, and almost overlaps with the horizontal line close to 0. This shows that under extreme class imbalance samples, the accuracy of both the training set and the validation set is extremely low. Figure 5 ,Under the algorithm of this application, it can be seen that the accuracy of the training set and the test set is gradually improving, and the convergence speed of the loss function is significantly accelerated.

[0115] Refer to Table 1 and Table 2 below.

[0116] A=[250, 750, 1250, 1750, 2250, 2750, 3250, 3750, 4370, 5000]

[0117] D=[5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000]

[0118] For example, using the unbalanced data set A and the balanced data set D, we can see that even with extremely imbalanced samples, the algorithm in this project performs very similarly to the algorithm under balanced conditions. Even with a significant difference in sample size, the difference in model training results is between 0.6 and 0.7.

[0119]

[0120] Table 1

[0121]

[0122] Table 2

[0123] The following is a comparison of the effects under different data sets:

[0124] Dynamic sampling of imbalanced samples and data augmentation methods can effectively enhance model training efficiency. This algorithm was applied not only to the CIFAR-10 dataset but also to the MINST dataset. Table 4 shows the final training results for the same model on the MINST dataset. To demonstrate that our model achieves similar results on different datasets, we switched from CIFAR-10 to MNIST, maintaining the same model parameters. We also see some performance comparisons of the basic CNN model from Tables 3 to 4. Detailed comparison results are shown in the table. The primary purpose of MNIST data classification is to demonstrate the generalizability of our model. We compared it to a traditional CNN using a vertical orientation, accurately capturing the data. The model performed well on minimally imbalanced data, achieving a classification accuracy of 0.88 and approaching 100% accuracy for this project.

[0125]

[0126] Table 3. Training results of traditional CNN

[0127]

[0128] Table 4 Results of model training under MINST

[0129] In summary, the innovations of this application are mainly reflected in:

[0130] (1) Using ResNet-50 Enhanced Base Model Based on the original dynamic sampling method, this application changes the pre-trained VGGNet model to the new ResNet-50 model proposed by He et al. (2016). This model is based on the deep learning residual network framework.

[0131] (2) Offline data enhancement

[0132] Before the training process, a new dataset is generated to implement offline data augmentation. The generated dataset is large enough to ensure that there are enough samples to resample the classes with low F1 scores during the training phase. Compared with online data augmentation, offline data augmentation has the following advantages:

[0133] (a) Easy to read;

[0134] (b) facilitate subsequent implementation and adjustments;

[0135] (c) Generating visualization images helps control the enhanced images.

[0136] (3) Resampling algorithm based on F1-score and augmentation In order to increase the feature learning depth of minority classes, this application calculates the F1-Scores on the validation dataset of each class for every five training epochs. These F1-level scores will be used as weights to determine the target number of samples for the next class training iteration. The number of target training samples is equal in all iterations. The F1 score of a class with a lower number of samples means that the class was learned less in the previous iteration. Therefore, more samples will be selected from this class in the next iteration. The proposed method adopts a new weight-based algorithm adjustment method to correct the class distribution bias in the imbalanced data classification task. In this method, the feature extraction depth training process of the minority class is increased in an incremental manner, thereby improving the classification performance of the minority class.

[0137] Further references Figure 6 As an implementation of the above-mentioned method, the present application provides an embodiment of an unbalanced image data classification device based on dynamic sampling. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0138] refer to Figure 6 , a device for classifying imbalanced image data based on dynamic sampling, comprising:

[0139] An unbalanced image data creating module 101, configured to manually create unbalanced image data;

[0140] A preprocessing module 102 is configured to normalize and one-hot encode the created unbalanced image data to obtain training samples;

[0141] A basic model building module 103 is configured to build a basic model for image classification and freeze the model;

[0142] A dynamic sampling module 104 is configured to dynamically sample training samples using an f1-score algorithm;

[0143] The model training module 105 is configured to unfreeze the model and start training using the data dynamically sampled by the f1-score, and then output the loss result and the accuracy of each type of image recognition based on the confusion matrix;

[0144] The imbalanced image data classification module 106 is configured to classify the imbalanced image data using the trained model.

[0145] Reference below Figure 7 , which shows a structural diagram of a computer system 200 suitable for implementing an electronic device of an embodiment of the present application. Figure 7The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0146] like Figure 7 As shown, the computer system 200 includes a central processing unit (CPU) 201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 202 or a program loaded from a storage unit 208 into a random access memory (RAM) 203. Various programs and data required for the operation of the system 200 are also stored in the RAM 203. The CPU 201, the ROM 202, and the RAM 203 are connected to each other via a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0147] The following components are connected to the I / O interface 205: an input section 206 including a keyboard, a mouse, and the like; an output section 207 including a liquid crystal display (LCD) and speakers; a storage section 208 including a hard disk; and a communication section 209 including a network interface card such as a LAN card or a modem. The communication section 209 performs communication processing via a network such as the Internet. A drive 220 is also connected to the I / O interface 205 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 220 as needed, so that a computer program read therefrom can be installed in the storage section 208 as needed.

[0148] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 209, and / or installed from a removable medium 211. When the computer program is executed by the central processing unit (CPU) 201, the above-mentioned functions defined in the method of the present application are performed.

[0149] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device implements the following when executing. Figure 1 The method shown in .

[0150] It should be noted that the computer-readable storage medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0151] The computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0152] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0153] The above describes specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0154] In the description of this application, it should be understood that the terms "upper", "lower", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limiting this application. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The simple fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used for improvement. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. A method for classifying unbalanced image data based on dynamic sampling, characterized by: The method comprises the following steps: S1: Manually create unbalanced image data; S2: Normalize and one-hot encode the created unbalanced image data to obtain training samples; S3: Build a basic image classification model and freeze the model; specifically: S31: Build the basic model structure based on the residual network resnet-50, remove the top model, and set the input_shape according to the parameter shape=(32,32,3); S32: Change the new top layer to a flattened fully connected layer; S33: Freeze the basic model; S4: Use the f1-score algorithm to dynamically sample training samples; specifically: S41: Dynamically update the data capacity of the current training sample according to the f1-score; S42: Detect whether the sample size updated by the f1-score is larger than the imbalance manually set at the beginning. If so, perform image enhancement, otherwise perform random sampling; S5: Unfreeze the model and start training using the data dynamically sampled by f1-score, and then output the loss results and the accuracy of each type of image recognition based on the confusion matrix; S6: Use the trained model to classify unbalanced image data.

2. The method for classifying unbalanced image data based on dynamic sampling according to claim 1, characterized in that: Said S1 specifically includes: Read the CIFAR-10 dataset and manually convert each type of data into unbalanced data according to the ratio of [250, 750, 1250, 1750, 2250, 2750, 3250, 3750, 4250, 5000] as an example.

3. The method for classifying unbalanced image data based on dynamic sampling according to claim 1, wherein: In S4, a softmax activation function is used for activation, and parameters are tuned by SGD stochastic gradient descent.

4. The method for classifying unbalanced image data based on dynamic sampling according to claim 3, wherein: In S4, parameter tuning is performed using three sets of unbalanced data and one set of balanced data.

5. The method for classifying unbalanced image data based on dynamic sampling according to claim 4, characterized in that: The three sets of unbalanced data are: A=[250, 750, 1250, 1750, 2250, 2750, 3250, 3750, 4370, 5000]； B=[250, 250, 250, 500, 500, 500, 2500, 2500, 2500, 5000]； C=[500, 500, 500, 500, 500, 5000, 5000, 5000, 5000, 5000]； The set of balance data is: D=[5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000, 5000]。 6. The method for classifying unbalanced image data based on dynamic sampling according to claim 1, characterized in that: In S5, the number of images of different categories is manually changed to obtain the current F1 score after the results of every 5 epochs, and the resampling and training model are repeated in groups of 5 epochs.

7. A device for classifying unbalanced image data based on dynamic sampling, characterized by: The device comprises: an unbalanced image data creation module configured to manually create unbalanced image data; A preprocessing module is configured to normalize and one-hot encode the created unbalanced image data to obtain training samples; The basic model construction module is configured to build a basic image classification model and freeze the model. Specifically, it includes: S31: building a basic model structure based on the residual network ResNet-50, removing the top model, and setting the input_shape according to the parameter shape=(32, 32, 3); S32: changing the new top layer to a flattened fully connected layer; S33: freezing the basic model; The dynamic sampling module is configured to dynamically sample training samples using the f1-score algorithm; specifically, it includes: S41: dynamically updating the data capacity of the current training sample according to the f1-score; S42: detecting whether the sample size updated by the f1-score is larger than the imbalance manually set at the beginning; if so, performing image enhancement, otherwise performing random sampling; The model training module is configured to unfreeze the model and start training using data dynamically sampled by the f1-score, and then output the loss results and the accuracy of each type of image recognition based on the confusion matrix; The imbalanced image data classification module is configured to classify imbalanced image data using the trained model.

Images