An image classification method and system

Abstract

The present invention relates to the field of image processing, and specifically provides an image classification method. First, the image is adjusted, then the central position is cropped, and the comprehensive value F of each image is obtained through the resolution evaluation module of the image complexity; then, the resolution is adjusted. By statistically analyzing the comprehensive value F of the data set, the image complexity is classified, and the data set is divided into 5 levels according to an arithmetic progression. The resolution of each part is 96, 112, 168, 196, and 224 from small to large, and the resolution of the image is adjusted according to the value; finally, the picture of the new size is input into the modified residual convolutional neural network ResNet-50, thereby completing more accurate image classification. Compared with the prior art, the present invention can adjust the size of the input neural network according to the image complexity, making simple pictures obtain a smaller size without affecting the result, which can reduce the training difficulty of the neural network and improve the accuracy of the classification network.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and specifically provides an image classification method and system. Background Art

[0002] Image classification is the most basic task in computer vision. Before 2012, image classification was usually performed by traditional algorithms. However, since the CNN model AlexNet achieved a historic breakthrough in 2012, with its performance far exceeding that of traditional methods, the development process has gradually evolved from traditional algorithms to using deep learning models for classification tasks.

[0003] Subsequently, networks such as VGG, GoogLeNet, and ResNet gradually emerged, and the classification accuracy gradually improved. However, as the network depth gradually increases, network training becomes increasingly difficult. Since the resolution of the images input to the network is generally limited to 224x224, various pictures in the dataset need to be adjusted in resolution before being input into the network, which results in complex pictures and simple pictures being adjusted to the same size. However, the network can also accurately detect simple pictures with a smaller resolution and calculate faster.

[0004] Therefore, how to improve the difficult problem of training large classification networks is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] The present invention aims at the deficiencies of the above-mentioned prior art and provides a highly practical image classification method.

[0006] A further technical task of the present invention is to provide a reasonably designed, safe and applicable image classification system.

[0007] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0008] An image classification method, first adjust the image, then crop the center position, and obtain the comprehensive value F of each image through the resolution evaluation module of image complexity;

[0009] Then, adjust the resolution, classify the picture complexity by statistically calculating the comprehensive value F of the dataset, and divide the dataset into 5 levels according to an arithmetic progression. The resolution of each part is 96, 112, 168, 196, 224 from small to large, and adjust the resolution of the images according to the values;

[0010] Finally, input the pictures of the new size into the modified Residual Convolutional Neural Network ResNet-50 to complete more accurate image classification.

[0011] Furthermore, it has the following steps:

[0012] S1. Extract the two-dimensional entropy, contrast, energy, and correlation of each image in the dataset;

[0013] S2. Adjust the resolution size of the image according to the image complexity;

[0014] S3. Input the new-sized image obtained in step S2 into the changed classification network for classification;

[0015] S4. Finally, pass through the Softmax layer to obtain the recognition result.

[0016] Furthermore, in step S1, it further includes:

[0017] S1-1. Adjust the dataset image to a size between 96x96 and 512x512, and then crop the center position;

[0018] S1-2. Calculate the two-dimensional entropy of the image I in the dataset according to the following formula 1 :

[0019]

[0020] H(I 1 ) is the two-dimensional entropy of the image I 1 . (i,j) is a feature binary tuple composed of the neighborhood gray mean and the pixel gray value of the image. i represents the gray value of the pixel between 0 and 225, and j represents the neighborhood gray mean between 0 and 225. P(i,j) is the joint probability, and the joint probability is calculated according to the following formula:

[0021] where f(i,j) is the frequency of occurrence of the feature binary tuple (i,j), and N is the scale of the image;

[0022]

[0023] S1-3. Calculate the contrast of the image I 1 according to the following formula:

[0024]

[0025] G(I 1 ) is the contrast of the image I 1 . It is a statistic used to describe the texture fineness and can reflect the clarity of the image. (i - j) represents the difference between the neighborhood gray mean and the pixel gray value of the image.

[0026] Furthermore, in step S1, it further includes:

[0027] S1-4. Calculate the energy of the image I 1 according to the following formula:

[0028]

[0029] E(I 1 ) is the energy of image I 1 , a statistic representing the consistency of the gray-scale distribution, and is a measure of the uniformity of the gray-scale distribution of the image;

[0030] S1-5. Calculate the correlation degree of image I 1 according to the following formula:

[0031]

[0032] R(I 1 ) is the correlation degree of image I 1 , which can measure the similarity degree of the elements of the gray-level co-occurrence matrix in the row and column directions of the image. Where μ represents the mean of P x and P y , and σ represents the standard deviation of P x and P y , where P x and P y are calculated according to the following formula:

[0033]

[0034]

[0035] S1-6. After obtaining the two-dimensional entropy, contrast, energy, and correlation degree, perform weighted summation. Since the two-dimensional entropy and contrast are positively correlated with the image complexity, the more complex the image, the greater the two-dimensional entropy and contrast values. Therefore, the weights of H(I 1 ) and G(I 1 ) are 1, while the energy and correlation degree are negatively correlated with the image complexity. The simpler the image, the greater the energy and correlation degree. Therefore, the weights of E(I 1 ) and R(I 1 ) are -1;

[0036] The comprehensive index F(I 1 ) can be obtained by calculating through the following formula:

[0037] F(I 1 ) = H(I 1 ) + G(I 1 ) - E(I 1 ) - R(I 1 ).

[0038] Furthermore, in step S2, it includes:

[0039] S2-1. Obtain the F values of all images in the dataset through step S1, and use the min and max functions to find the minimum F value F min and the maximum F value max . The resolution sizes set according to the image complexity range from 96 to 224, which are 96, 112, 168, 196, and 224 respectively. According to the general term formula of an arithmetic sequence:

[0040] A n = a1+(n - 1)d

[0041] where A n is the last term of the sequence, a1 is the first term, d is the common difference, and n is the number of terms.

[0042] Furthermore, in step S2, it further includes:

[0043] S2-2. Substitute n = 5, A n = F max , a1 = F min into the formula to obtain d = (F max - F min ) / 4. Therefore, the image I 1 will output R(I 1 ) through the resolution evaluation module;

[0044]

[0045] Furthermore, in step S3, it includes:

[0046] S3-1. First, the new resolution image R(I 1 ) passes through the convolutional layer of the large classification network ResNet-50 to obtain the feature X(R(I 1 )), and then through the improved batch normalization layer BN;

[0047] The formula is as follows:

[0048]

[0049] where i is the resolution corresponding to the current input feature, μ i is the mean of the input feature X(R(I 1 )), is the standard deviation of the input feature X(R(I 1 )), γ i , β 1 are the learnable parameters for the input feature X(R(I 1 )), and ∈ is a parameter to prevent division by zero and does not need to be set for the input feature.

[0050] Further, in step S3, it further includes:

[0051] S3-2. Obtain the feature map FeatureMap(I 1 )) from the X(R(I 1 )) through the ResNet-50 residual network;

[0052] S3-3. Input FeatureMap(I 1 ) into the global average pooling layer to obtain the output GAP(I 1 );

[0053] S3-4. Convert the inputs of different sizes into the same-sized outputs, and finally obtain the FC through the fully connected layer.

[0054] An image classification system adjusts an image, then crops the central position, and obtains the comprehensive value F of each image through the resolution evaluation module of the image complexity;

[0055] Then, the resolution is adjusted. By statistically analyzing the comprehensive value F of the data set, the image complexity is classified, and the data set is divided into 5 levels according to an arithmetic progression. Each part has resolutions of 96, 112, 168, 196, and 224 from small to large, and the images are adjusted in resolution according to the values;

[0056] Finally, the images of the new size are input into the modified ResNet-50 residual convolutional neural network to complete more accurate image classification.

[0057] Compared with the prior art, an image classification method and system of the present invention have the following outstanding beneficial effects:

[0058] For the current neural network, on the one hand, the present invention can adjust the size of the input neural network according to the image complexity. Without affecting the results, simple images can obtain smaller sizes, which can reduce the training difficulty of the neural network.

[0059] On the other hand, through the batch normalization layers created for multiple resolutions in the network, the accuracy of the classification network can also be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0061] Attached Figure 1It is a schematic flow chart of an image classification method;

[0062] Appendix Figure 2 It is a schematic flow chart of calculating the complexity of pictures in an image classification method;

[0063] Appendix Figure 3 It is a flow chart of adjusting the resolution size of an image according to the complexity of the picture in an image classification method;

[0064] Appendix Figure 4 It is a schematic diagram of an improved classification network for multiple-resolution inputs in an image classification method. Detailed implementation manners

[0065] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention will be further described in detail below in conjunction with specific implementation manners. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0066] The following gives an optimal embodiment:

[0067] As Figure 1 shown, in an image classification method of this embodiment, first, the image is adjusted to a size of 256x256, and then the part with a center size of 224x224 is cropped. Then, the comprehensive value F of each image is obtained through the resolution evaluation module of the image complexity.

[0068] Secondly, resolution adjustment is performed. By statistically aggregating the value F of the data set, the complexity of the pictures is classified, and the data set is divided into 5 levels according to an arithmetic progression. Each part corresponds to sizes with resolutions of 96, 112, 168, 196, and 224 from small to large, and the resolution of the image is adjusted according to the value.

[0069] Finally, the picture of the new size is input into the modified residual convolutional neural network ResNet-50. Thus, a more accurate image classification process is completed.

[0070] The specific steps are as follows:

[0071] S1. Extract the two-dimensional entropy, contrast, energy, and correlation of each picture in the data set;

[0072] Further including:

[0073] S1-1. Adjust the data set image to a size of 256x256, and then crop the part with a center of 224x224;

[0074] S1-2. Calculate the two-dimensional entropy of the image I in the dataset according to the following formula: 1 :

[0075]

[0076] H(I 1 ) is the two-dimensional entropy of the image I 1 . (i, j) is a feature binary tuple composed of the neighborhood gray-scale mean and the pixel gray-scale of the image. i represents the gray-scale value of the pixel between 0 and 225, and j represents the neighborhood gray-scale mean between 0 and 225. P(i, j) is the joint probability, and the joint probability is calculated according to the following formula:

[0077] where f(i, j) is the frequency of occurrence of the feature binary tuple (i, j), and N is the scale of the image;

[0078]

[0079] S1-3. Calculate the contrast of the image I 1 according to the following formula:

[0080]

[0081] G(I 1 ) is the contrast of the image I 1 , a statistic used to describe the texture fineness, which can reflect the clarity of the image. (i - j) represents the difference between the neighborhood gray-scale mean and the pixel gray-scale of the image.

[0082] S1-4. Calculate the energy of the image I 1 according to the following formula:

[0083]

[0084] E(I 1 ) is the energy of the image I 1 , a statistic representing the consistency of the gray-scale distribution, which is a measure of the uniformity of the gray-scale distribution of the image;

[0085] S1-5. Calculate the correlation of the image I 1 according to the following formula:

[0086]

[0087] R(I 1 ) is the correlation of the image I 1 , which can measure the similarity degree of the elements in the gray-level co-occurrence matrix of the image in the row and column directions. Among them, μ represents the mean of P x and P y , and σ represents the standard deviation of P x and Py The standard deviation of which P x and P y is calculated according to the following formula:

[0088]

[0089]

[0090] S1-6. After obtaining the two-dimensional entropy, contrast, energy, and correlation, weighted summation is performed. Since the two-dimensional entropy and contrast are positively correlated with the image complexity, the more complex the image, the larger the two-dimensional entropy and contrast values. Therefore, the weights of H(I 1 ) and G(I 1 ) are 1. However, the energy and correlation are negatively correlated with the image complexity. The simpler the image, the larger the energy and correlation values. Therefore, the weights of E(I 1 ) and R(I 1 ) are -1;

[0091] The comprehensive index F(I 1 ) can be obtained by calculating through the following formula:

[0092] F(I 1 ) = H(I 1 ) + G(I 1 ) - E(I 1 ) - R(I 1 ).

[0093] Among them, the resolution selection module process is as Figure 2 shown.

[0094] S2. Adjust the resolution size of the image according to the image complexity;

[0095] Further includes:

[0096] The basic process of adjusting the resolution size of the image according to the image complexity is as Figure 3 shown.

[0097] S2-1. Obtain the F values of all pictures in the dataset through step S1, and use the min and max functions to find the minimum F value F min and the maximum value F max . The resolution sizes set according to the image complexity range from 96 to 224, which are 96, 112, 168, 196, and 224 respectively. According to the general term formula of the arithmetic sequence:

[0098] A n = a1 + (n - 1)d

[0099] where A n is the last term of the sequence, a1 is the first term, d is the common difference, and n is the number of terms.

[0100] S2-2. Substitute n = 5 and A n = F max , a1 = F min into the formula to obtain d = (F max - F min ) / 4. Therefore, the image I 1 will output R(I 1 ) through the resolution evaluation module;

[0101]

[0102] S3. Input the new size obtained in step S2 into the changed classification network for classification;

[0103] Furthermore, it includes:

[0104] The improved classification network ResNet-50 for multiple resolution inputs is as Figure 4 .

[0105] S3-1. First, the new resolution image R(I 1 ) passes through the convolutional layer of the large classification network ResNet-50 to obtain the feature X(R(I 1 )), and then through the improved batch normalization layer BN;

[0106] The formula is as follows:

[0107]

[0108] where i is the resolution corresponding to the current input feature, μ i is the mean of the input feature X(R(I 1 )), is the standard deviation of the input feature X(R(I 1 )), γ i , β i are the learnable parameters for the input feature X(R(I 1 )), and ∈ is a parameter to prevent division by zero and does not need to be set for the input feature.

[0109] S3-2. Pass X(R(I 1 )) through the residual network of ResNet-50 to obtain the feature map FeatureMap(I 1 );

[0110] S3-3. Input FeatureMap(I 1 ) into the global average pooling layer to obtain the output GAP(I 1 );

[0111] S3-4. Convert inputs of different sizes into outputs of the same size, and finally obtain FC through a fully connected layer.

[0112] S4. Finally, pass through the Softmax layer to obtain the recognition result.

[0113] Based on the above method, in an image classification system of this embodiment, the image is adjusted, then the central position is cropped, and the comprehensive value F of each image is obtained through the resolution evaluation module of the image complexity.

[0114] Then, the resolution is adjusted. By statistically analyzing the comprehensive value F of the data set, the image complexity is classified, and the data set is divided into 5 levels according to an arithmetic progression. The resolution of each part is 96, 112, 168, 196, and 224 from small to large, and the resolution of the image is adjusted according to the value.

[0115] Finally, input the image of the new size into the modified Residual Convolutional Neural Network ResNet-50 to complete more accurate image classification.

[0116] The above specific implementation manners are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above specific implementation manners. Any appropriate changes or substitutions made by those of ordinary skill in the art in any technical field that conform to the claims of an image classification method and system of the present invention shall fall within the patent protection scope of the present invention.

[0117] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made in these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An image classification method, characterized in that, first, the image is adjusted, then the central position is cropped, and the comprehensive value F of each image is obtained through the resolution evaluation module of the image complexity; then, the resolution is adjusted. By statistically calculating the comprehensive value F of the data set, the image complexity is classified, and the data set is divided into 5 levels according to an arithmetic progression. Each part has resolutions of 96, 112, 168, 196, and 224 from small to large, and the images are adjusted in resolution according to the values; finally, the image of the new size is input into the modified Residual Convolutional Neural Network ResNet-50 to complete more accurate image classification; The steps are as follows: S1. Extract the two-dimensional entropy, contrast, energy, and correlation of each image in the data set; It also includes: S1-1. Adjust the data set images to a size between 96x96 and 512x512, and then crop the central position; S1-2. Calculate the two-dimensional entropy of the image I in the dataset according to the following formula: 1 : H(I 1 ) is the two-dimensional entropy of image I 1 . (i, j) is a feature binary tuple composed of the neighborhood gray mean of the image and the pixel gray value of the image. i represents that the gray value of the pixel is between 0 and 225, and j represents that the neighborhood gray mean is between 0 and 225. P(i, j) is the joint probability, and the joint probability is calculated according to the following formula: where f(i,j) is the frequency of occurrence of the feature binary group (i,j), and N is the scale of the image; S1-3. Calculate the contrast of image I according to the following formula 1 ; G(I 1 ) is the contrast of image I 1 , a statistic used to describe the coarseness of the texture, which can reflect the clarity of the image. (i - j) represents the difference between the neighborhood gray mean of the image and the pixel gray value of the image; S1-4. Calculate the energy of image I according to the following formula 1 ; E(I 1 ) is the energy of image I 1 , a statistic representing the consistency of the gray-scale distribution, which is a measure of the uniformity of the gray-scale distribution of the image; S1-5. Calculate the relevance of image I according to the following formula: 1 : R(I 1 ) is the relevance of image I 1 . It measures the similarity degree of the elements of the gray-level co-occurrence matrix in the row and column directions in the image. Here, μ represents the mean value of P x and P y , and σ represents the standard deviation of P x and P y . Here, P x and P y are calculated according to the following formula: S1-6. After obtaining the two-dimensional entropy, contrast, energy, and correlation, perform weighted summation. Since the two-dimensional entropy and contrast are positively correlated with the image complexity, the more complex the image, the greater the two-dimensional entropy and contrast values. Therefore, the weights of H(I 1 ) and G(I 1 ) are 1. However, the energy and correlation are negatively correlated with the image complexity. The simpler the image, the greater the energy and correlation. Therefore, the weights of E(I 1 ) and R(I 1 ) are -1; The comprehensive index F(I 1 ) can be calculated by the following formula: F(I 1 ) = H(I 1 ) + G(I 1 ) - E(I 1 ) - R(I 1 ); S2. Adjust the resolution size of the image according to the image complexity; It includes: S2-1. Obtain the F values of all the images in the dataset through step S1, and use the min and max functions to find the minimum F value F min and the maximum F value max . The resolution sizes set according to the image complexity range from 96 to 224, which are 96, 112, 168, 196, and 224 respectively. According to the general term formula of an arithmetic sequence: A n = a1 + (n - 1)d where A n is the last term of the sequence, a1 is the first term, d is the common difference, and n is the number of terms; S2-2. Substitute n = 5, A n = F max , a1 = F min into the formula to obtain d = (F max - F min ) / 4. Therefore, the image I 1 will output R(I 1 ) through the resolution evaluation module; S3. Input the new size input image obtained in step S2 into the modified classification network for classification; First, the new resolution image R(I 1 ) passes through the convolutional layer of the large classification network ResNet-50 to obtain the feature X(R(I 1 )) and then passes through the improved batch normalization layer BN; The formula is as follows: where i is the resolution corresponding to the current input feature, μ i is the mean of the input feature X(R(I 1 )) is the variance of the input feature X(R(I 1 )) i , γ i , β i are learnable parameters for the input feature X(R(I 1 )); ∈ is a parameter to prevent division by zero and does not need to be set for the input feature S4. Finally, pass through the Softmax layer to obtain the recognition result.

2. An image classification method according to claim 1, characterized in that, in step S3, it further includes: S3-1. Obtain the feature map Feature Map(I 1 ) of X(R(I 1 ) through the ResNet-50 residual network; S3-2. Input the Feature Map (I 1 ) into the global average pooling layer to obtain the output GAP (I 1 ); S3-3. Convert the inputs of different sizes into the same size output, and finally obtain FC through the fully connected layer.

3. An image classification system, characterized in that, the image is adjusted, then the central position is cropped, and the comprehensive value F of each image is obtained through the resolution evaluation module of the image complexity; then, the resolution is adjusted. By statistically calculating the comprehensive value F of the data set, the image complexity is classified, and the data set is divided into 5 levels according to an arithmetic progression. Each part has resolutions of 96, 112, 168, 196, and 224 from small to large, and the images are adjusted in resolution according to the values; finally, the image of the new size is input into the modified Residual Convolutional Neural Network ResNet-50 to complete more accurate image classification; The operations are as follows: S1. Extract the two-dimensional entropy, contrast, energy, and correlation of each image in the data set; It also includes: S1-1. Adjust the data set images to a size between 96x96 and 512x512, and then crop the central position; S1-2. Calculate the two-dimensional entropy of the image I in the dataset according to the following formula 1 as follows: H(I 1 ) is the two-dimensional entropy of image I 1 , (i, j) is a feature binary tuple composed of the neighborhood gray mean of the image and the pixel gray of the image. i represents that the gray value of the pixel is between 0 and 225, j represents that the neighborhood gray mean is between 0 and 225, and P(i, j) is the joint probability, where the joint probability is calculated according to the following formula: where f(i,j) is the frequency of occurrence of the feature binary group (i,j), and N is the scale of the image; S1-3. Calculate the contrast of image I according to the following formula 1 ; G(I 1 ) is the contrast of image I 1 , a statistic used to describe the coarseness of texture, which can reflect the clarity of the image. (i - j) represents the difference between the neighborhood gray value mean of the image and the pixel gray value of the image; S1-4. Calculate the energy of image I according to the following formula 1 ; E(I 1 ) is the energy of image I 1 , a statistic representing the consistency of the gray-scale distribution, which is a measure of the uniformity of the gray-scale distribution of the image; S1-5. Calculate the relevance of image I according to the following formula: 1 : R(I 1 ) is the relevance of image I 1 . It measures the similarity degree of the elements of the gray-level co-occurrence matrix in the row and column directions in the image. Here, μ represents the mean value of P x and P y , and σ represents the standard deviation of P x and P y . Among them, P x and P y are calculated according to the following formula: S1-6. After obtaining the two-dimensional entropy, contrast, energy, and correlation, perform weighted summation. Since the two-dimensional entropy and contrast are positively correlated with the image complexity, the more complex the image, the greater the two-dimensional entropy and contrast values. Therefore, the weights of H(I 1 ) and G(I 1 ) are 1. However, the energy and correlation are negatively correlated with the image complexity. The simpler the image, the greater the energy and correlation. So, the weights of E(I 1 ) and R(I 1 ) are -1; The comprehensive index F(I 1 ) can be calculated by the following formula: F(I 1 ) = H(I 1 ) + G(I 1 ) - E(I 1 ) - R(I 1 ); S2. Adjust the resolution size of the image according to the image complexity; It includes: S2-1. Obtain the F values of all the pictures in the dataset through step S1, and use the min and max functions to find the minimum F value F min and the maximum value F max . The resolution sizes set according to the image complexity range from 96 to 224, which are 96, 112, 168, 196, and 224 respectively. According to the general term formula of the arithmetic sequence: A n = a1 + (n - 1)d Where A n is the last term of the sequence, a1 is the first term, d is the common difference, and n is the number of terms; S2-2. Substitute n = 5 and A n = F max , a1 = F min into the formula to obtain d = (F max - F min ) / 4. Therefore, the image I 1 will output R(I 1 ) through the resolution evaluation module; S3. Input the new size input image obtained in step S2 into the modified classification network for classification; First, the new resolution image R(I 1 ) passes through the convolutional layer of the large classification network ResNet-50 to obtain the feature X(R(I 1 ), and then passes through the improved batch normalization layer BN; The formula is as follows: where i is the resolution corresponding to the current input feature, μ i is the mean of the input feature X(R(I 1 )) is the variance of the input feature X(R(I 1 )) i , γ i , β i are learnable parameters for the input feature X(R(I 1 )); ∈ is a parameter to prevent division by zero and does not need to be set for the input feature S4. Finally, pass through the Softmax layer to obtain the recognition result.

Images