Image classification and recognition method, device and equipment

Abstract

An image classification and recognition method, apparatus, and device are disclosed. One image classification and recognition method includes: processing an image through a first convolution operation, a first pooling operation, and a first fully connected operation to obtain classification and recognition parameters; and processing the image based on the classification and recognition parameters through a second convolution operation, a second pooling operation, and a second fully connected operation to obtain a classification and recognition result. Each of the first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation includes a weight initialization operation. The weight initialization operation ensures that the convolution variance and gradient variance in the first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation remain consistent. This method effectively solves the problems of gradient explosion and gradient vanishing during image processing, resulting in a high accuracy in the obtained classification and recognition result.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, specifically to a method, apparatus, and device for image classification and recognition. Background Technology

[0002] Data classification and recognition is an important and classic problem in natural language processing. It involves mapping a piece of data containing information to a pre-defined category or several categories. Data classification and recognition can be categorized into various complex scenario classification problems depending on the task type. It is commonly used in areas such as the construction of dynamic digital scenarios, social communication case analysis, dynamic real-time hot topic push, and filtering of spam or junk files. It provides technical support for the search and research of dynamic semantic databases and is currently one of the main research hotspots.

[0003] Convolutional Neural Networks (CNNs) are well-suited for data classification and recognition, especially the Lenet-5 model. The Lenet-5 model is commonly used for handwritten digit classification and recognition because it can use backpropagation to train the data. However, during training, particularly with the MNIST handwritten digit dataset, the vanishing or exploding gradient problem occurs. This causes most of the gradients obtained from backpropagation to be ineffective or even have a negative effect, resulting in low accuracy in the final classification and recognition results. Summary of the Invention

[0004] This invention provides a method, apparatus, and device for image classification and recognition.

[0005] An image classification and recognition method, comprising:

[0006] The image is processed by the first convolution operation, the first pooling operation, and the first fully connected operation to obtain classification and recognition parameters;

[0007] Based on the classification and recognition parameters, the image is processed by a second convolution operation, a second pooling operation, and a second fully connected operation to obtain the classification and recognition result.

[0008] The first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation all include a weight initialization operation;

[0009] The weight initialization operation ensures that the convolution variance and gradient variance in the first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation are consistent.

[0010] In the image classification and recognition method described above, the weight initialization operation ensures that the convolution variance is 1 during forward propagation of the image in the first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation, and the gradient variance is 1 during backward propagation.

[0011] In the image classification and recognition method described above, the first convolution operation, the first pooling operation, and the first fully connected operation are implemented sequentially through convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, fully connected layer 1, and fully connected layer 2.

[0012] The second convolution operation, the second pooling operation, and the second fully connected operation are implemented sequentially through convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, fully connected layer 1, and fully connected layer 2.

[0013] In the image classification and recognition method described above, a prediction operation is provided before the second convolution operation, and the prediction operation is used to predict the accuracy of the image;

[0014] If the accuracy rate is greater than or equal to the first threshold, the result is output directly.

[0015] If the accuracy is less than the first threshold, then the image is processed by performing a second convolution operation, a second pooling operation, and a second fully connected operation.

[0016] The prediction operation includes the formaly function, cvtColor function, softmax function, and squeeze function;

[0017] The `formarry` and `cvtColor` functions are both used to perform image style conversion before the accuracy operation of the predicted image;

[0018] The softmax function is used to achieve the predicted image accuracy operation.

[0019] The squeeze function is used to perform output enlargement and scaling operations on the image.

[0020] In the image classification and recognition method described above, the image undergoes a preprocessing operation before being processed by the first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation. The preprocessing operation is used to augment the data of the image.

[0021] The preprocessing operation includes processing the image using center cropping, resize, ToTensor, and normalization operations in the transforms network.

[0022] The present invention also provides an image classification and recognition device, comprising:

[0023] The preprocessing module is used to preprocess the image and perform data augmentation.

[0024] The convolution module is used to process the image through convolution operations and perform feature extraction.

[0025] The pooling module is used to process the image using pooling operations to reduce the output size;

[0026] The fully connected module is used for fully connected operation processing of the image, classification and output of recognition results;

[0027] The weight initialization module is used to control the convolution variance and gradient variance in the convolution module, pooling module, and fully connected module.

[0028] The prediction module is used to predict the accuracy of the image and determine whether to enable the subsequent convolution module, pooling module, and fully connected module.

[0029] The present invention also provides an image classification and recognition device, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement an image classification and recognition method as described in any of the above claims.

[0030] The present invention also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements an image classification and recognition method as described in any of the preceding claims.

[0031] The beneficial effects of this invention are as follows:

[0032] (1) The image classification and recognition method provided by the present invention includes a weight initialization operation in the convolution operation, pooling operation and fully connected operation. The weight initialization operation keeps the convolution variance and gradient variance in the convolution operation, pooling operation and fully connected operation consistent, so that the forward propagation gradient and convolution output in the forward and backward propagation are more stable during the image processing. This effectively solves the problems of gradient explosion and gradient vanishing in the existing image processing process. While ensuring the processing speed, it significantly improves the accuracy of the classification and recognition results.

[0033] (2) The image classification and recognition method provided by the present invention includes a prediction module before the convolution operation, pooling operation and fully connected operation. The prediction module can predict the accuracy of the input image and then decide whether to continue to perform the convolution operation, pooling operation and fully connected operation. It can further improve the accuracy of the classification and recognition results while shortening the image classification and recognition time.

[0034] (3) The image classification and recognition method provided by the present invention preprocesses the image before formal processing, which can expand the number of images and avoid the impact of class imbalance on the processing effect of subsequent images. In this method, the classification and recognition parameters obtained from the preprocessed image are combined with convolution, pooling and fully connected operations, and then the preprocessed image is formally processed, thereby improving the accuracy of the classification and recognition results.

[0035] (4) The image classification and recognition method provided by the present invention has the advantages of low training loss, low suppression performance, low training time and memory requirements, and high training accuracy when classifying and recognizing MNIST handwritten character dataset, Arabic handwritten digit dataset and clothing classification dataset. Attached Figure Description

[0036] The solutions and advantages of this application will become clear to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0037] In the attached diagram:

[0038] Figure 1 This is a flowchart illustrating the classification and recognition method in the embodiment;

[0039] Figure 2 This is a schematic diagram of the Lenet-5 model in the embodiment;

[0040] Figure 3 This is a schematic diagram of the pooling operation in the embodiment;

[0041] Figure 4 This is a schematic diagram illustrating the Lenet-5 optimization model processing the MNIST handwritten character dataset in an embodiment.

[0042] Figure 5 This is a schematic diagram illustrating the processing of an Arabic handwritten digit dataset by the Lenet-5 optimization model in this embodiment.

[0043] Figure 6 This is a schematic diagram illustrating how the Lenet-5 optimization model processes the clothing classification dataset in this embodiment.

[0044] Figure 7 This is a schematic diagram of the classification and identification device in the embodiment;

[0045] Figure 8 This is a schematic diagram of the classification and identification device in the embodiment. Detailed Implementation

[0046] Exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings.

[0047] Example

[0048] This embodiment provides an image classification and recognition method, see [link to relevant documentation]. Figure 1 ,include:

[0049] The image is processed by the first convolution operation, the first pooling operation, and the first fully connected operation to obtain classification and recognition parameters;

[0050] Based on the classification and recognition parameters, the image is processed by a second convolution operation, a second pooling operation, and a second fully connected operation to obtain the classification and recognition result.

[0051] The first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation all include a weight initialization operation;

[0052] The weight initialization operation ensures that the convolution variance and gradient variance in the first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation are consistent.

[0053] Specifically:

[0054] 1. Convolution operations, pooling operations, and fully connected operations

[0055] The first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation all use convolution, pooling, and fully connected operations. The convolution operation is used to extract feature information; the pooling operation is used to reduce the size of the image data output; and the fully connected operation is used to classify the image results.

[0056] The first convolution operation, the first pooling operation, and the first fully connected operation are implemented sequentially through convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, fully connected layer 1, and fully connected layer 2.

[0057] The second convolution operation, the second pooling operation, and the second fully connected operation are implemented sequentially through convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, fully connected layer 1, and fully connected layer 2.

[0058] In this embodiment, the applicant of the present invention utilizes the Lenet-5 model to implement the first convolution operation, the first pooling operation, and the first fully connected operation, as well as the convolution operation, the second pooling operation, and the fully connected operation in the second convolution operation, the second pooling operation, and the second fully connected operation. See [link to relevant documentation]. Figure 2 .

[0059] The Lenet-5 model consists of seven processing layers: Convolutional Layer 1, Pooling Layer 1, Convolutional Layer 2, Pooling Layer 2, Fully Connected Layer 1, Fully Connected Layer 2, and the output layer. Convolutional Layers 1 and 2 perform convolution operations to extract feature information; Pooling Layers 1 and 2 perform pooling operations to reduce the size of the output image data; Fully Connected Layers 1 and 2 perform fully connected operations to classify the image results; and the output layer outputs the classification and recognition results.

[0060] Convolutional layer 1 contains five 3×3×3 convolutional kernels. These kernels extract features from the input image through convolution operations, followed by a ReLU activation function to increase the nonlinearity of the Lenet-5 model. The ReLU activation function is calculated using formula (1).

[0061] f(x)=max(x,0) (1)

[0062] x is the horizontal axis, ranging from negative infinity to positive infinity, and the vertical axis represents the value mapped by the function f(x), ranging from 0 to positive infinity. Throughout the ReLU curve, when x ranges from negative infinity to 0, f(x) remains 0; when x ranges from 0 to positive infinity, f(x) increases proportionally along the y = x function. In other words, the output of the ReLU function is only 0 or a positive number. Compared to the sigmoid and tanh functions used in traditional methods, the ReLU function has the advantages of overcoming the vanishing gradient problem and accelerating model training.

[0063] The convolution operation uses the torch.nn.Con2d() function, which is based on the maximum likelihood estimation principle and is similar to the minimum mean square error function. The calculation formula is shown in formula (2).

[0064]

[0065] The feature map output by convolutional layer 1 is 8×18×18 in size and is passed to pooling layer 1 for feature selection and information filtering.

[0066] Pooling layer 1 uses max pooling, which takes the maximum value among the elements covered by the window. The pooling kernel size is 2×2, the stride is 2, and the output size is 8×9×9. Pooling operations can reduce the amount of computation and the number of parameters.

[0067] For pooling operations, see Figure 3The function used is `torch.nn.MaxPool2d`. Let the input size be (M, B, K, L), where M represents the maximum pooling window size (which can be a single value or a tuple); B represents the stride (which can be a single value or a tuple); K represents the padding (which can be a single value or a tuple); and L represents the stride of elements within the control window. The output size is (M, B, K). out ,L out ) and pooling window size kernel size(kH,kW) The relationship is: out(M i B j ,K,L)=imput(M i B j ,stride[0]*K+m,stride1]*L+n); kernel_size is k, which does not represent the traditional convolution kernel, but the size function of the sliding window. The size of this window can be arbitrarily specified. If the input is a single value, such as 3, then the window size is 3×3. If the input is a tuple, such as (3, 20), then the window size is 3×2. H is a boolean type category, which returns the index of the maximum value position; W is a boolean type state. If it is True, the output shape is calculated by rounding up; the default is rounding down. m represents the number of sliding windows; n represents this, which is also a boolean type value. It determines whether to use rounding up or rounding down and how to calculate the output shape when calculating the output shape.

[0068] After activation, convolution, and pooling operations, the image size can be reduced to 164×4. The 16×4×4 image is then flattened into a 1D vector, which is passed through two fully connected layers and one output layer to obtain the result. In the fully connected layer, each neuron is fully connected to all neurons in the previous layer, which can integrate local information with class discrimination from the convolutional or pooling layers.

[0069] Fully connected operations use the `torch.nn.Linear` function to perform linear transformations on the input image, in the form y = xF. T +G, where F is the weight matrix, T is the transpose matrix, and G is the bias. The activation function of each neuron in the fully connected layer also uses the ReLU function. The number of neurons in the two fully connected layers are 120 and 140, respectively.

[0070] The output layer outputs the probability value of the correct classification of the image, that is, the output of the image is the classification result. This value is calculated by the softmax function, and its calculation formula is shown in formula (3):

[0071]

[0072] Among them, y i Let be the output value of the i-th node. The softmax function can be used to transform the output value to the range [0, 1].

[0073] 2. By adding weight initialization operations to convolution, pooling, and fully connected operations, the Lenet-5 optimized model is obtained.

[0074] To achieve higher accuracy in image processing, the applicant of this invention incorporates a weight initialization operation in the first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation. This weight initialization operation ensures that the convolution variance and gradient variance remain consistent across these operations, and that the convolution variance is 1 during forward propagation and the gradient variance is 1 during backward propagation.

[0075] In this embodiment, the applicant of this invention added a weight initialization module based on the Lenet-5 model. The weight initialization module is used to implement the weight initialization operation, thereby obtaining the optimized Lenet-5 model. The optimized Lenet-5 model is used to implement the first convolution operation, the first pooling operation, and the first fully connected operation to process the image.

[0076] The weight initialization module is applied throughout the entire LeNet-5 model structure, primarily controlling and modifying the convolution, pooling, and fully connected operations within the LeNet-5 model. The weight initialization module is activated upon image input and remains activated until the entire image classification process is complete, after which it is deactivated. The purpose of the weight initialization module is to ensure that the convolution variance and gradient variance in the convolution, pooling, and fully connected operations remain consistent. This allows the values ​​of the convolution output and forward gradient in the LeNet-5 optimized model to tend towards a stable state during forward or backward propagation, effectively solving the gradient explosion and vanishing problems in existing image processing methods. This results in more accurate final classification and recognition results while maintaining image processing speed.

[0077] The parameters of the weight initialization module still satisfy the condition that the mean is 0, and the mean of the weights remains 0 throughout the update process. During forward propagation of the image in the weight initialization module, the variance of the convolution calculation result for each layer is 1; during backward propagation of the image, the variance of the gradient of each layer as it continues to propagate forward is 1.

[0078] In the weight initialization module, the calculation formula is shown in formula (4):

[0079] Y l =W l X l +B l (4)

[0080] Y l X represents the output value at a certain position. l The input to be convolved has a k×k×c shape, where k represents the kernel size and c represents the number of input channels. Let n = k×k×c, then n represents the number of input values ​​used to calculate an output value. W has a d×n shape, where d represents the number of output channels. The subscript l indicates the layer number, X... l =f(Y) l-1 f represents the ReLU activation function. l =d l-1 This indicates that the output of the previous layer becomes the input of the next layer after passing through the activation function, and that the number of input channels in the next layer of the network is equal to the number of output channels in the previous layer.

[0081] In this embodiment, var(X1+…+X k ) = var(X1) + ... var(X) k ), var() is the variance, X represents a random variable, and the subscript k represents the kth one. In the weight initialization module, the variance of each convolutional layer is 1 during image forward propagation. An output y is obtained by multiplying n inputs x and their n weights and then summing them. The weights are independent and identically distributed, and the values ​​are also independent and identically distributed. The values ​​and weights are independent of each other, and the calculation formula is shown in formula (5):

[0082] var(y p ) = n p var(x p ·w p (5)

[0083] Here, y, w, and x represent three different random variables. The reason for the difference between the three random variables is that the weight initialization module only contains convolution and ReLU activation functions. The default is a VGG network, without residual and concat structures, and without BN layers. p represents the p-th layer in which the weight values ​​are output.

[0084] To help those skilled in the art understand the operating mechanism of weight initialization, we will use an example here:

[0085] y=w1×x1+w2×x2+w3×x3+w4×x4+w5×x5+w6×x6 (6)

[0086] Among them, w 几×x 几 Considering them as a whole, and with 1 to 6 being independent of each other, the calculation formula can be further obtained as shown in formula (7):

[0087] var(y)=var(w1×x1)+var(w2×c2)+var(w3×c3)+var(w4×x4)+var(w5×x5)+var(w6×x6) (7)

[0088] When w 几 ×x 几 Since they are identically distributed, their variances are the same, and the calculation formula is as shown in formula (8):

[0089] var(y)=6var(w×x) (8)

[0090] Furthermore, since w1 and x1 are independent of each other, the calculation formula is as shown in formula (9):

[0091] var(y l ) = n l [var(w l )var(x l )+var(w l (Ex) l )2 2 +(Ew l ) 2 var(x l (9)

[0092] In the weight initialization, the mean of the weights is set to 0, and it is assumed that the mean of the weights remains 0 throughout the update process. Then E(w) l E(x) = 0, but x1 is obtained from the previous layer through the ReLU activation function, so E(x) = 0. l Since ) ≠ 0, the calculation formula is as shown in formula (10):

[0093] var(y l ) = n l [var(w l )var(x l )+var(w l (Ex) l ) 2 ] = n l var(w l )(var(x l )+(Ex l ) 2 (10)

[0094] in, Then formula (10) changes to:

[0095]

[0096] Next, please ask... This expectation is obtained by using the output of the (l-1)th layer, x l =f0(y l-1 ), where f represents the ReLU activation function, and the calculation formula is shown in formula (12):

[0097]

[0098] y l-1 When ∈(-∞,0), f(y) l-1 = 0. Remove the intervals less than 0, and when f(y) is greater than 0... l-1 )=y l-1 The calculation formula is shown in formula (13):

[0099]

[0100] Assuming the distribution is symmetrical around 0 and has a mean of 0, then y l-1 It is also symmetrically distributed around 0, and has a mean of 0 (assuming a bias of 0). The calculation formula can be further obtained as shown in formula (14):

[0101]

[0102] middle y l-1 If the mean is 0, then Then formula (14) can be further derived as follows:

[0103]

[0104] Substituting formula (15) into formula (11) yields the calculation formula as shown in formula (16):

[0105]

[0106] Then, by performing forward propagation from the first layer forward, the variance of a certain layer can be obtained as follows:

[0107]

[0108] Here y l The input image samples are normalized to obtain var(y1) = 1. Then, the output variance of each layer is equal to 1, resulting in the calculation formulas shown in formulas (18) and (19).

[0109]

[0110]

[0111] In the weight initialization operation, the input size of the convolutional layer is 32×16×16, representing the number of channels, height, and width, respectively. The kernel size is 64×32×3×3, representing the number of output channels, the number of input channels, the kernel height, and the kernel width, respectively. Therefore, the weights of this layer are:

[0112]

[0113] The initial bias is 0.64 × 32 × 3 × 3 = 184.32.

[0114] In this embodiment, the variance of the gradient during backpropagation is 1, then Here, Δ represents the derivative of the loss function with respect to it. Unlike the normal backpropagation derivation, here we assume ΔY. l This represents d channels, each of size k×k. As with forward propagation, ΔY l There are c channels, ΔX l There are d channels. The size is Therefore, ΔX l The shape is c×l. It differs from W only by a transpose. Similarly, a ΔX l The value is many Δy l Having obtained the variance, we can then calculate the variance of a single variable (gradient) using multiple independent and identically distributed variables. Assume the random variable... and Δy l They are all independently and identically distributed. If the distribution of Δx is symmetrical around 0, then Δx l The mean value of L is 0 for each layer, i.e., E(Δx) l ) = 0. Because during forward propagation, x l+1 =f(y l Therefore, the backpropagation is Δy. l =f'(y l )Δx l+1 Since f is a ReLU activation function, its derivative is either 0 or 1. Let's assume each derivative has half the value, and also assume f'(y) l ) and Δx l+1 Since they are independent of each other, the calculation formula is as shown in formula (20):

[0115]

[0116] The probability is divided into two parts: one part corresponds to a ReLU activation function derivative of 0, and the other part corresponds to a ReLU activation function derivative of 1 (and both parts are assumed to be 50% probabilities). Formula (20) represents that for a given Δy l The value of has a 50% probability that the derivative of the ReLU activation function is 0, and a 50% probability that it is 1. The calculation formula is shown in formula (21):

[0117]

[0118]

[0119] Formula (22) can also be derived from formula (7) using a similar method, resulting in the calculation formula shown in formula (23):

[0120]

[0121] Therefore, following the forward derivation method, the final calculation formula is:

[0122]

[0123] Following the final example of forward propagation, the value here should be...

[0124] 3. By adding prediction operations to the Lenet-5 optimization model, a better Lenet-5 model is obtained.

[0125] In order to shorten image processing time and further improve the accuracy of image processing results, the applicant of this invention provides a prediction operation before the second convolution operation. The prediction operation is used to predict the accuracy of the input image. If the accuracy is greater than or equal to a first threshold, the result is directly output. If the accuracy is less than the first threshold, the second convolution operation, the second pooling operation, and the second fully connected operation are performed to process the image.

[0126] In this embodiment, the applicant of the present invention added a prediction module based on the above-mentioned Lenet-5 optimized model. Specifically, a prediction module was added before the Lenet-5 optimized model. The prediction module is used to perform prediction operations to obtain a better Lenet-5 model. The better Lenet-5 model is used to implement the above-mentioned second convolution operation, second pooling operation, and second fully connected operation to process the image.

[0127] Therefore, the difference between the first convolution, pooling, and fully connected operations and the second convolution, pooling, and fully connected operations is that the first convolution, pooling, and fully connected operations (Lenet-5 optimized model) do not include a prediction module, but they do include a weight initialization module; the second convolution, pooling, and fully connected operations (Lenet-5 better model) include both a weight initialization module and a prediction module.

[0128] The prediction module is essentially used to predict the accuracy of the input image. If the accuracy requirement is met, the processing steps can be reduced and the result can be output directly. If the accuracy requirement is not met, the Lenet-5 optimization model processing will be performed next.

[0129] The processing steps of the prediction module are as follows: when an image enters the Lenet-5 better model, it first enters the prediction module. In order to ensure that the style of the image conforms to the processing standard of the prediction module when it enters the prediction module, so that it is easy for the prediction module to process, the input image needs to be style-transformed before the accuracy prediction operation is performed.

[0130] Specifically, the prediction module first reads the image, converts the image to an array using the `formarry` function, and then calls the `cvtColor` function from the OpenCV library to perform conversions in different spaces within the image. After the data style conversion, the prediction module performs an accuracy operation on the predicted image. Formula (3) is called to perform the image accuracy prediction operation. If the prediction accuracy is greater than or equal to the first threshold, the weight initialization operation, convolution operation, pooling operation, and fully connected operation (i.e., the Lenet-5 optimization model) are skipped. If this step is skipped, the NumPy library and the `squeeze` function are called to perform image output enlargement and scaling operations to ensure that the size of the image output is exactly consistent with the format required by the Lenet-5 optimization model. Then, the classification recognition result is output using the output layer. If the prediction accuracy is less than the first threshold, the image must undergo subsequent weight initialization operation, convolution operation, pooling operation, and fully connected operation (i.e., the Lenet-5 optimization model) before the classification recognition result is output from the output layer. To ensure the accuracy of the output classification recognition result, the inventors of this application set the first threshold to 99.8%. NumPy, mentioned above, is a third-party library for the Python language that supports a large number of high-dimensional arrays and matrix operations. NumPy also provides a large number of mathematical functions for array operations.

[0131] The code for running the prediction module is as follows:

[0132] for image in os.listdir(args.data_folder):

[0133] image = cv2.imread(os.path.join(args.data_folder, image)) image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

[0134] image_IS = Image.fromarray(image)

[0135] image_I = data_transform(image_IS)

[0136] image_I = image_I.to(device)

[0137] image_I = image_I.unsqueeze(0)

[0138] output = model(image_I)

[0139] output = torch.softmax(output, dim=1)

[0140] output = output.squeeze()

[0141] predict_idx = torch.argmax(output).cpu().numpy()

[0142] name = class_names[str(predict_idx)]

[0143] score = output[predict_idx]

[0144] res = "Category:{} Probability:{:.3f}".format(name, score)

[0145] plt.imshow(image_IS)

[0146] plt.title(res)

[0147] plt.show()

[0148] 4 Preprocess the image

[0149] To avoid the image processing effect being affected by class imbalance, the inventors of this application preprocessed the image before formal processing, namely before the first convolution operation, the first pooling operation, and the first fully connected operation (Lenet-5 optimized model), and before the second convolution operation, the second pooling operation, and the second fully connected operation (Lenet-5 better model). The purpose of this preprocessing is to expand the image data and ensure that the preprocessed image is not affected by class imbalance.

[0150] In this embodiment, the applicant of this invention performs preprocessing operations on the image based on the transforms network module in PyTorch. Specifically, the image is subjected to center cropping, Resize(), ToTensor(), and normalization operations to expand the image data.

[0151] During the image centering process, the centering parameter is set to (20, 12) for Chinese datasets and (20, 15) for non-Chinese datasets.

[0152] The Lenet-5 optimization model processes the preprocessed image to obtain classification and recognition parameters.

[0153] In this embodiment, the applicant of the present invention processes the preprocessed image through a first convolution operation, a pooling operation, and a fully connected operation to obtain classification and recognition parameters; specifically, the preprocessed image is processed using a Lenet-5 optimization model containing a weight initialization module to obtain classification and recognition parameters.

[0154] Because Chinese and non-Chinese datasets differ significantly, they are not suitable for training together. To achieve better classification and recognition results, different images are trained separately in the Lenet-5 optimized model to obtain different classification and recognition parameters. The training epochs are set to 20, and the learning rate is set to 0.001, which yields better classification and recognition parameters.

[0155] By combining the 6 classification parameters with the superior Lenet-5 model, a classification model is obtained.

[0156] In this embodiment, the inventors of this application combine classification and recognition parameters with a second convolution operation, pooling operation, and fully connected operation, thereby improving the accuracy of image classification. Specifically, different classification and recognition parameters are combined with a Lenet-5 improved model that includes a weight initialization module and a prediction module. Specifically, based on the obtained different classification and recognition parameters, the relevant parameters in the corresponding Lenet-5 improved model are adjusted, thereby achieving the purpose of matching and combining different classification and recognition parameters with the corresponding Lenet-5 improved model, thus obtaining different classification and recognition models and improving the recognition accuracy of the classification and recognition model.

[0157] The 7-classification recognition model processes the preprocessed image to obtain the classification and recognition results.

[0158] To obtain better classification and recognition results, different classification and recognition models are used to identify the corresponding images during image recognition. The images processed by the classification and recognition models are all preprocessed. For details of the preprocessing steps, please refer to Section 4 above. Finally, the classification and recognition results are obtained.

[0159] To verify the effectiveness of the classification and recognition method provided in this embodiment, the inventors of this application conducted relevant experiments. The classification and recognition method provided in this embodiment differs from other existing classification and recognition methods.

[0160] The images used in the experiment were from three well-known datasets: the MNIST handwritten character dataset, the Arabic handwritten digit dataset, and the clothing classification dataset. To ensure that each image was persuasive and recognizable, each image was obtained from different institutions and individuals, including hospitals, factories, universities, and government agencies.

[0161] First, the three preprocessed datasets are fed into the Lenet-5 optimized model for training, resulting in three classification and recognition parameters: model 1, model 2, and model 3. Then, when performing image classification and recognition, refer to... Figure 4-6 The improved LeNet-5 model was combined with Model 1 to classify and recognize the preprocessed MNIST handwritten character dataset; Model 2 was combined with the improved LeNet-5 model to classify and recognize the preprocessed Arabic handwritten digit dataset; and Model 3 was combined with the improved LeNet-5 model to classify and recognize the preprocessed clothing classification dataset. Simultaneously, other existing baseline models were also used to classify and recognize these three datasets. The experimental results are shown in Tables 1-3. The Improved LeNet-5 model is an abbreviation for the classification and recognition model obtained through the classification and recognition method provided in this embodiment, and will not be described again below.

[0162] Table 1: Comparison of the improved LeNet-5 model with other baseline models on the MNIST handwritten character dataset.

[0163]

[0164]

[0165] Table 2: Comparison of the improved LeNet-5 model with other baseline models on the Arabic handwritten digit dataset.

[0166]

[0167] Table 3: Comparison of the Improved LeNet-5 model with other baseline models on the clothing classification dataset

[0168]

[0169] The experimental results in Tables 1-3 show that the classification and recognition method provided in this embodiment has advantages in classifying and recognizing the MNIST handwritten character dataset, Arabic handwritten digit dataset, and clothing classification dataset, with low training loss, low suppression performance, low training time and memory requirements, and high training accuracy. Moreover, its performance in all aspects is significantly better than other model baselines, proving that the classification and recognition method provided in this embodiment has a fast classification and recognition speed and a high classification and recognition accuracy.

[0170] The present invention discloses an image classification and recognition device. See above. Figure 7 The device includes: a preprocessing module for preprocessing the image and performing data augmentation; a convolution module for processing the image using convolution operations and performing feature extraction; a pooling module for processing the image using pooling operations and reducing the output size; a fully connected module for processing the image using fully connected operations, classifying and outputting the recognition result; a weight initialization module for controlling the convolution variance and gradient variance in the convolution module, pooling module, and fully connected module; and a prediction module for predicting the accuracy of the image and determining whether to enable the subsequent convolution module, pooling module, and fully connected module. This device executes all the steps of the image classification and recognition method disclosed in the above embodiments, and this device can be set up independently in hardware.

[0171] This invention also discloses an image classification and recognition device, see [link to relevant documentation]. Figure 8 The system includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement an image classification and recognition method disclosed in the above embodiments.

[0172] In addition, the present invention also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements an image classification and recognition method disclosed in the above embodiments.

[0173] This embodiment provides an image classification and recognition method that incorporates a weight initialization operation within the convolution, pooling, and fully connected operations. This initialization ensures consistency between the convolution variance and gradient variance during these operations, resulting in more stable forward propagation gradients and convolution outputs during image processing. This effectively solves the problems of gradient explosion and vanishing gradients in existing image processing methods, significantly improving the accuracy of classification and recognition results while maintaining processing speed. Furthermore, the method includes a prediction module before the convolution, pooling, and fully connected operations. This prediction module predicts the accuracy of the input image and then determines whether to continue with the convolution and pooling operations. The method employs preprocessing and fully connected operations to further improve the accuracy of image classification and recognition results while shortening the image classification and recognition time. Furthermore, preprocessing images before formal processing expands the number of images available, preventing class imbalance from affecting subsequent image processing. The method combines the classification and recognition parameters obtained from the preprocessed images with convolution, pooling, and fully connected operations before formally processing the preprocessed images, thereby improving the accuracy of the classification and recognition results. In addition, it offers advantages such as low training loss, low suppression performance, low training time and memory requirements, and high training accuracy when classifying and recognizing MNIST handwritten character datasets, Arabic handwritten digit datasets, and clothing classification datasets.

[0174] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for classifying and recognizing images, characterized in that, include: The image is preprocessed using the transforms network module in PyTorch, including center cropping, resize, ToTensor, and normalization operations to augment the image data. For the MNIST handwritten character dataset, Arabic handwritten digit dataset, and clothing classification dataset, the preprocessed datasets are input into the Lenet-5 optimized model containing a weight initialization module for training, thereby obtaining classification and recognition parameters corresponding to different datasets. The first convolution operation, the first pooling operation, and the first fully connected operation in the Lenet-5 optimized model are implemented through convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, fully connected layer 1, and fully connected layer 2, respectively. The weight initialization module is used to keep the convolution variance and gradient variance consistent. Different classification and recognition parameters are combined with corresponding Lenet-5 superior models containing weight initialization and prediction modules, and the relevant parameters in the corresponding Lenet-5 superior models are adjusted according to the obtained different classification and recognition parameters to match the different classification and recognition parameters with the corresponding Lenet-5 superior models, resulting in different classification and recognition models for recognizing the corresponding images; the second convolution operation, the second pooling operation, and the second fully connected operation in the Lenet-5 superior model are implemented sequentially through convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, fully connected layer 1, and fully connected layer 2; In image recognition, the image to be recognized is preprocessed and then processed by the corresponding classification model. The Lenet-5 superior model performs its prediction module's prediction operation before executing its weight initialization, second convolution, second pooling, and second fully connected operations. The prediction module first reads the image, converts it to an array using the `formarry` function, then performs image space transformation using the `cvtColor` function from the OpenCV library, and finally uses the `softmax` function to predict image accuracy after style conversion. When the accuracy is greater than or equal to the first threshold, skip the subsequent weight initialization, second convolution, second pooling, and second fully connected operations of the Lenet-5 optimized model, and call the NumPy function library and the squeeze function to enlarge and scale the image output to make the image output size consistent with the format required by the Lenet-5 optimized model before outputting the classification and recognition result; when the prediction accuracy is less than the first threshold, perform the subsequent weight initialization, second convolution, second pooling, and second fully connected operations of the Lenet-5 optimized model before outputting the classification and recognition result; the first threshold is 99.8%.

2. The classification and recognition method according to claim 1, characterized in that, The weight initialization operation ensures that the convolution variance is 1 during forward propagation of the image in the first convolution operation, the first pooling operation, the first fully connected operation, the second convolution operation, the second pooling operation, and the second fully connected operation, and that the gradient variance is 1 during backward propagation.

3. An image classification and recognition device, characterized in that, To implement the classification and recognition method of claim 1, the method includes: The preprocessing module is used to preprocess images and augment data. The convolution module is used to process images using convolution operations and to extract features. The pooling module is used to process images using pooling operations, reducing the output size. The fully connected module is used for fully connected operations to process images, classify them, and output the recognition results. The weight initialization module is used to control the convolution variance and gradient variance in the convolution module, pooling module, and fully connected module; The prediction module is used to predict the accuracy of the image and determine whether to enable the subsequent convolution module, pooling module, and fully connected module.

4. An image classification and recognition device, characterized in that, The system includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement an image classification and recognition method according to any one of claims 1-2.

5. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements an image classification and recognition method according to any one of claims 1-2.

Images