A digital recognition method based on convolutional neural network

Abstract

The present invention discloses a digital recognition method based on a convolutional neural network, which pre-processes, centers and digitally segments an image to be feature-extracted, extracts and processes the foreground area of ​​a grayscale character image through a circumscribed rectangle, and centers the image according to a unified standard. The convolutional neural network is further improved, and the features of different convolutional layers are restored to the size of the initial input through error back propagation, and the predicted output obtained by the forward propagation operation is visualized and compared and analyzed, and the network model is trained using a mean square error function as an error function to update the weight and bias term parameters. At the same time, the activation function is further improved, and the operation output of each layer in the convolutional neural network is used as the input of the activation function, and the weighted sum of the input end is converted into a neuron output value, thereby increasing the nonlinear fitting ability of the neural network and achieving handwritten digital recognition with a high accuracy rate.

Description

Technical Field

[0001] The present invention relates to the technical field of convolutional neural networks, and in particular to a digital recognition method based on convolutional neural networks. Background Art

[0002] At present, digital recognition can be divided into printed text recognition and handwriting recognition. According to the type of handwriting recognition, the handwriting method is divided into online and offline forms. The offline form mainly uses handwriting images collected by scanners, cameras and webcams; the online form uses the handwriting dynamic information left on the interactive interface by electronic pens or touch screens, which is stored in the computer in a certain data format. In the field of optical character recognition, handwritten digital recognition is more difficult than printed text, and more challenges need to be solved.

[0003] When using convolutional neural network training to extract image features, traditional CNN model training will introduce a large number of parameters, resulting in the inability to quickly train and obtain effective results. In addition, activation functions need to be introduced during the training process to increase nonlinear factors. Nonlinear factors determine the accuracy of feature image extraction. Commonly used activation functions, such as Sigmoid, Tanh and ReLU functions, each activation function has certain defects during training. In practical applications, it is easy to experience gradient disappearance and inability to train, and it may take a long time to update the weights, making the extraction efficiency of feature images not high. Summary of the invention

[0004] In view of the above technical problems, this technical solution provides a digital recognition method based on a convolutional neural network. When training the network model, the activation function is further improved, the operation output of each layer in the convolutional neural network is used as the input of the activation function, and the weighted sum of the input end is converted into a neuron output value, so as to increase the nonlinear fitting ability of the neural network. At the same time, through the back propagation algorithm, the chain derivative rule and the gradient descent algorithm are used to update the model parameters to achieve the effect of quickly updating the network weights, avoiding the gradient vanishing problem, improving the recognition effect of the model, and realizing handwritten digital recognition with a high accuracy rate; it can effectively solve the above problems.

[0005] The present invention is achieved through the following technical solutions:

[0006] A digital recognition method based on a convolutional neural network preprocesses, centers and digitally segments an image to be feature extracted, extracts the foreground area of ​​a grayscale character image through an external rectangle, and centers it according to a unified standard; the specific steps include:

[0007] S1: Preprocess the original data in the MNIST dataset, use normalization preprocessing to convert the data of the original image matrix into data from 0 to 1, and adjust the image size to be consistent with the input requirements of the next layer;

[0008] S2: traverse the preprocessed input image through the convolution kernel to extract relevant features of the image;

[0009] S3: Use an improved activation function to process the output results into nonlinearity, thereby increasing the nonlinear fitting ability of the neural network;

[0010] S4: In order to reduce the dimension of feature mapping, that is, to reduce the scale of the model, the output feature map is divided into multiple small areas, each area is represented by a value, and the image-related features are processed using pooling operations to reduce the size of each feature map and retain the most important feature information;

[0011] S5: Perform full connection operation on the feature image data to make it highly abstract into a one-dimensional array;

[0012] S6: Perform digital classification operation on the abstract one-dimensional array to obtain output results; the algorithm steps of the classification operation include:

[0013] Step 1: Initialize all neural network parameter weights with random numbers;

[0014] Step 2: Input the image to be trained, perform the forward propagation step, and calculate the output feature image corresponding to each category;

[0015] Step 3: Calculate the recognition probability of the output layer;

[0016] Step 4: Calculate the gradient of the output error value relative to all neuron parameters or weight values ​​of the neuron through the back propagation algorithm, and then adjust the connection weights and update the relevant parameters. After multiple iterations, the optimal solution is obtained and the weight adjustment is stopped.

[0017] Step 5: Use the Softmax regression function to normalize the output values ​​to make the classification results more intuitive.

[0018] Furthermore, the input raw data described in step S1 is preprocessed, and the characteristic values ​​of the original samples are linearly transformed according to each dimension using a normalization method, so that the results are mapped to the range of [0,1] to achieve geometric scaling and improve the recognition efficiency of the recognizer; the normalization calculation formula is:

[0019]

[0020] In the above formula, h' is the normalized image pixel value, h is the image pixel value, and h min is the minimum value of image pixels, h max is the maximum value of image pixels.

[0021] Furthermore, the preprocessed input image is traversed through the convolution kernel in step S2. The convolution operation on the image is generally a multi-channel input. The convolution kernel needs to have the same number of channels. Each convolution kernel channel is convolved with the corresponding channel of the input layer. The calculation formula of the convolution operation is:

[0022]

[0023]

[0024]

[0025] In the above formula, is the input of the mth neuron in the kth layer, is the convolution kernel of the kth layer, is the output of the mth neuron after the kth layer of convolution; is the bias of the k-th layer neuron, It is the final output accumulated after multi-layer channel convolution operation; is the output after processing by the improved activation function, and f(.) is a certain activation function used to introduce nonlinear characteristics.

[0026] Furthermore, the formula of the improved activation function described in step S3 is:

[0027]

[0028] In the above formula, x is the activation value;

[0029] The activation function is a nonlinear mapping operation that has the characteristics of retaining, mapping, and activating neurons. The activation function achieves the effect of simulating the propagation of data in human neurons by nonlinearly transforming the input signal. It is mainly used between adjacent layers to connect neural networks and add nonlinear factors to neural networks. In order to fit any function, nonlinear operations are added. The CNN model needs to use the gradient descent method to update the weight data, so the corresponding activation function must be continuous and differentiable. When iterating, the value range of the function must be small to control the output data within a certain range, making it easier for the training results to converge.

[0030] The improved activation function is:

[0031]

[0032]

[0033] In the above formula, f(x) is the improved function and f'(x) is the derivative function.

[0034] Furthermore, the pooling operation described in step S4 is used to process image-related features, reduce the size of each feature map, and retain the most important feature information; in fact, the maximum pooling operation is used to reduce feature parameters and retain important feature information; the downsampling process of calculating the maximum value of the region can be expressed as:

[0035]

[0036]

[0037] In the above formula, M is the downsampling interval, a j is the jth element in the pooling area, is the output of the jth neuron, and f(.) is some activation function used to introduce nonlinear characteristics.

[0038] Furthermore, in step S5, the feature image data is fully connected to make it highly abstracted into a one-dimensional array. The weight corresponding to each neuron is combined with the output result of the previous layer of neurons through the full connection operation to calculate the output of the current layer of neurons. The calculation expression is:

[0039]

[0040]

[0041] In the above formula, is the weight coefficient of the jth neuron in the fully connected layer; x k-1 is the output of the previous layer of neurons; is the output of the kth neuron in the connection layer; is the output after processing by the improved activation function; f(.) is a certain activation function used to introduce nonlinear characteristics.

[0042] Furthermore, in the updating of the relevant parameters described in Step 4, the loss function of the model is calculated based on the error between the predicted output of the image and the actual label value, and the network model is trained with the mean square error function as the error function to calculate the loss of the model; the calculation formula for calculating the model loss is:

[0043]

[0044] In the above formula, y' is the expected output, y is the actual output, and N is the total number of samples.

[0045] Furthermore, in the training process of the back propagation algorithm described in Step 4, the first step is to calculate the total error δ of the network. The calculation formula of the error δ is:

[0046]

[0047] The error formula generated by the last layer of back propagation, that is, the jth neuron in the kth layer, is:

[0048]

[0049] in, Represents the output of the jth neuron in the last layer, i.e. the kth layer;

[0050] The chain derivation formula of the weight parameter is:

[0051]

[0052] Among them, x k-1 is the output of the neurons in the k-1 layer;

[0053] The chain derivation formula of the bias term parameter is:

[0054]

[0055] Use the gradient descent method to update the weight parameters and bias parameters. The calculation formula is as follows:

[0056]

[0057]

[0058] Among them, η represents the learning rate. Generally, the value of the learning rate is very small, and its function is to control the amplitude of parameter update.

[0059] Furthermore, in step 5, the Softmax regression function is used to normalize the output value. The Softmax regression function is used to combine the fully connected layer with the output layer to intuitively output the classification probability of the sample. The formula of the Softmax regression function is:

[0060]

[0061] In the above formula, x i is the input value of the i-th neuron in the output layer, n is the total number of neurons, Y i is the output value / classification probability of the i-th neuron.

[0062] Beneficial Effects

[0063] Compared with the prior art, the digital recognition method based on convolutional neural network proposed in the present invention has the following beneficial effects:

[0064] (1) The present invention uses the Mnist data set to train the model, and preprocesses, centers and digitally segments the image from which features are to be extracted. Since the sizes of the same characters in the data set are different, the foreground area of ​​the grayscale character image is extracted and processed by a circumscribed rectangle and centered according to a unified standard.

[0065] (2) In order to study the effect of the features extracted by the model on the final model recognition, the present invention further improves the convolutional neural network. The features of different convolutional layers are restored to the size of the initial input through error back propagation, and the predicted output obtained by the forward propagation operation is visualized and compared and analyzed. The network model is trained with the mean square error function as the error function to update the adjustment weights and bias parameters, so as to reduce the loss rate of the feature image and increase the recognition effect.

[0066] (3) The present invention further improves the activation function, uses the operation output of each layer in the convolutional neural network as the input of the activation function, and converts the weighted sum of the input end into a neuron output value, thereby increasing the nonlinear fitting ability of the neural network and achieving handwritten digit recognition with higher accuracy.

[0067] (4) The present invention uses Deep WorkNet Designer to design training network parameters. In order to increase the extraction effect of feature images, a 19-layer network model is constructed for training, and an improved activation function is added to the designed network model to improve the training effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0069] Figure 2 Schematic diagram of the improved activation function in the present invention.

[0070] Figure 3 This is a training flow chart of the convolutional neural network algorithm in the present invention.

[0071] Figure 4 This is a design diagram of the convolutional neural network model in the present invention.

[0072] Figure 5 It is a simulation diagram of the training results of the present invention. DETAILED DESCRIPTION

[0073] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Under the premise of not departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solutions of the present invention should all fall within the protection scope of the present invention.

[0074] Embodiment 1:

[0075] like Figure 1 As shown, a digital recognition method based on a convolutional neural network preprocesses, centers and digitally segments the image to be feature extracted, extracts the foreground area of ​​the grayscale character image through an external rectangle, and centers it according to a unified standard; the specific steps include:

[0076] S1: Preprocess the input raw data, use normalization preprocessing to convert the data of the original image matrix into data from 0 to 1, and adjust the image size to be consistent with the input requirements of the next layer.

[0077] The input original image is preprocessed, and the eigenvalues ​​of the original sample are linearly transformed according to each dimension using the normalization method, so that the result is mapped to the range of [0,1] to achieve geometric scaling and improve the recognition efficiency of the recognizer; the normalization calculation formula is:

[0078]

[0079] In the above formula, h' is the normalized image pixel value, h is the image pixel value, and h min is the minimum value of image pixels, h max is the maximum value of image pixels.

[0080] S2: The preprocessed input image is traversed through the convolution kernel, and feature images are extracted from the processed image to weaken the influence of unimportant factors on the judgment result. The forward propagation step is performed to extract the relevant features of the image and calculate the output feature image corresponding to each category.

[0081] The convolution operation of the image is generally multi-channel input, so the convolution kernel needs to have the same number of channels, and each convolution kernel channel is convolved with the corresponding channel of the input layer. Use a trainable convolution kernel to convolve the input image. The convolution kernel traverses the image with a certain step size, and performs a convolution operation each time it moves, and finally obtains the image feature matrix. The convolution operation of the image is generally multi-channel input, so the convolution kernel needs to have the same number of channels, and each convolution kernel channel is convolved with the corresponding channel of the input layer. The calculation formula for the convolution operation is:

[0082]

[0083]

[0084]

[0085] In the above formula, is the input of the mth neuron in the kth layer, is the convolution kernel of the kth layer, is the output of the mth neuron after the kth layer of convolution; is the bias of the k-th layer neuron, It is the final output accumulated after multi-layer channel convolution operation; is the output after being processed by the improved activation function, and f(.) is a certain activation function used to introduce nonlinear characteristics. S3: Use the improved activation function to process the output result into nonlinearity, thereby increasing the nonlinear fitting ability of the neural network. The feature values ​​extracted by the convolution operation are used as the input of the improved activation function to connect the neural network and add nonlinear factors to the neural network. For the improved formula of the activation function, the activation function is a nonlinear mapping operation with the characteristics of retaining, mapping, and activating neurons. The formula of the improved activation function is:

[0086]

[0087] In the above formula, x is the activation value.

[0088] The activation function achieves the effect of simulating the propagation of data in human neurons by nonlinearly transforming the input signal. It is mainly used between adjacent layers to connect neural networks and add nonlinear factors to neural networks. In order to fit any function, nonlinear operations are added. The CNN model needs to use the gradient descent method to update the weight data, so the corresponding activation function must be continuous and differentiable. When iterating, the function's value range must be small to control the output data within a certain range, making it easier to converge the training results. The improved activation function image is as follows: Figure 2 As shown. The improved activation function is:

[0089]

[0090]

[0091] In the above formula, f(x) is the improved function and f'(x) is the derivative function.

[0092] S4: In order to reduce the dimension of feature mapping, that is, to reduce the scale of the model, the output feature map is divided into multiple small areas, each area uses one value to represent multiple values ​​in the area, and the maximum pooling operation is used to reduce the feature parameters. The output feature map is divided into multiple small areas, each area uses one value to represent multiple values ​​in the area, so as to reduce the size of each feature map and retain the most important feature information. The downsampling process of calculating the maximum value of the area can be expressed as:

[0093]

[0094]

[0095] In the above formula, M is the downsampling interval, a j is the jth element in the pooling area, is the output of the jth neuron, and f(.) is some activation function used to introduce nonlinear characteristics.

[0096] S5: Perform a full connection operation on the feature image data to make it highly abstracted into a one-dimensional array; through the full connection operation, the weight corresponding to each neuron is combined with the output result of the neurons in the previous layer to calculate the output of the neurons in the current layer. The calculation expression is:

[0097]

[0098]

[0099] In the above formula, is the weight coefficient of the jth neuron in the fully connected layer; x k-1 is the output of the previous layer of neurons; is the output of the kth neuron in the connection layer; is the output after processing by the improved activation function; f(.) is a certain activation function used to introduce nonlinear characteristics.

[0100] S6: Perform digital classification operations on the abstract one-dimensional array, calculate the gradient of the output error value relative to all neuron parameters or weight values ​​of the neuron through the back propagation algorithm, and then adjust the connection weights accordingly, update the relevant parameters, and stop adjusting the weights after multiple iterations until the optimal solution is obtained. For the update of model parameters, the loss function of the model should be calculated based on the error between the predicted output of the image and the actual label value. In this embodiment, the training of the model uses the mean square error loss function as the error function to calculate the loss of the model; the output result is obtained. The algorithm steps of the classification operation include:

[0101] Step 1: Initialize all neural network parameter weights with random numbers.

[0102] Step 2: Input the image to be trained, perform the forward propagation step, and calculate the output feature image corresponding to each category.

[0103] Step 3: Calculate the recognition probability of the output layer.

[0104] Step 4: Use the back propagation algorithm to calculate the gradient of the output error value relative to all neuron parameters or weight values ​​of the neuron, and then adjust the connection weights accordingly, update the relevant parameters, and stop adjusting the weights after multiple iterative calculations until the optimal solution is obtained.

[0105] To update the model parameters, the loss function of the model should be calculated based on the error between the predicted output of the image and the actual label value. The training of the model in this embodiment uses the mean square error loss function as the error function to calculate the loss of the model. The calculation formula for calculating the model loss is:

[0106]

[0107] In the above formula, y' is the expected output, y is the actual output, and N is the total number of samples.

[0108] Based on the predicted output of the forward propagation process, the model parameters are updated through back propagation using the chain derivative rule and gradient descent algorithm. The first step of the back propagation training process is to calculate the total error δ of the network. The calculation formula of the error δ is:

[0109]

[0110] The error formula generated by the last layer of back propagation, that is, the jth neuron in the kth layer, is:

[0111]

[0112] in, Represents the output of the jth neuron in the last layer, i.e. the kth layer;

[0113] The chain derivation formula of the weight parameter is:

[0114]

[0115] Among them, x k-1 is the output of the neurons in the k-1 layer;

[0116] The chain derivation formula of the bias term parameter is:

[0117]

[0118] Use the gradient descent method to update the weight parameters and bias parameters. The calculation formula is as follows:

[0119]

[0120]

[0121] Among them, η represents the learning rate. Generally, the value of the learning rate is very small, and its function is to control the amplitude of parameter update.

[0122] Step 5: Use the Softmax regression function to normalize the output values ​​to make the classification results more intuitive.

[0123] The Softmax regression function is used to combine the fully connected layer with the output layer to intuitively output the classification probability of the sample; the formula of the Softmax regression function is:

[0124]

[0125] In the above formula, x i is the input value of the i-th neuron in the output layer, n is the total number of neurons, Y i is the output value (classification probability) of the i-th neuron.

[0126] When training the network, all the weights of different neural network parameters are initialized with random numbers. The preprocessed MNIST data set is used as input data to perform forward propagation operations and calculate the output feature images corresponding to each layer. The output results of each layer are processed into nonlinearity using an improved activation function to increase the nonlinear fitting ability of the neural network. Then, the gradient of the output error value relative to all neuron parameters or weight values ​​is calculated through the back-propagation algorithm, and the connection weights are adjusted accordingly to update the relevant parameters. Finally, the output value is normalized using the Softmax regression function to make the classification results more intuitive.

[0127] like Figure 3 As shown, the convolutional neural network parameters are trained as follows:

[0128] In order to verify the training effect of the improved activation function, the network model parameters are set for training.

[0129] Input: Image input, size is 28×28×1, each dimension represents width, height and channel

[0130] conv1: the first convolution operation, the convolution kernel is 11×11, the number is 96, and the step size is 4;

[0131] acti1: Use improved activation function;

[0132] norm1: cross-channel normalization, channel window size is 5;

[0133] pool1: pooling operation, the pooling mode is maximum pooling, the pooling area is 3×3, the step size is 2, and the filling mode is zero filling;

[0134] conv2: the second convolution operation, the convolution kernel is 5×5, divided into two groups, each with 128 cells and a step size of 1;

[0135] acti2: Use improved activation function;

[0136] norm2: cross-channel normalization, channel window size is 5;

[0137] pool2: pooling operation, the pooling method is maximum pooling, the pooling area is 3×3, the step size is 2, and the filling method is zero filling;

[0138] conv3: convolution operation, the convolution kernel is 3×3, divided into two groups, each with 192 elements and a stride of 1.

[0139] acti3: Use an improved activation function.

[0140] conv4: convolution operation, the convolution kernel is 3×3, divided into two groups, each with 192 elements and a stride of 1.

[0141] acti4: Use an improved activation function.

[0142] conv5: convolution operation, the convolution kernel is 3×3, divided into two groups, each with 128 elements and a stride of 1.

[0143] acti5: Uses an improved activation function.

[0144] pool5: pooling operation, the pooling method is maximum pooling, the pooling area is 3×3, the step size is 2, and the filling method is zero filling;

[0145] fc1: the first fully connected layer, a total of 4090 features are expanded and mapped to the sample space;

[0146] fc2: The second fully connected layer, with a total of 4090 features expanded and mapped to the sample space;

[0147] fc3: The third fully connected layer, a total of 10 features are expanded and mapped to the sample space;

[0148] output: output result.

[0149] Construct a 19-layer convolutional network model for training. The constructed network model is as follows Figure 4 shown.

[0150] Training results analysis

[0151] After the network parameters and model construction were completed, the improved activation function was used for small batch training. Each batch was set to 468 data, and 10 batches were trained, with a total of 4680 training data. The results after training are as follows Figure 5 And as shown in Table 1.

[0152] Table 1 shows the specific data of the accuracy and loss accuracy when constructing the model training

[0153]

[0154]

[0155] From the experimental data in the table, we can see that the effect is quite obvious. After the 8th batch, the recognition accuracy has reached 100.00%, and the loss accuracy has been reduced to 0.0002. This shows that the construction of the model and the improvement of the activation function can achieve a good recognition effect.

[0156] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes, substitutions and improvements within the technical scope disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A digital recognition method based on convolutional neural network, characterized in that: The image to be feature extracted is preprocessed, centered and digitally segmented, the foreground area of ​​the grayscale character image is extracted and processed by the circumscribed rectangle, and centered according to a unified standard; The specific steps include: S1: Preprocess the original data in the MNIST dataset, use normalization preprocessing to convert the data of the original image matrix into data from 0 to 1, and adjust the image size to be consistent with the input requirements of the next layer; S2: Traverse the preprocessed input image through the convolution kernel to extract relevant features of the image; the convolution kernel traverses the preprocessed input image, and the convolution operation of the image is a multi-channel input. The convolution kernel needs to have the same number of channels, and each convolution kernel channel is convolved with the corresponding channel of the input layer; the calculation formula of the convolution operation is: In the above formula, is the input of the mth neuron in the kth layer, is the convolution kernel of the kth layer, is the output of the mth neuron after the kth layer of convolution; is the bias of the k-th layer neuron, It is the final output accumulated after multi-layer channel convolution operation; is the output after being processed by the improved activation function, and f(.) is the improved activation function used to introduce nonlinear characteristics; S3: Using an improved activation function to process the output result into nonlinearity, thereby increasing the nonlinear fitting ability of the neural network; the formula of the improved activation function is: In the above formula, x is the activation value; The activation function is a nonlinear mapping operation that has the characteristics of retaining, mapping, and activating neurons. The activation function achieves the effect of simulating the propagation of data in human neurons by nonlinearly transforming the input signal. It is mainly used between adjacent layers to connect neural networks and add nonlinear factors to neural networks. In order to fit any function, nonlinear operations are added. The CNN model needs to use the gradient descent method to update the weight data, so the corresponding activation function must be continuous and differentiable. When iterating, the value range of the function must be small to control the output data within a certain range, making it easier for the training results to converge. The improved activation function is: In the above formula, f(x) is the improved function and f'(x) is the derivative function; S4: In order to reduce the dimension of feature mapping, that is, to reduce the scale of the model, the output feature map is divided into multiple small areas, each area is represented by a value, and the image-related features are processed using pooling operations to reduce the size of each feature map and retain the most important feature information; S5: Perform full connection operation on the feature image data to make it highly abstract into a one-dimensional array; S6: Perform digital classification operation on the abstract one-dimensional array to obtain output results; the algorithm steps of the classification operation include: Step 1: Initialize all neural network parameter weights with random numbers; Step 2: Input the image to be trained, perform the forward propagation step, and calculate the output feature image corresponding to each category; Step 3: Calculate the recognition probability of the output layer; Step 4: Calculate the gradient of the output error value relative to all neuron parameters or weight values ​​of the neuron through the back propagation algorithm, and then adjust the connection weights and update the relevant parameters. After multiple iterations, the optimal solution is obtained and the weight adjustment is stopped. Step 5: Use the Softmax regression function to normalize the output values ​​to make the classification results more intuitive.

2. The method for digital recognition based on convolutional neural network according to claim 1, characterized in that: In step S1, the original data in the MNIST data set is preprocessed, and the eigenvalues ​​of the original samples are linearly transformed according to each dimension using the normalization method, so that the results are mapped to the range of [0,1] to achieve geometric scaling and improve the recognition efficiency of the recognizer; the normalization calculation formula is: In the above formula, h' is the normalized image pixel value, h is the image pixel value, and h min is the minimum value of image pixels, h max is the maximum value of image pixels.

3. The method for digital recognition based on convolutional neural network according to claim 1, characterized in that: The pooling operation described in step S4 processes image-related features, reduces the size of each feature map, and retains the most important feature information; in fact, the maximum pooling operation is used to reduce feature parameters and retain important feature information; the downsampling process of calculating the maximum value of the region can be expressed as: In the above formula, M is the downsampling interval, a j is the jth element in the pooling area, is the output of the jth neuron, and f(.) is some activation function used to introduce nonlinear characteristics.

4. The method for digital recognition based on convolutional neural network according to claim 1, characterized in that: The step S5 performs a full connection operation on the feature image data to make it highly abstracted into a one-dimensional array. The weight corresponding to each neuron is combined with the output result of the previous layer of neurons through the full connection operation to calculate the output of the current layer of neurons. The calculation expression is: In the above formula, is the weight coefficient of the jth neuron in the fully connected layer; x k-1 is the output of the previous layer of neurons; is the output of the kth neuron in the connection layer; is the output after processing by the improved activation function; f(.) is a certain activation function used to introduce nonlinear characteristics.

5. The method for digital recognition based on convolutional neural network according to claim 1, characterized in that: To update the relevant parameters described in Step 4, the loss function of the model should be calculated based on the error between the predicted output of the image and the actual label value. The network model is trained with the mean square error function as the error function to calculate the loss of the model. The calculation formula for calculating the model loss is: In the above formula, y' is the expected output, y is the actual output, and N is the total number of samples.

6. The method for digital recognition based on convolutional neural network according to claim 5, characterized in that: In the training process of the back propagation algorithm described in step 4, the first step is to calculate the total error δ of the network. The calculation formula of the error δ is: The error formula generated by the last layer of back propagation, that is, the jth neuron in the kth layer, is: in, Represents the output of the jth neuron in the last layer, i.e. the kth layer; The chain derivation formula of the weight parameter is: Among them, x k-1 is the output of the neurons in the k-1 layer; The chain derivation formula of the bias term parameter is: Use the gradient descent method to update the weight parameters and bias parameters. The calculation formula is as follows: Among them, η represents the learning rate, which controls the amplitude of parameter update.

7. The method for digital recognition based on convolutional neural network according to claim 1, characterized in that: In step 5, the Softmax regression function is used to normalize the output value. The Softmax regression function is used to combine the fully connected layer and the output layer to intuitively output the classification probability of the sample; the formula of the Softmax regression function is: In the above formula, x i is the input value of the i-th neuron in the output layer, n is the total number of neurons, Y i is the output value / classification probability of the i-th neuron.

Images