Image recognition method for handwritten numerals and terminal

Abstract

The invention discloses an image recognition method for handwritten digits and a terminal, and the method comprises the steps: setting algorithm parameters, such as momentum and learning rate, for a digit recognition model according to an obtained sample data set of the handwritten digits, calculating an adaptive learning rate and related parameters on a model training data set through gradient second-order information parameters, and carrying out network weight training; in the training process, a verification data set is used for evaluating a loss function value to judge whether training is completed or not, whether the adaptive learning rate is adjusted or not is judged in the verification stage, and if the adaptive learning rate is adjusted, adaptive learning rate parameters are updated through operations such as calculation of a negative gradient of the loss function value and normalization according to network performance optimization requirements; and finally, selecting a test data set for network prediction, and taking the weight corresponding to the minimum loss function value as the final weight of the model. Therefore, the advantages of the stochastic gradient descent optimization algorithm fusing the adaptive learning rate and the second-order information in improving the training efficiency and prediction accuracy of the digital recognition model are fully displayed.

Description

Technical Field

[0001] This invention relates to the technical field of image deep learning, and in particular to an image recognition method and terminal for handwritten digits. Background Technology

[0002] In computer vision, handwritten digit recognition, as a fundamental task of optical character recognition, is widely used in scenarios such as financial document processing, postal sorting, and automated form entry. Traditional methods rely on a combination of manually designed features and shallow classifiers, but their recognition accuracy is limited by the robustness of feature extraction and cannot adapt to complex situations such as varied handwriting styles, stroke deformation, and noise interference in handwritten digits.

[0003] In recent years, deep learning-based recognition methods have significantly improved model performance. Convolutional neural networks (CNNs) automatically learn abstract features of digital images through multi-layer nonlinear transformations, achieving near-human-level recognition accuracy on public datasets such as MNIST and SVHN. However, the training performance of such models is highly dependent on the choice of optimization algorithm. Current mainstream methods generally employ stochastic gradient descent (SGD) and its adaptive improvement algorithms (such as AdaGrad, RMSProp, and Adam) to update network weights. These algorithms iteratively calculate the gradient of the loss function and adjust parameters to gradually reduce the error between the predicted values ​​and the true labels.

[0004] However, in practical applications of handwritten digit recognition, traditional optimization algorithms face significant bottlenecks:

[0005] (1) Poor adaptability of fixed learning rate: Local features of handwritten digit images, such as stroke thickness, tilt angle, and broken stroke noise, will cause the loss function surface to be highly non-convex. Fixed learning rate is difficult to quickly approach the optimal solution region in the early stage of training and will prolong the convergence time; while when it is close to convergence, it is easy to skip the optimal solution due to excessive step size, which will cause weight oscillation (e.g., confusion region for similar digits "5" / "6", "7" / "1").

[0006] (2) The limitations of first-order information are prominent: Although adaptive algorithms such as Adam adjust the learning rate by adjusting the historical gradient magnitude, they ignore the trend of gradient direction change and the curvature information of the loss function. When the model processes digital samples with large differences in writing style (such as different users writing "2" with connected or broken strokes), relying solely on the first-order gradient can easily cause the optimization to get stuck in local optima, resulting in a decrease in the model's generalization ability.

[0007] (3) Imbalance between computational resources and efficiency: To improve the robustness of recognition in complex scenarios, the number of parameters in modern network structures continues to increase. Traditional optimization algorithms, due to the lack of accurate modeling of gradient dynamics, require more iterations to converge, resulting in a significant increase in training time and computational cost, which restricts the deployment of lightweight scenarios such as embedded devices.

[0008] Therefore, while existing technologies have attempted to improve learning rate decay strategies, their pre-defined rules cannot dynamically respond to the local abrupt changes in the loss surface during training. Furthermore, introducing second-order optimization is difficult to apply to large-scale neural networks due to the high computational complexity of the Hessian matrix. Thus, designing an efficient optimization mechanism that adapts to the characteristics of handwritten digit recognition tasks while ensuring training stability has become a key challenge for improving model performance. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide an image recognition method and terminal for handwritten digits, which can effectively improve the recognition efficiency and accuracy of handwritten digits.

[0010] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0011] A method for image recognition of handwritten digits, comprising the following steps:

[0012] Obtain a sample dataset of handwritten digits, and set lightweight parameters for the digit recognition model based on the image structure of the sample dataset. The lightweight parameters include momentum coefficient, initial learning rate, and sampling frequency.

[0013] The sample dataset is divided into a training dataset, a validation dataset, and a test dataset. The network weights of the digit recognition model are trained using the training dataset by a stochastic gradient descent optimization method that integrates adaptive learning rate and second-order information.

[0014] The network weights are validated using the validation dataset, the loss function value of the network weights is calculated, and the training of the network weights is determined based on the loss function value. If the training is not completed, the process returns to the step of training the network weights of the digit recognition model using the training dataset.

[0015] If training is complete, determine whether to adjust the adaptive learning rate based on the loss function value. If yes, update the adaptive learning rate by calculating the negative gradient normalized data of the loss function value. Otherwise, compare the loss function values ​​of the network weights obtained in each training round on the test dataset, and select the network weights corresponding to the minimum loss function value as the network weights of the digit recognition model.

[0016] The digit recognition model is used to recognize and classify input handwritten digit images.

[0017] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows:

[0018] A handwritten digit image recognition terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps of the handwritten digit image recognition method described above.

[0019] The beneficial effects of this invention are as follows: Based on the obtained sample dataset of handwritten digits, algorithm parameters such as momentum and learning rate are set for the digit recognition model. On the model training dataset, the adaptive learning rate and related parameters are calculated using second-order gradient information parameters to train the network weights. During training, a validation dataset is used to evaluate the loss function value to determine whether training is complete. During the validation phase, it is determined whether the adaptive learning rate needs adjustment. If adjustment is needed, the adaptive learning rate parameters are updated according to network performance optimization requirements by calculating and normalizing the negative gradient of the loss function value. Finally, a test dataset is selected for network prediction, and the weights corresponding to the minimum loss function value are used as the final model weights. This fully demonstrates the advantages of the stochastic gradient descent optimization algorithm, which integrates adaptive learning rate and second-order information, in improving the training efficiency and prediction accuracy of the digit recognition model. Attached Figure Description

[0020] Figure 1 This is a flowchart of an image recognition method for handwritten digits according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of an image recognition terminal for handwritten digits according to an embodiment of the present invention;

[0022] Figure 3 This is a flowchart of the adaptive learning rate calculation steps in an embodiment of the present invention;

[0023] Figure 4 This is a flowchart of the weight update process according to an embodiment of the present invention.

[0024] Label Explanation:

[0025] 1. An image recognition terminal for handwritten digits; 2. A memory; 3. A processor. Detailed Implementation

[0026] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0027] Please refer to Figure 1 This invention provides an image recognition method for handwritten digits, comprising the following steps:

[0028] Obtain a sample dataset of handwritten digits, and set lightweight parameters for the digit recognition model based on the image structure of the sample dataset. The lightweight parameters include momentum coefficient, initial learning rate, and sampling frequency.

[0029] The sample dataset is divided into a training dataset, a validation dataset, and a test dataset. The network weights of the digit recognition model are trained using the training dataset by a stochastic gradient descent optimization method that integrates adaptive learning rate and second-order information.

[0030] The network weights are validated using the validation dataset, the loss function value of the network weights is calculated, and the training of the network weights is determined based on the loss function value. If the training is not completed, the process returns to the step of training the network weights of the digit recognition model using the training dataset.

[0031] If training is complete, determine whether to adjust the adaptive learning rate based on the loss function value. If yes, update the adaptive learning rate by calculating the negative gradient normalized data of the loss function value. Otherwise, compare the loss function values ​​of the network weights obtained in each training round on the test dataset, and select the network weights corresponding to the minimum loss function value as the network weights of the digit recognition model.

[0032] The digit recognition model is used to recognize and classify input handwritten digit images.

[0033] As described above, the beneficial effects of this invention are as follows: Based on the obtained sample dataset of handwritten digits, algorithm parameters such as momentum and learning rate are set for the digit recognition model. On the model training dataset, the adaptive learning rate and related parameters are calculated using second-order gradient information parameters to train the network weights. During training, a validation dataset is used to evaluate the loss function value to determine whether training is complete. During the validation phase, it is determined whether the adaptive learning rate needs adjustment. If adjustment is needed, the adaptive learning rate parameters are updated according to network performance optimization requirements by calculating and normalizing the negative gradient of the loss function value. Finally, a test dataset is selected for network prediction, and the weights corresponding to the minimum loss function value are used as the final model weights. This fully demonstrates the advantages of the stochastic gradient descent optimization algorithm, which integrates adaptive learning rate and second-order information, in improving the training efficiency and prediction accuracy of the digit recognition model.

[0034] Furthermore, the network weights of the digit recognition model are trained using the training dataset through a stochastic gradient descent optimization method that integrates second-order information and an adaptive learning rate, including:

[0035] In each training round, the adaptive learning rate α is calculated based on second-order information. l :

[0036]

[0037] S l =βSl-1 +(1-β)Clip(H l -θ||δ l || 2 )

[0038]

[0039] In the formula, α0 represents the initial learning rate, S l D represents the numerator parameter of the learning rate in round l. l H represents the denominator parameter of the learning rate in the l-th round, β represents the weights controlling the historical gradient data, and H... l Let g represent the second-order information parameters of the gradient of the network weights in the l-th round. l Let δ represent the gradient of the network weights in the l-th round, ∈ represent a preset constant, and δ l The Clip function represents the normalized value of the gradient, indicating that the variable is brought closer to 0 so that the learning rate α is equal to the normalized value of the gradient. l The value of is not less than 0;

[0040] The momentum parameter m is calculated based on the adaptive learning rate. l :

[0041] m l =γm l-1 +α l δ l

[0042] In the formula, γ represents the momentum coefficient, and m l-1 This represents the momentum parameter of the previous cycle;

[0043] Update the network weights based on the momentum parameters:

[0044] W l =W l-1 -(m l +δ l )

[0045] In the formula, W l-1 W represents the network weight in round l-1. l This represents the network weight in round l.

[0046] As described above, existing adaptive learning rate algorithms often suffer from excessive learning rate fluctuations during training due to inflexible learning rate adjustment strategies, leading to instability and even failure to converge. This invention addresses this issue by employing multiple mechanisms to ensure training stability. Firstly, the Clip function is used to bring the adaptive learning rate parameter closer to zero, ensuring that the learning rate α... l The value range is [0,1], which avoids abnormal parameter updates due to excessively large learning rates; on the other hand, the dynamic calculation of momentum parameters makes the weight update process smoother.

[0047] Furthermore, training the network weights of the digit recognition model using the training dataset further includes:

[0048] The training of the network weights of the digit recognition model ends when the number of training rounds reaches the preset maximum number of rounds, or when the loss function value of the verification dataset decreases by less than a threshold for a preset number of consecutive times.

[0049] As described above, limiting the maximum number of rounds can prevent overfitting in handwritten digit recognition; real-time detection on the validation dataset, by limiting the threshold of loss decrease, automatically stops training when the model's handwritten digit recognition performance no longer improves, thus avoiding wasted computation.

[0050] Further, the network weights are validated using the validation dataset, and the loss function values ​​of the network weights are calculated, including:

[0051] The validation dataset is used to predict the digit recognition model based on the network weights, and the loss function value of the network weights is calculated based on the prediction result of the digit recognition model on the validation dataset and the actual classification result of the validation dataset.

[0052] As described above, by verifying the loss calculation between the predicted results and the actual results of the dataset, it is possible to discover the confusion tendency of the model to approximate handwritten numbers; and the loss function value directly reflects the generalization ability. For example, when recognizing machine-printed numbers and handwritten numbers, the extraction of digit edge features can be enhanced in a targeted manner.

[0053] Further, based on the loss function value, it is determined whether to adjust the adaptive learning rate. If so, the adaptive learning rate is updated by calculating the negative gradient normalized data of the loss function value, including:

[0054] If the loss function value meets the decrease range condition or fluctuation range condition, then the negative gradient of the loss function value is normalized, and the adaptive learning rate is adjusted using the negative gradient normalization weight.

[0055] Determine whether the adjusted adaptive learning rate has reached the learning rate saturation point. If not, update the adaptive learning rate for the next round of training based on the adjusted adaptive learning rate.

[0056] As described above, by using the decrease magnitude condition or fluctuation magnitude condition, abnormal samples can be automatically detected, triggering negative gradient normalization adjustment. Furthermore, learning rate saturation point detection can prevent over-adjustment of handwritten digits, thus allowing the updated learning rate to quickly adapt to handwritten digit handwriting. Adjusting the learning rate after network weight training is completed is to dynamically optimize the learning rate parameter based on the loss function value. By calculating the negative gradient normalization data, the learning rate is precisely fine-tuned, avoiding weight oscillations or convergence stagnation caused by inappropriate learning rates in the later stages of training. This allows the selection of weights corresponding to the minimum loss function value on the test dataset, further improving the model's prediction accuracy and generalization ability.

[0057] Please refer to Figure 2 Another embodiment of the present invention provides a handwritten digit image recognition terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps of the above-described handwritten digit image recognition method.

[0058] The image recognition method and terminal for handwritten digits described above are applicable to effectively improving the recognition efficiency and accuracy of handwritten digits. The following is a detailed description of specific implementation methods:

[0059] Please refer to Figure 1 and Figure 3 Embodiment 1 of the present invention is as follows:

[0060] S1. Obtain a sample dataset of handwritten digits, and set lightweight parameters for the digit recognition model based on the image structure of the sample dataset. The lightweight parameters include momentum coefficient, initial learning rate, and sampling frequency.

[0061] In this embodiment, the sample dataset is the MNIST dataset, which contains 60,000 training images and 10,000 test images. Each image is a 28×28 pixel grayscale image of a handwritten digit. The goal is to train the model to accurately classify these images into the ten digit categories from 0 to 9.

[0062] The lightweight parameters include the momentum coefficient γ, the initial learning rate α0, and the sampling frequency. The momentum coefficient is used to accelerate convergence, especially during the feature mapping stage of the fully connected layers, and is preferably 0.9. The initial learning rate is used to avoid parameter oscillations during the initial training of convolutional layers, and is preferably 0.001. The sampling frequency, which is the number of training samples per batch, is set to 64 to balance memory consumption and gradient estimation stability. These parameters lay the foundation for the subsequent training process.

[0063] S2. Divide the sample dataset into a training dataset, a validation dataset, and a test dataset. Use the training dataset to train the network weights of the digit recognition model by using a stochastic gradient descent optimization method that integrates adaptive learning rate and second-order information.

[0064] In this embodiment, the network structure of the recognition network model includes convolutional layers, pooling layers, and fully connected layers. The convolutional layer includes a 3×3 convolutional kernel with 1 input channel and 32 output channels, used to extract edge features; the pooling layer includes 2×2 max pooling, used to reduce feature dimensionality; the fully connected layer can convert the flattened pixel count (784) into the number of categories (10), used for probability prediction.

[0065] Initialize convolution kernel weights W c : Follows a normal distribution with a mean of 0 and a standard deviation of 0.1, and fits the image pixel value range [0, 255]; initializes the fully connected layer weights W f Random initialization, bias term b c b f Set to 0.

[0066] Forward propagation: Input 64 randomly selected MNIST images (such as numbers like "7" and "2"), size 28×28, normalized to [0,1]; Convolutional layer calculation: FeatureMap = ReLU(W c *Input+b c Extract basic features such as "vertical lines" and "horizontal lines";

[0067] Pooling layer output: PooledFeature = MaxPooling(FeatureMap, 2×2), size reduced to 14×14;

[0068] Fully connected layer output: Logits = W f ×Flatten(PooledFeature)+b f The classification probability is obtained by Softmax.

[0069] Calculate the cross-entropy loss:

[0070]

[0071] Among them, y i This represents the actual label, such as "5" corresponding to the one-hot encoding [0,0,0,0,0,1,0,0,0,0]). This represents the predicted probability value of the digit recognition model obtained after processing the i-th sample using the Softmax function.

[0072] Backpropagation: Calculates the gradient of the fully connected layer, reflecting the impact of classification error on the weights.

[0073]

[0074] Convolutional layer gradient: calculated through backpropagation using the chain rule. Optimization directions for the feature extraction layer.

[0075] Calculate the second-order information parameter H of the gradient l If the current batch contains a large number of "complex numbers" (such as illegible "9"), ||δ l || 2 Increase, H l The algorithm responds quickly, suggesting that the learning rate should be adjusted carefully.

[0076] Step S2 specifically includes the following steps:

[0077] S21. Initialize the training rounds l=0 and accumulate the gradient average.

[0078] S22. Randomly traverse the training samples of the current round from the sample dataset according to the sampling frequency, calculate the partial derivative of the loss function gradient with respect to the weights, and obtain the gradient value of the current batch:

[0079]

[0080] S23. Calculate the second-order information parameter H of the gradient. l :

[0081]

[0082] In the formula, G represents l transpose, |g l | 2 Represents the square of the gradient magnitude. This represents the exponential moving average of the square of the gradient magnitude, i.e. μ represents the coefficient of the squared exponential moving average of the gradient magnitude, μ∈(0,1).

[0083] S24. Calculate the numerator parameter S of the adaptive learning rate. l The initial value is 0:

[0084] S l =βS l-1 +(1-β)Clip(H l -θ||δ l || 2 )

[0085] In the formula, the Clip function is used to move the variable closer to 0, ensuring that the learning rate α is not less than 0; β is used to control the influence of the variance of the accumulated historical gradients, controlling the weight of the historical gradients, β∈(0,1); when H l Much greater than θ||δ l || 2 When (e.g., the curvature of the loss function surface is large), S l Increase the denominator D of the learning rate l As it increases, it forces the learning rate α to increase. l Reduce the step size to avoid parameter oscillations caused by "excessive step size" in steep regions.

[0086] Wherein, the denominator parameter D in the adaptive learning rate formula l The calculation formula is:

[0087] S25. Calculate the adaptive learning rate α l :

[0088]

[0089] In this embodiment, during the initial training phase (e.g., the first 5 rounds): S l Smaller, α l ≈0.01, quickly adjust the convolution kernel weights to capture edge features;

[0090] Later in the training period (e.g., after round 10): S l Increase, α l The weights of the fully connected layers are reduced to below 0.001, and the weights are finely optimized to improve classification accuracy.

[0091] S26. Momentum calculation: m l =γm l-1 +α l δ l (γ=0.9)

[0092] In the convolutional layer, the historical momentum m l-1 It can accumulate edge features from multiple batches to optimize the direction, avoiding gradient bias caused by a single batch of "noisy samples" (such as blurred images).

[0093] S27, Weight Update: M l+1 =W l -(m l +δ l )

[0094] For the weights Wf of the fully connected layer, the momentum and the current gradient are combined to make small step adjustments at the classification boundary (such as near the probability thresholds of "4" and "9") to reduce misclassification.

[0095] S28. Determine whether training is complete. If yes, proceed to step S3; otherwise, return to step S22.

[0096] The maximum number of rounds is preset to 20, or the validation set loss decreases by less than 1e-4 for three consecutive rounds.

[0097] If the condition is not met, return to the step of calculating the gradient of the loss function and continue training until the model's classification error for difficult samples such as "tilted numbers" and "broken pen numbers" stabilizes and decreases.

[0098] Optimizing the training set alone is insufficient to fully validate the model's generalization ability and actual performance, as good performance on the training set does not guarantee equally good performance on new data. Therefore, to comprehensively evaluate the model's training effect and avoid overfitting, it is necessary to test the currently trained weights using a validation dataset independent of the training set. Step S3 involves using the validation dataset to predict the weights w(l) during training, calculating the loss function value J(w(l)) of the current weights, and further determining whether the model has reached an ideal training state. This determines whether to continue adjusting the weights to optimize the model or proceed to the testing phase to obtain the final prediction results.

[0099] S3. Use the verification dataset to verify the network weights, calculate the loss function value of the network weights, and determine whether the training of the network weights is complete based on the loss function value. If the training is not complete, return to the step of training the network weights of the digit recognition model using the training dataset.

[0100] Specifically, the validation dataset is used to make predictions on the weights w(l) during the training process, the loss function value J(w(l)) of the current weights is calculated, and it is determined whether the training is complete. If it is complete, the next step is taken.

[0101] In step S2, network weights are trained using a stochastic gradient descent optimization algorithm that integrates adaptive learning rate and second-order information on a deep network training sample dataset. The network weights are continuously adjusted to minimize the loss function. In step S3, based on the weights generated during training in step S2, predictions are made using a validation dataset, and the loss function value corresponding to the current weight is calculated to evaluate the training effect and determine whether training is complete. Step S3 performs performance verification based on the weights obtained in step S2, providing a basis for whether to continue training or adjust parameters. Together, these two steps constitute a complete process from model training to performance verification, ensuring that the training process progresses towards the optimal weights.

[0102] After completing step S3, which involves prediction on the validation dataset and evaluating the training effect, if the network's loss function value on the validation set has converged to an ideal level and the model performance is stable through adjusting the adaptive learning rate and repeated training, the training process is confirmed to be complete. At this point, to comprehensively test the model's generalization ability and prediction accuracy in real-world application scenarios, the trained model needs to be placed on a test dataset that was not involved in training or validation for final testing. Therefore, in step S4, network prediction is performed on the test dataset. By comparing the loss function value J(w(l)) under each round of weights, the weight w(l*) corresponding to the minimum loss function value J(w*) is selected as the prediction result, thereby obtaining the optimal weights and the best prediction effect, completing the entire model training, optimization, and evaluation process.

[0103] S4. If training is complete, determine whether to adjust the adaptive learning rate based on the loss function value. If yes, update the adaptive learning rate by calculating the negative gradient normalized data of the loss function value. Otherwise, compare the loss function values ​​of the network weights obtained in each training round on the test dataset, and select the network weights corresponding to the minimum loss function value as the network weights of the digit recognition model.

[0104] Specifically, network prediction is performed on the test dataset. The loss function value J(w(l)) under the weights in each round is compared, and the weight w(l*) corresponding to the minimum loss function value J(w*) is taken as the prediction result to obtain the optimal weight and the best prediction result.

[0105] The weight update process in step S4 is as follows:

[0106] S41. Begin weight update.

[0107] S42. Determine whether the adaptive learning rate has reached the saturation point. If not, continue; if so, proceed to step S45.

[0108] S43. Update the adaptive learning rate for round l+1 based on the updated adaptive learning rate parameters, specifically as follows:

[0109] Update the momentum parameter m of wheel l+1 l+1 =γm l +α l+1 g l+1 ;

[0110] The update method for calculating the gradient in round l+1:

[0111]

[0112] According to v l+1 =m l+1 +g l+1Accumulate momentum and gradient, based on ω l+1 =ω l -m l+1 Calculate the weights for round l+1.

[0113] S44. Adaptive learning rate update complete, reacquire loss function value J(w(l)).

[0114] S45. When the adaptive learning rate reaches its saturation point, it does not need to be updated again. Using the updated adaptive learning rate parameter α(l+1), a new round of weight updates is performed, i.e., the l+1th round, and the process returns to step S42.

[0115] S5. Use the digit recognition model to recognize and classify the input handwritten digit image.

[0116] Therefore, in stage S4, a complete test dataset is selected, and network prediction is performed on the test dataset. By comparing the loss function values ​​of the weights obtained in each training round on the test set, the weight corresponding to the minimum loss function value is selected as the final prediction result. The optimal weights obtained at this time are the best model parameters obtained after training, optimization, and validation in this image recognition task. The model built using these weights can achieve the best prediction performance in the handwritten digit recognition task and accurately classify newly input handwritten digit images.

[0117] Through practical application in the specific scenario of handwritten digit recognition, this embodiment realizes the complete process from parameter setting, network training, effect verification to final testing, and fully demonstrates the advantages of the stochastic gradient descent optimization algorithm that integrates adaptive learning rate and second-order information in improving model training efficiency and prediction accuracy.

[0118] Embodiment 2 of the present invention is as follows:

[0119] Based on Example 1, steps S24 to S28 can be modified as follows:

[0120] S24': Calculate the sum parameter S in the numerator of the adaptive learning rate formula according to the following formula. l The initial value is 0:

[0121]

[0122] The function Clip is used to move the variable closer to 0, ensuring that the value of α is not less than 0. The parameter β controls the influence of the variance of the accumulated historical gradient value and controls the weight of the historical gradient, with a value range of 0-1. μ is the coefficient of the squared exponential moving average of the gradient magnitude, μ∈(0,1).

[0123] S25': Calculate the update rate parameter R in the numerator of the adaptive learning rate formula according to the following formula. l:

[0124] R l =S l -S l-1

[0125] S26': Calculate the parameter D in the denominator of the adaptive learning rate formula according to the following formula. l :

[0126]

[0127] Where θ is the learning rate sensitivity coefficient, θ∈(0,1).

[0128] S27': Calculate the algorithm's adaptive learning rate, momentum, and partial derivatives, and accumulate the gradients. Specifically, this includes: calculating the adaptive learning rate:

[0129]

[0130] And based on the calculated momentum parameters:

[0131]

[0132] Where γ is the momentum coefficient;

[0133] Calculate the gradient value for the current batch:

[0134]

[0135] According to v l =m l +g l Perform an accumulation operation on momentum and gradient.

[0136] S28': Calculate the weights ω for the new round. l And the weight ω from the previous round l-1 With momentum parameter m l Add them together and update the weights:

[0137] ω l =ω l-1 -m l

[0138] Where the label l represents the number of training iterations, l is a natural number, and ω l-1 This is the weight of the previous round, i.e., the (l-1)th round; due to the momentum parameter m l Essentially, it is the accumulation of gradients, with the direction being the upward direction of the loss function. However, weight updates need to be performed along the negative gradient direction of the downward direction of the loss function. Therefore, the momentum parameter is subtracted from the previous round of weights to achieve iteration towards the optimal solution.

[0139] S29': Determine if training is complete. If complete, proceed to the next step; otherwise, return to step S22'.

[0140] Therefore, in stage S2, network weight training begins, with training epoch l initialized to 0. In S22, 64 images are randomly selected from the training dataset as the current batch of training samples. The partial derivative of the gradient of the loss function (here, the cross-entropy loss function is used to measure the difference between the model's prediction and the true label) with respect to the weights is calculated. This calculates the gradient generated by the current batch of training samples on the weights of each layer of the neural network, determining the direction and magnitude of weight updates. In S23, second-order information parameters are calculated based on the gradients. This second-order information reflects the trend of gradient changes, providing richer information for adaptive adjustment of the learning rate. In S24'-S26', based on the second-order gradient information and historical gradients, the sum parameter of the numerator, the update rate parameter, and the parameter of the denominator in the adaptive learning rate formula are calculated to obtain the adaptive learning rate for the current epoch. In S27'-S28', the gradient is accumulated by combining the adaptive learning rate and momentum, and the network weights are updated accordingly. This allows the model to continuously optimize parameters during training, gradually improving the recognition accuracy of the training samples. In S29', it is determined whether the current training has met the preset conditions (such as reaching the maximum number of training rounds, the loss function value no longer decreasing significantly, etc.). If not, it returns to S22' to continue to extract new training samples for training; if it is completed, it proceeds to the next stage.

[0141] Please refer to Figure 4 Embodiment 3 of the present invention is as follows:

[0142] Building upon Example 1, while the adaptive learning rate can dynamically adjust its step size during deep network training, the initial combination of adaptive learning rate parameters may not always remain optimal due to the complexity of data distribution, the diversity of network structures, and uncertainties in the training process. As training epochs progress, the network may experience problems such as slower convergence speed and oscillations near local optima, preventing further improvement in model performance.

[0143] Therefore, after evaluating the training weights using the validation dataset in step S3, in order to better adapt the adaptive learning rate to the dynamic changes in network training and further improve the model's generalization ability and convergence efficiency, it is necessary to dynamically evaluate and adjust the adaptive learning rate. By determining whether to adjust the adaptive learning rate, the algorithm can flexibly optimize the learning rate parameters while ensuring training stability, ensuring that the network can run with the optimal parameter configuration when entering the test dataset prediction stage in step S4, thereby obtaining better prediction results. The specific judgment and operation are as follows:

[0144] Determine whether to perform adaptive learning rate adjustment. If yes, proceed to step S31; otherwise, proceed to step S4.

[0145] S31. Adjust the adaptive learning rate. Based on network performance optimization, adjust the adaptive learning rate and determine whether the network parameters are optimized. If yes, proceed to step S32; otherwise, proceed to step S29.

[0146] S32. Calculate the function value J(w(l)) = f(w) according to the following formula. l ), where w(l) are the training weights, and J(w(l)) is used to evaluate the network's prediction performance.

[0147] S33. Find the negative gradient of J(w(l)) and normalize it:

[0148]

[0149] In the formula, J'(l) is the gradient of the function value J(w(l)), l represents the iteration, and is the weight of the normalized negative gradient of the function value J(w(l)). The specific method for calculating and normalizing the negative gradient of the function value J(w(l)) is as follows: During training, an exponential smoothing function is used to update the weight of the negative gradient of the function value based on the exponential moving average of the square of the gradient magnitude. Let... The weights of the negative gradient, which is the exponential moving average of the squared magnitude of the gradient, are:

[0150]

[0151] After normalizing the exponential smoothing coefficients, the negative gradient of the function value is calculated using the following formula:

[0152]

[0153] When |J'(l)|≠0, j'(l) is the normalized weight.

[0154] S34. Update the parameters of the adaptive learning rate using the weights normalized by the negative gradient of the function value J(w(l)), obtaining new adaptive learning rate parameter values. This is done using the learning rate saturation point formula:

[0155]

[0156] The adaptive learning rate parameter is updated using the above formula. If α(l+1) reaches the learning rate saturation point, the adaptive learning rate no longer changes. Returning to step S33, the negative gradient of the function value J(w(l)) is recalculated and normalized. Here, μ is the coefficient of the squared exponential moving average of the gradient magnitude, and Clip is used to move the variable closer to 0, ensuring that the value of α is not less than 0. The expression for the function Clip is: when the value of ∈ is greater than or equal to the threshold ∈ lim When α = α, the value of the function is θ, where θ is the learning rate sensitivity coefficient, and α is the learning rate sensitivity coefficient.l+1 This represents the adaptive learning rate in the (l+1)th round.

[0157] S35. Determine if the adaptive learning rate has reached saturation. If yes, return to S33; otherwise, update the adaptive learning rate for round l+1 based on the updated adaptive learning rate parameter value and return to S27. When the update condition α(l+1)≠α(l) is met, the adaptive learning rate update ends, return to step S27, and use the updated adaptive learning rate parameter α(l+1) to continue updating the weights for a new round, i.e., round l+1.

[0158] In the training of deep learning models, the dynamic adjustment of the adaptive learning rate is a crucial step in balancing training efficiency and convergence. The steps described above intelligently determine whether the adaptive learning rate needs optimization through real-time evaluation of network performance. If the determination is "yes," steps S31-S35 fine-tune the learning rate parameter based on the gradient information of the loss function, enabling the model to find the optimal solution more efficiently in the complex parameter space. If the determination is "no," it indicates that the current learning rate setting is sufficient for training requirements, and the model can directly proceed to the testing phase. This approach avoids the computational overhead of unnecessary parameter adjustments and allows for timely optimization when model performance is hindered, effectively ensuring the stability and efficiency of the training process. It ensures that the model reaches its optimal training state before entering the final test, thus providing a foundation for obtaining better prediction results.

[0159] Therefore, between S3 and S4, a judgment is made regarding the adaptive learning rate adjustment. If it is determined that the learning rate needs adjustment (for example, if the loss function value on the validation set decreases slowly or fluctuates), then proceed to steps S31-S35, where the parameters of the adaptive learning rate are updated by calculating and normalizing the negative gradient of the loss function value, so that the learning rate better adapts to the needs of model training; if it is determined that no adjustment is needed, then proceed directly to stage S4.

[0160] Please refer to Figure 2 Embodiment four of the present invention is as follows:

[0161] A handwritten digit image recognition terminal 1 includes a memory 2, a processor 3, and a computer program stored in the memory 2 and executable on the processor 3. When the processor 3 executes the computer program, it implements the various steps of a handwritten digit image recognition method according to any one of the embodiments 1 to 3.

[0162] In summary, the image recognition method and terminal for handwritten digits provided by this invention have the following beneficial effects:

[0163] (1) Existing stochastic gradient descent algorithms for handwritten digit recognition models often employ fixed learning rates or simple adaptive learning rate adjustment strategies, making it difficult to achieve a balance between rapid convergence in the early stages of training and avoiding excessive oscillations in the later stages, resulting in long training times and high resource consumption. This invention, however, significantly improves training efficiency by combining second-order gradient information with an adaptive learning rate mechanism in the handwritten digit recognition model. This is achieved by calculating the second-order gradient information parameters and using the exponential moving average to capture the gradient change trend, combined with the sum parameter S in the numerator of the adaptive learning rate formula. l Update rate parameter R l and the parameter D in the denominator l The dynamic calculation allows the learning rate to be adjusted in real time according to the training progress. In the early stages of training, a larger learning rate can be used to quickly move towards the optimal solution; as training progresses, the learning rate automatically decreases when approaching the optimal solution, avoiding missing the optimal solution due to excessively large step sizes, thereby effectively shortening the overall training time and reducing the consumption of computing resources.

[0164] (2) Traditional stochastic gradient descent algorithms for handwritten digit recognition models rely solely on first-order derivative information to update weights, making them prone to getting trapped in local optima and struggling to find globally optimal network parameters. This invention introduces second-order information and deeply integrates it into the adaptive learning rate calculation process, providing richer information dimensions for parameter optimization. In the specific implementation, by processing the second-order gradient information using an exponential moving average and combining it with the variance control parameter β of the historical gradient accumulation value, the adjustment of the adaptive learning rate considers not only the current gradient direction but also the curvature information of the gradient change. This allows for a more accurate determination of the parameter update direction when facing complex loss function surfaces, avoiding getting trapped in local optima. For example, when calculating the adaptive learning rate, by combining second-order information with gradient magnitude information, the numerator parameter of the learning rate is dynamically adjusted according to the gradient characteristics of different training stages, thereby achieving more refined weight updates, significantly improving the accuracy of parameter optimization, and enabling the model to approach the globally optimal solution more closely.

[0165] (3) In the training process of handwritten digit recognition models, existing technologies often lack a dynamic feedback mechanism for adjusting the learning rate parameter to reflect changes in network performance. This can easily lead to the model performing well on the training set but declining in performance on new data such as the test set, resulting in overfitting. The adaptive learning rate adjustment mechanism introduced in this invention evaluates the training weights in real time using a validation dataset and dynamically optimizes the learning rate parameter based on network performance, effectively enhancing the model's generalization ability. Specifically, when the algorithm determines that a learning rate adjustment is needed, it calculates and normalizes the negative gradient of the loss function value, updates the negative gradient weights using the second-order gradient information moving average, and then updates the adaptive learning rate parameter. For example, in the process of judging and adjusting the learning rate saturation point, the learning rate is kept in an optimal state throughout the training process, avoiding overfitting due to an excessively large learning rate or underfitting due to an excessively small learning rate. Through this dynamic adjustment mechanism, the model can better adapt to the data distribution characteristics during training, reducing overfitting to the training data, thereby maintaining good predictive performance when facing new data and significantly improving generalization ability.

[0166] (4) Some existing adaptive learning rate algorithms suffer from excessive learning rate fluctuations during training due to inflexible learning rate adjustment strategies, leading to instability and even failure to converge. This invention ensures the stability of the training process through multiple mechanisms. On the one hand, the Clip function is used to bring the adaptive learning rate parameter closer to 0, ensuring that the learning rate α is within the range of [0,1], thus avoiding abnormal parameter updates due to an excessively large learning rate. On the other hand, dynamic calculation of the momentum parameter, combined with gradient accumulation operation v, is used. l =m l +g l This makes the weight update process smoother. Simultaneously, during the adaptive learning rate adjustment, strict saturation point judgment and parameter update logic stop unnecessary adjustments when the learning rate reaches saturation, ensuring the stability of the training process. These techniques work together to effectively suppress drastic fluctuations in the learning rate during training, enabling the model to optimize parameters in a stable state and providing a reliable guarantee for successful training.

[0167] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for image recognition of handwritten digits, characterized in that, Including the following steps: Obtain a sample dataset of handwritten digits, and set lightweight parameters for the digit recognition model based on the image structure of the sample dataset. The lightweight parameters include momentum coefficient, initial learning rate, and sampling frequency. The sample dataset is divided into a training dataset, a validation dataset, and a test dataset. The network weights of the digit recognition model are trained using the training dataset by a stochastic gradient descent optimization method that integrates adaptive learning rate and second-order information. The network weights are validated using the validation dataset, the loss function value of the network weights is calculated, and the training of the network weights is determined based on the loss function value. If the training is not completed, the process returns to the step of training the network weights of the digit recognition model using the training dataset. If training is complete, determine whether to adjust the adaptive learning rate based on the loss function value. If yes, update the adaptive learning rate by calculating the negative gradient normalized data of the loss function value. Otherwise, compare the loss function values ​​of the network weights obtained in each training round on the test dataset, and select the network weights corresponding to the minimum loss function value as the network weights of the digit recognition model. The digit recognition model is used to recognize and classify input handwritten digit images.

2. The image recognition method for handwritten digits according to claim 1, characterized in that, The network weights of the digit recognition model are trained using the training dataset through a stochastic gradient descent optimization method that integrates second-order information and an adaptive learning rate, including: In each training round, the adaptive learning rate α is calculated based on second-order information. l : S l =βS l-1 +(1-β)Clip(H l -θ||δ l || 2 ) In the formula, α0 represents the initial learning rate, S l D represents the numerator parameter of the learning rate in round l. l H represents the denominator parameter of the learning rate in the l-th round, β represents the weights controlling the historical gradient data, and H... l Let g represent the second-order information parameters of the gradient of the network weights in the l-th round. l Let δ represent the gradient of the network weights in the l-th round, ∈ represent a preset constant, and δ l The Clip function represents the normalized value of the gradient, indicating that the variable is brought closer to 0 so that the learning rate α is equal to the normalized value of the gradient. l The value of is not less than 0; The momentum parameter m is calculated based on the adaptive learning rate. l : m l =γm l-1 +a l d l In the formula, γ represents the momentum coefficient, and m l-1 This represents the momentum parameter of the previous cycle; Update the network weights based on the momentum parameters: W l =W l-1 -(m l +δ l ) In the formula, W l-1 W represents the network weight in round l-1. l This represents the network weight in round l.

3. The image recognition method for handwritten digits according to claim 2, characterized in that, Training the network weights of the digit recognition model using the training dataset further includes: The training of the network weights of the digit recognition model ends when the number of training rounds reaches the preset maximum number of rounds, or when the loss function value of the verification dataset decreases by less than a threshold for a preset number of consecutive times.

4. The image recognition method for handwritten digits according to claim 1, characterized in that, The network weights are validated using the validation dataset, and the loss function values ​​of the network weights are calculated, including: The validation dataset is used to predict the digit recognition model based on the network weights, and the loss function value of the network weights is calculated based on the prediction result of the digit recognition model on the validation dataset and the actual classification result of the validation dataset.

5. The image recognition method for handwritten digits according to claim 1, characterized in that, Based on the loss function value, determine whether to adjust the adaptive learning rate. If so, update the adaptive learning rate by calculating the negative gradient normalized data of the loss function value, including: If the loss function value meets the decrease range condition or fluctuation range condition, then the negative gradient of the loss function value is normalized, and the adaptive learning rate is adjusted using the negative gradient normalization weight. Determine whether the adjusted adaptive learning rate has reached the learning rate saturation point. If not, update the adaptive learning rate for the next round of training based on the adjusted adaptive learning rate.

6. A handwritten digit image recognition terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it performs the following steps: Obtain a sample dataset of handwritten digits, and set lightweight parameters for the digit recognition model based on the image structure of the sample dataset. The lightweight parameters include momentum coefficient, initial learning rate, and sampling frequency. The sample dataset is divided into a training dataset, a validation dataset, and a test dataset. The network weights of the digit recognition model are trained using the training dataset by a stochastic gradient descent optimization method that integrates adaptive learning rate and second-order information. The network weights are validated using the validation dataset, the loss function value of the network weights is calculated, and the training of the network weights is determined based on the loss function value. If the training is not completed, the process returns to the step of training the network weights of the digit recognition model using the training dataset. If training is complete, determine whether to adjust the adaptive learning rate based on the loss function value. If yes, update the adaptive learning rate by calculating the negative gradient normalized data of the loss function value. Otherwise, compare the loss function values ​​of the network weights obtained in each training round on the test dataset, and select the network weights corresponding to the minimum loss function value as the network weights of the digit recognition model. The digit recognition model is used to recognize and classify input handwritten digit images.

7. The image recognition terminal for handwritten digits according to claim 6, characterized in that, The network weights of the digit recognition model are trained using the training dataset through a stochastic gradient descent optimization method that integrates second-order information and an adaptive learning rate, including: In each training round, the adaptive learning rate α is calculated based on second-order information. l : S l =βS l-1 +(1-β)Clip(H l -θ||δ l || 2 ) In the formula, α0 represents the initial learning rate, S l D represents the numerator parameter of the learning rate in round l. l H represents the denominator parameter of the learning rate in the l-th round, β represents the weights controlling the historical gradient data, and H... l Let g represent the second-order information parameters of the gradient of the network weights in the l-th round. l Let δ represent the gradient of the network weights in the l-th round, ∈ represent a preset constant, and δ l The Clip function represents the normalized value of the gradient, indicating that the variable is brought closer to 0 so that the learning rate α is equal to the normalized value of the gradient. l The value of is not less than 0; The momentum parameter m is calculated based on the adaptive learning rate. l : m l =γm l-1 +a l d l In the formula, γ represents the momentum coefficient, and m l-1 This represents the momentum parameter of the previous cycle; Update the network weights based on the momentum parameters: W l =W l-1 -(m l +δ l ) In the formula, W l-1 W represents the network weight in round l-1. l This represents the network weight in round l.

8. The image recognition terminal for handwritten digits according to claim 7, characterized in that, Training the network weights of the digit recognition model using the training dataset further includes: The training of the network weights of the digit recognition model ends when the number of training rounds reaches the preset maximum number of rounds, or when the loss function value of the verification dataset decreases by less than a threshold for a preset number of consecutive times.

9. The image recognition terminal for handwritten digits according to claim 6, characterized in that, The network weights are validated using the validation dataset, and the loss function values ​​of the network weights are calculated, including: The validation dataset is used to predict the digit recognition model based on the network weights, and the loss function value of the network weights is calculated based on the prediction result of the digit recognition model on the validation dataset and the actual classification result of the validation dataset.

10. An image recognition terminal for handwritten digits according to claim 6, characterized in that, Based on the loss function value, determine whether to adjust the adaptive learning rate. If so, update the adaptive learning rate by calculating the negative gradient normalized data of the loss function value, including: If the loss function value meets the decrease range condition or fluctuation range condition, then the negative gradient of the loss function value is normalized, and the adaptive learning rate is adjusted using the negative gradient normalization weight. Determine whether the adjusted adaptive learning rate has reached the learning rate saturation point. If not, update the adaptive learning rate for the next round of training based on the adjusted adaptive learning rate.

Images