A method and system for handwritten digit recognition based on fractional-order back-propagation neural network

Abstract

The present invention belongs to the field of machine learning technology and is a method and system for handwritten digit recognition using a fractional-order back-propagation neural network. The method comprises the following steps: preprocessing images of a handwritten digit dataset; constructing a dual cross-entropy loss function with a regularization term; constructing a fractional-order back-propagation neural network model with a regularization term; setting relevant parameters of the fractional-order back-propagation neural network based on the preprocessed images, training the fractional-order back-propagation neural network to obtain a neural network model suitable for the handwritten digit dataset, inputting a handwritten digit image to be recognized, and classifying the handwritten digit image. The present invention introduces a squared penalty term and a new dual cross-entropy loss function into the fractional-order neural network, resulting in high precision, rapid convergence, and good generalization performance, enabling accurate and rapid handwritten digit recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to a handwritten digit recognition method and system based on a fractional-order back-propagation neural network with a regularization term. Background Art

[0002] In the age of intelligent computing, the importance of handwritten digit recognition is reflected in all aspects of daily life. For example, in the financial market, bills contain a large amount of handwritten digits. Handwritten digit recognition can effectively solve the low efficiency, time-consuming and high labor costs of traditional manual processing methods. Therefore, the development of handwritten digit recognition technology has important practical significance.

[0003] With the development of machine learning theory, the effectiveness and feasibility of many machine learning methods have been guaranteed, such as neural network methods. Due to their long memory and more degrees of freedom, fractional-order derivatives have been successfully adopted in the field of machine learning. Compared with traditional integer-order neural network methods, fractional-order derivative neural network methods have been shown to have higher accuracy and faster convergence speed. First, good generalization performance is very important for neural networks to learn data features, especially in the case of small sample data sets. Second, the gradient vanishing problem causes the network weights to not be updated, ultimately leading to training failure; therefore, alleviating the gradient vanishing problem is quite important for neural networks. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a handwritten digit recognition method and system based on a fractional-order back-propagation neural network with a regularization term. A square penalty term and a new double cross-entropy loss function are introduced into the fractional-order neural network. The method has the advantages of high precision, fast convergence and good generalization performance, and can accurately and quickly recognize handwritten digits.

[0005] The technical solution adopted by the method of the present invention is as follows: a handwritten digit recognition method based on a fractional-order back-propagation neural network with a regularization term, comprising the following steps:

[0006] Preprocess the images of the handwritten digit dataset;

[0007] Construct a double cross entropy loss function with regularization term;

[0008] Construct a fractional-order back-propagation neural network model with regularization term;

[0009] According to the preprocessed images, the relevant parameters of the fractional-order back-propagation neural network are set, and the fractional-order back-propagation neural network is trained to finally obtain a neural network model suitable for the handwritten digit dataset;

[0010] In the neural network model suitable for the handwritten digit dataset, the handwritten digit image to be recognized is input and the handwritten digit image is classified.

[0011] Preferably, the preprocessing of the image includes:

[0012] The input data of the original handwritten digit dataset is a rectangular image matrix, which is reordered row by row into a single-row and multi-column vector;

[0013] Convert the labels of the handwritten digit dataset into one-hot vectors.

[0014] Preferably, the constructing of a double cross entropy loss function with a regularization term includes:

[0015] Define separately and is an i-dimensional real number set and a j×l-dimensional real number set; given J training samples (x j ,o j ),in is the jth input, is the jth ideal output, j = 1, 2, ..., J; define a three-layer neural network, where the number of neurons in the input layer, hidden layer, and output layer are p, n, and q respectively; the output of the neural network is written as:

[0016] y=f(UG(Vx j ))

[0017] where V is the weight matrix between the input layer and the hidden layer, and U is the weight matrix between the output layer and the hidden layer; G(·)=(g(·),g(·),...,g(·)) T , g(·) and f(·) are the activation functions of the hidden layer and the output layer respectively; where:

[0018]

[0019] is the element of the weight matrix U between the output layer and the hidden layer;

[0020] is the element of the weight matrix V between the input layer and the hidden layer;

[0021] make The loss function in the neural network is represented as E(w);

[0022] set up is the actual output of the neural network model, where the lth element f jl Represents x j The actual probability of being assigned to the lth class; the double cross entropy loss function is designed as:

[0023]

[0024] where f jl =f(u l G(Vx j )),β is the penalty coefficient for misclassification, β≥0; (1-o jl )ln(1-f jl ) is the penalty term; definition:

[0025] ζ jl =u l G(Vx j ) and F jl =F(ζ jl )=-o jl ln(f jl )-β(1-o jl )ln(1-f jl ).

[0026] The double cross entropy loss function with regularization term is:

[0027]

[0028] where L2(w) = λ / 2‖w‖ 2 ,‖w‖ represents the Euclidean norm of w, λ is the regularization parameter, λ≥0.

[0029] Preferably, the Caputo derivative operator of the Caputo fractional derivative is The order is The Caputo fractional derivative is defined as:

[0030]

[0031] in, is a set of positive integers; is the objective function, yes The r-order derivative with respect to τ is [c,t] is a closed integration interval, Γ(·) is the Gamma function,

[0032] Preferably, the constructing of a fractional-order back-propagation neural network model with a regularization term includes:

[0033] The Caputo fractional derivative is expressed as a Taylor series expansion based on the Caputo derivative, wherein the high-order terms of the Taylor series expansion are truncated, while the first-order terms of the loss function E(w) are retained;

[0034] Design truncated fractional-order neural networks;

[0035] Design a hybrid truncated fractional-order BP neural network;

[0036] The weight update rules of truncated fractional-order neural networks and hybrid truncated fractional-order neural networks are obtained.

[0037] The technical solution adopted by the system of the present invention is as follows: a handwritten digit recognition system using a fractional-order back-propagation neural network, comprising the following modules:

[0038] A preprocessing module, used to preprocess the images of the handwritten digit dataset;

[0039] Loss function construction module, used to construct the double cross entropy loss function with regularization term;

[0040] Network model construction module, used to build a fractional-order back-propagation neural network model with regularization terms;

[0041] A training module is used to set relevant parameters of the fractional-order back-propagation neural network based on the preprocessed image, train the fractional-order back-propagation neural network, and ultimately obtain a neural network model suitable for the handwritten digit dataset;

[0042] The classification module is used to input the handwritten digit images to be recognized into the neural network model suitable for the handwritten digit dataset and classify the handwritten digit images.

[0043] Compared with the prior art, the technical effects achieved by the present invention include:

[0044] 1. Based on a fractional-order back-propagation neural network with a regularization term, this invention efficiently and accurately establishes a systematic relationship between handwritten digit images and their corresponding digits. Furthermore, a visual interface displays the handwritten digit images to be recognized and their corresponding digits, providing a more intuitive display of classification and recognition results, meeting the requirements for handwritten digit recognition.

[0045] 2. The present invention introduces a square penalty term into the fractional-order neural network, thereby achieving better generalization performance.

[0046] 3. This paper proposes a new dual cross entropy loss function as the loss function of the neural network, thereby achieving higher accuracy.

[0047] 4. The fractional-order neural network used in the present invention has the ability to converge to true extreme points, ensuring the true convergence ability of the proposed neural network. In the technical solution of the present invention, the fractional-order neural network with high precision, fast convergence and good generalization performance can accurately and quickly recognize handwritten digits. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Flowchart of a handwritten digit recognition method using a fractional-order back-propagation neural network in an embodiment of the present invention.

[0049] Figure 2 is the loss function value E of the fractional-order back propagation neural network in the embodiment of the present invention L2 The convergence diagram of .

[0050] Figure 3a 、 3b All are the training accuracy of the fractional-order back propagation neural network in the embodiment of the present invention. The convergence diagram of .

[0051] Figure 4 This is a schematic diagram of the visual interface for the TFO-BPNN neural network to recognize handwritten digits in an embodiment of the present invention.

[0052] Figure 5 This is a schematic diagram of the visual interface for the HTFO-BPNN neural network to recognize handwritten digits in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention will be further described below with reference to examples and drawings, but the embodiments of the present invention are not limited thereto.

[0054] Example 1

[0055] Based on a fractional-order back-propagation neural network with a regularization term, this method efficiently and accurately establishes a systematic relationship between handwritten digit images and their corresponding digits. Furthermore, a visualization interface is constructed that displays the handwritten digit image to be recognized and the corresponding output digit, providing a more intuitive display of the classification and recognition results, meeting the requirements for handwritten digit recognition.

[0056] like Figure 1 As shown, this embodiment provides a handwritten digit recognition method based on a fractional-order back-propagation neural network with a regularization term, comprising the following steps:

[0057] S1. Preprocess the images of the handwritten digit dataset.

[0058] In this embodiment, step S1 pre-processes the image, specifically including:

[0059] S11. This invention is based on a fractional-order back-propagation neural network with a regularization term. Before training the neural network, the input data needs to be preprocessed. The input data of the original handwritten digit dataset is a 28×28 rectangular image matrix, which is reordered row by row into a single-row, multi-column vector of 1×784.

[0060] S12. The loss function of this neural network is the cross entropy function. Therefore, the labels of the handwritten digit dataset are converted into one-hot vectors. That is, the label "0" is converted into (1,0,0,0,0,0,0,0,0,0), the label "1" is converted into (0,1,0,0,0,0,0,0,0,0,0), and so on.

[0061] S2. Construct a double cross entropy loss function with regularization term.

[0062] In this embodiment, step S2 specifically includes:

[0063] S21. Give the following definition of Caputo fractional derivative.

[0064] The Caputo derivative operator is denoted as The order is The Caputo fractional derivative is defined as:

[0065]

[0066] in, is a set of positive integers; is the objective function, yes The r-order derivative with respect to τ is [c,t] is a closed integration interval, Γ(·) is the Gamma function,

[0067] S22, respectively define and is an i-dimensional real number set and a j×l-dimensional real number set. Given J training samples (x j ,o j ),in is the jth input, is the jth ideal output, where j = 1, 2, ..., J; define a three-layer neural network, where the number of neurons in the input layer, hidden layer, and output layer are p, n, and q respectively; then, the output of the neural network is written as:

[0068] y=f(UG(Vx j ))(2)

[0069] where V is the weight matrix between the input layer and the hidden layer, and U is the weight matrix between the output layer and the hidden layer; G(·)=(g(·),g(·),...,g(·)) T , g(·) and f(·) are the activation functions of the hidden layer and the output layer respectively, where:

[0070]

[0071] is the element of the weight matrix U between the output layer and the hidden layer;

[0072] are the elements of the weight matrix V between the input layer and the hidden layer.

[0073] For simplicity, The loss function in a neural network is denoted as E(w).

[0074] S23, in the training sample data set {(x j ,o j )|j=1,2,...,J}, o j is x j The label, and o j is a one-hot vector; then, o jl =1 indicates that x is assigned to the lth class j The probability of is expected to be 1, and o jl =0, the opposite is true. is the actual output of the neural network model, where the lth element f jl Represents x j The actual probability that θ will be assigned to class l.

[0075] In order to adjust the penalty term (1-o jl )ln(1-f jl ) on the loss function E(w), this embodiment designs the following dual cross entropy loss function:

[0076]

[0077] where f jl =f(u l G(Vx j )),β is the penalty coefficient for misclassification, β≥0. For simplicity, define:

[0078] ζ jl =u l G(Vx j ) and F jl =F(ζ jl )=-o jl ln(f jl )-β(1-o jl )ln(1-f jl ), as β increases, the regularization term β(1-o jl )ln(1-f jl ) has a greater impact on the loss function E(w).

[0079] In addition, based on the loss function (3), a double cross entropy loss function with regularization term is given as:

[0080]

[0081] where L2(w) = λ / 2‖w‖ 2 ,‖w‖ represents the Euclidean norm of w, λ is the regularization parameter, λ ≥ 0; and the following definitions are made:

[0082]

[0083]

[0084]

[0085] This embodiment uses the double cross entropy loss function (4) with regularization term as the loss function of the fractional-order neural network, and its convergence diagram is as follows: Figure 2 shown.

[0086] S3. Construct a fractional-order back-propagation neural network model with regularization terms.

[0087] In this embodiment, step S3 specifically includes:

[0088] S31. Assumptions can be expanded into a Taylor series, and its Caputo fractional derivative can be expressed as follows:

[0089]

[0090] in:

[0091]

[0092]

[0093] Formula (5) is called the Taylor series expansion based on the Caputo derivative. The high-order terms of the expansion are truncated, while the first-order terms of the loss function E(w) are retained. Therefore, Formula (5) is rewritten as follows:

[0094]

[0095] in Denotes the truncated Caputo derivative operator. Here, since the squared penalty term L2(w) has a simple structure, the present invention simultaneously considers the classical Caputo derivative and the truncated Caputo derivative for the squared penalty term L2(w), corresponding to the hybrid truncated fractional neural network (HTFO-BPNN) and the truncated fractional neural network (TFO-BPNN), respectively.

[0096] S32. Design a truncated fractional-order neural network, specifically:

[0097] When considering the truncated Caputo derivative of the square penalty term L2(w), we have:

[0098]

[0099]

[0100] in:

[0101]

[0102]

[0103]

[0104] and and represents the truncated fractional differential operator, γ>0 is the learning rate; and w k is the initial weight w 0 The kth weight vector updated iteratively; κ is the number of generations in the current iteration. In this embodiment, a sufficiently small positive number ∈ is given to avoid the denominator being zero.

[0105] S33. Design a hybrid truncated fractional-order BP neural network, specifically:

[0106] When considering the classic Caputo derivative of the squared penalty term L2(w), we have:

[0107]

[0108]

[0109] in, and stands for mixed fractional differential operator.

[0110] S34. Obtain weight update rules for truncated fractional-order neural networks and hybrid truncated fractional-order neural networks.

[0111] For the truncated fractional-order neural network TFO-BPNN and the hybrid truncated fractional-order neural network HTFO-BPNN, when λ = 0, both cases become loss functions without the square penalty term L2(w).

[0112] Therefore, the weight update rules of TFO-BPNN and HTFO-BPNN are defined as:

[0113]

[0114]

[0115] in:

[0116]

[0117]

[0118] S4. According to the image preprocessed in step S1, relevant parameters of the fractional-order back-propagation neural network are set, and the fractional-order back-propagation neural network is trained to finally obtain a neural network model suitable for the handwritten digit dataset.

[0119] In this embodiment, step S4 specifically includes:

[0120] S41. According to the handwritten digit dataset, set the learning parameters, network structure (p, n, q) and initial weight w of the fractional-order neural network 0 , the learning parameters include the learning rate γ, the regularization coefficient λ, the penalty term coefficient β and the fractional order And set the number of mini-batch samples and the maximum number of iterations for the network training process.

[0121] S42, input the input data of the handwritten digital data set, and iteratively train the neural network weights according to the weight update rules of the truncated fractional order neural network and the hybrid truncated fractional order neural network shown in formula (11) and formula (12); stop training until the maximum number of iterations is reached, and save the neural network model obtained after training. Figure 3a 、 Figure 3b shown.

[0122] S5. In the neural network model obtained in step S4, the handwritten digit picture to be recognized is input and the handwritten digit picture is classified; and a visual interface is constructed to display the input handwritten digit picture and the digit output by the neural network model, such as Figure 4 、 Figure 5 shown.

[0123] In this embodiment, step S5 is specifically as follows: constructing a handwritten digit GUI visualization interface based on MatlabR2016a; secondly, inputting a handwritten digit image to be recognized, and selecting a network method for recognizing the image from TFO-BPNN and HTFO-BPNN; finally, the handwritten digit GUI visualization interface displays the recognized handwritten digit image and the recognition result.

[0124] Example 2

[0125] This embodiment provides a handwritten digit recognition system based on a fractional-order back-propagation neural network with a regularization term, which specifically includes the following modules:

[0126] A preprocessing module, used to preprocess the images of the handwritten digit dataset;

[0127] Loss function construction module, used to construct the double cross entropy loss function with regularization term;

[0128] Network model construction module, used to build a fractional-order back-propagation neural network model with regularization terms;

[0129] A training module is used to set relevant parameters of the fractional-order back-propagation neural network based on the preprocessed image, train the fractional-order back-propagation neural network, and ultimately obtain a neural network model suitable for the handwritten digit dataset;

[0130] The classification module is used to input the handwritten digit images to be recognized into the neural network model suitable for the handwritten digit dataset and classify the handwritten digit images.

[0131] The loss function construction module constructs a double cross entropy loss function with a regularization term, including:

[0132] Give the definition of Caputo fractional derivative;

[0133] Define separately and is an i-dimensional real number set and a j×l-dimensional real number set; given J training samples (x j ,o j ),in is the jth input, is the jth ideal output, j = 1, 2, ..., J; define a three-layer neural network, where the number of neurons in the input layer, hidden layer, and output layer are p, n, and q respectively; the output of the neural network is written as:

[0134] y=f(UG(Vx j ))

[0135] where V is the weight matrix between the input layer and the hidden layer, and U is the weight matrix between the output layer and the hidden layer; G(·)=(g(·),g(·),...,g(·)) T ,g(·) and f(·) are the activation functions of the hidden layer and the output layer respectively;

[0136] set up is the actual output of the neural network model, where the lth element f jl Represents x j The actual probability of being assigned to the lth class; the double cross entropy loss function is designed as:

[0137]

[0138] where f jl =f(u l G(Vx j )),β is the penalty coefficient for misclassification, β≥0; (1-o jl )ln(1-f jl ) is a penalty item;

[0139] The double cross entropy loss function with regularization term is:

[0140]

[0141] where L2(w) = λ / 2‖w‖ 2 ,‖w‖ represents the Euclidean norm of w, λ is the regularization parameter, λ≥0.

[0142] Furthermore, the network model construction module constructs a fractional-order back-propagation neural network model with a regularization term, including:

[0143] The Caputo fractional derivative is expressed as a Taylor series expansion based on the Caputo derivative, wherein the high-order terms of the Taylor series expansion are truncated, while the first-order terms of the loss function E(w) are retained;

[0144] Design truncated fractional-order neural networks;

[0145] Design a hybrid truncated fractional-order BP neural network;

[0146] The weight update rules of truncated fractional-order neural networks and hybrid truncated fractional-order neural networks are obtained.

[0147] The modules of this embodiment are respectively used to implement the steps of the handwritten digit recognition method of the fractional-order back-propagation neural network in Example 1. The detailed process is referred to Example 1 and will not be described here.

[0148] The above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications may be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the claims of the present invention.

Claims

1. A method for handwritten digit recognition using a fractional-order back-propagation neural network, characterized in that: The steps include: Preprocess the images of the handwritten digit dataset; Construct a double cross entropy loss function with regularization term; Construct a fractional-order back-propagation neural network model with regularization term; According to the preprocessed images, the relevant parameters of the fractional-order back-propagation neural network are set, and the fractional-order back-propagation neural network is trained to finally obtain a neural network model suitable for the handwritten digit dataset; In the neural network model suitable for the handwritten digit dataset, the handwritten digit image to be recognized is input and the handwritten digit image is classified.

2. The handwritten digit recognition method according to claim 1, characterized in that: The image preprocessing comprises: The input data of the original handwritten digit dataset is a rectangular image matrix, which is reordered row by row into a single-row and multi-column vector; Convert the labels of the handwritten digit dataset into one-hot vectors.

3. The handwritten digit recognition method according to claim 1, wherein: The construction of the double cross entropy loss function with a regularization term includes: Define separately and is an i-dimensional real number set and a j×l-dimensional real number set; given J training samples (x j ,o j ),in is the jth input, is the jth ideal output, j = 1, 2, ..., J; define a three-layer neural network, where the number of neurons in the input layer, hidden layer, and output layer are p, n, and q respectively; the output of the neural network is written as: y=f(UG(Vx j )) where V is the weight matrix between the input layer and the hidden layer, and U is the weight matrix between the output layer and the hidden layer; G(·) = (g(·), g(·), ..., g(·)) T , g(·) and f(·) are the activation functions of the hidden layer and the output layer respectively; where: is the element of the weight matrix U between the output layer and the hidden layer; is the element of the weight matrix V between the input layer and the hidden layer; make The loss function in the neural network is represented as E(w); set up is the actual output of the neural network model, where the lth element f jl Represents x j The actual probability of being assigned to the lth class; the double cross entropy loss function is designed as: where f jl =f(u l G(Vx j )),β is the penalty coefficient for misclassification, β≥0; (1-o jl )ln(1-f jl ) is the penalty term; definition: g jl =u l G(Vx j ) and F jl =F(ζ jl )=-o jl ln(f jl )-β(1-o jl )ln(1-f jl ); The double cross entropy loss function with regularization term is: where L2(w) = λ / 2‖w‖ 2 ,‖w‖ represents the Euclidean norm of w, λ is the regularization parameter, λ≥0.

4. The handwritten digit recognition method according to claim 3, characterized in that: Let the Caputo derivative operator of the Caputo fractional derivative be The order is The Caputo fractional derivative is defined as: in, is a set of positive integers; is the objective function, yes The r-order derivative with respect to τ is [c,t] is a closed integration interval, Γ(·) is the Gamma function, 5. The handwritten digit recognition method according to claim 3, wherein: The construction of the fractional-order back-propagation neural network model with a regularization term includes: The Caputo fractional derivative is expressed as a Taylor series expansion based on the Caputo derivative, wherein the high-order terms of the Taylor series expansion are truncated, while the first-order terms of the loss function E(w) are retained; Design truncated fractional-order neural networks; Design a hybrid truncated fractional-order BP neural network; The weight update rules of truncated fractional-order neural networks and hybrid truncated fractional-order neural networks are obtained.

6. The handwritten digit recognition method according to claim 5, characterized in that: The process of obtaining a neural network model suitable for handwritten digit datasets includes: According to the handwritten digit dataset, set the learning parameters, network structure (p, n, q) and initial weight w of the fractional-order neural network 0 , the learning parameters include the learning rate γ, the regularization coefficient λ, the penalty term coefficient β and the fractional order And set the number of mini-batch samples and the maximum number of iterations in the network training process; The input data of the handwritten digit dataset is input, and the neural network weights are iteratively trained according to the weight update rules of the truncated fractional-order neural network and the hybrid truncated fractional-order neural network.

7. The handwritten digit recognition method according to claim 1, wherein: The handwritten digit recognition method further comprises the steps of: Build a visualization interface to display the input handwritten digit images and the digits output by the neural network model.

8. A fractional-order back-propagation neural network handwritten digit recognition system, characterized in that: Includes the following modules: A preprocessing module, used to preprocess the images of the handwritten digit dataset; Loss function construction module, used to construct the double cross entropy loss function with regularization term; Network model construction module, used to build a fractional-order back-propagation neural network model with regularization terms; A training module is used to set relevant parameters of the fractional-order back-propagation neural network based on the preprocessed image, train the fractional-order back-propagation neural network, and ultimately obtain a neural network model suitable for the handwritten digit dataset; The classification module is used to input the handwritten digit images to be recognized into the neural network model suitable for the handwritten digit dataset and classify the handwritten digit images.

9. The handwritten digit recognition system according to claim 8, characterized in that: The loss function construction module constructs a double cross entropy loss function with a regularization term, including: Define separately and is an i-dimensional real number set and a j×l-dimensional real number set; given J training samples (x j ,o j ),in is the jth input, is the jth ideal output, j = 1, 2, ..., J; define a three-layer neural network, where the number of neurons in the input layer, hidden layer, and output layer are p, n, and q respectively; the output of the neural network is written as: y=f(UG(Vx j )) where V is the weight matrix between the input layer and the hidden layer, and U is the weight matrix between the output layer and the hidden layer; G(·) = (g(·), g(·), ..., g(·)) T , g(·) and f(·) are the activation functions of the hidden layer and the output layer respectively; where: is the element of the weight matrix U between the output layer and the hidden layer; is the element of the weight matrix V between the input layer and the hidden layer; make The loss function in the neural network is represented as E(w); set up is the actual output of the neural network model, where the lth element f jl Represents x j The actual probability of being assigned to the lth class; the double cross entropy loss function is designed as: where f jl =f(u l G(Vx j )),β is the penalty coefficient for misclassification, β≥0; (1-o jl )ln(1-f jl ) is the penalty term; definition: g jl =u l G(Vx j ) and F jl =F(ζ jl )=-o jl ln(f jl )-β(1-o jl )ln(1-f jl ); The double cross entropy loss function with regularization term is: where L2(w) = λ / 2‖w‖ 2 ,‖w‖ represents the Euclidean norm of w, λ is the regularization parameter, λ≥0.

10. The handwritten digit recognition system according to claim 8, characterized in that: The network model construction module constructs a fractional-order back-propagation neural network model with a regularization term, including: The Caputo fractional derivative is expressed as a Taylor series expansion based on the Caputo derivative, wherein the high-order terms of the Taylor series expansion are truncated, while the first-order terms of the loss function E(w) are retained; Design truncated fractional-order neural networks; Design a hybrid truncated fractional-order BP neural network; The weight update rules of truncated fractional-order neural networks and hybrid truncated fractional-order neural networks are obtained.

Images