A handwritten digit recognition system and method based on anti-interference convolutional neural network

Abstract

The present invention proposes a handwritten digit recognition system and method based on an anti-interference convolutional neural network. A handwritten text collection system obtains handwritten digit images and real digit labels written by different users. A generative adversarial network for interfering handwritten digit images is constructed, and the handwritten digit images are passed through the generative adversarial network for interfering handwritten digit images to predict the predicted labels of the noisy weakness feature images of the handwritten digit images. A loss function model is constructed in combination with the real digit labels, and the network is optimized using the RMSprop optimizer to obtain an optimized network. The handwritten digit images are collected by the handwritten text collection system and output to a host computer to obtain the predicted labels of the real-time handwritten digit images. The present invention utilizes a generative adversarial network to generate interference images using a special inverse mapping, and performs weakness training on the label convolutional neural network. This can recognize handwritten digit images with sloppy handwriting and stains, thereby improving the handwritten digit recognition accuracy of the handwritten digit recognition system.

Description

Technical Field

[0001] The present invention belongs to the field of information security, and in particular relates to a handwritten digit recognition system and method based on an anti-interference convolutional neural network. Background Art

[0002] In recent years, the technology used in text recognition has been developing rapidly and has been applied to many industries, greatly improving work efficiency for these industries. However, handwritten Chinese characters are random and irregular, and because the paper on which the handwritten Chinese characters are written is not necessarily very neat, it may easily cause label convolutional neural network recognition errors due to some accidents such as ink leakage, water droplets, or the paper itself is unclear, which greatly reduces the text recognition rate in slightly harsh environments. In order to solve or reduce these neural network recognition errors caused by harsh environments or human handwriting, a high-precision handwritten digital image neural network system and method based on generative adversarial network anti-interference is invented to solve this problem. We find a large number of such images in the data set, conduct targeted amplification of the neural network weaknesses, generate a large number of images or image backgrounds targeting the neural network weaknesses, and let the neural network perform targeted training to greatly improve the efficiency of the label convolutional neural network. The advantage of the present invention is that by training the label convolutional neural network on weaknesses, a higher-precision network can be obtained to recognize handwritten digital images with sloppy handwriting and stained images. Summary of the Invention

[0003] In order to solve the above technical problems, the present invention proposes a handwritten digit recognition system and method based on an anti-interference convolutional neural network.

[0004] The technical solution of the system of the present invention is a handwritten digit recognition system based on an anti-interference convolutional neural network, comprising:

[0005] Handwritten text collection system and host computer;

[0006] The handwritten text collection system is connected to the host computer;

[0007] The handwritten text collection system is used to collect a user's handwritten digital image and mark the handwritten digital image with a correct digital label according to the handwritten digital image.

[0008] The technical solution of the method of the present invention is a handwritten digit recognition method based on an anti-interference convolutional neural network, which is characterized by comprising the following steps:

[0009] Step 1: The handwritten text collection system obtains each handwritten digital image written by different users, as well as the true digital label of each handwritten digital image.

[0010] Step 2: Construct a generative adversarial network for interfering handwritten digit images. Pass each handwritten digit image through the interfering handwritten digit image generative adversarial network to predict the predicted labels of the noisy weak feature images of each handwritten digit image. Combined with the real digital labels of each handwritten digit image, the adversarial network loss function model is constructed. The optimized interfering handwritten digit image generative adversarial network is obtained through RMSprop iterative optimization training.

[0011] Step 3: Collect real-time handwritten digital images through the handwritten text collection system, and output the real-time handwritten digital images to the host computer. The host computer obtains the predicted label of the real-time handwritten digital image through the label convolutional neural network prediction of the optimized interference handwritten digital image generative adversarial network.

[0012] Each handwritten digital image in step 1 is defined as:

[0013] WD i ={wd i (x,y)|x∈[1,U],y∈[1,V]}

[0014] i∈[1,N]

[0015] Among them, WD i represents the i-th user handwritten digital image, wd i (x,y) represents the pixel at the xth row and yth column of the i-th handwritten digital image, U represents the number of rows of the i-th handwritten digital image, V represents the number of columns of the i-th handwritten digital image; N represents the number of handwritten digital images;

[0016] The true digital label of each handwritten digital image is defined as:

[0017] {Tlb i},Tlb i ∈[0,9]

[0018] Among them, Tlb i is the true digital label of the i-th handwritten digital image;

[0019] Preferably, the interference handwritten digital image generative adversarial network in step 2 is composed of a label convolutional neural network, an interference handwritten digital image generation module, a generator, a discriminator, a weakness feature extraction module, and a noise amplification module;

[0020] The label convolutional neural network, interference handwritten digital image generation module, generator, and discriminator are connected in sequence;

[0021] The generator, weakness feature extraction module, noise amplification module, and label convolutional neural network are connected in sequence;

[0022] Inputting each handwritten digital image into the interference handwritten digital image generation module;

[0023] The noise-added weakness feature image of each handwritten digit image is input into the label convolutional neural network, and the label convolutional neural network outputs the predicted label of the noise-added weakness feature image of each handwritten digit image;

[0024] Outputting the predicted labels of the noised weak point feature images of each handwritten digit image to the interference handwritten digit image generation module;

[0025] In the initial iterative optimization training of RMSprop, each handwritten digit image is used as the noised weakness feature image of each handwritten digit image;

[0026] The noise-added weakness feature image of each handwritten digital image is generated by the noise amplification module;

[0027] The interference handwritten digital image generation module selects Plb from multiple handwritten digital images. i ≠Tlb i The handwritten digital image is defined as an interference handwritten digital image, and an interference data set is constructed through multiple interference data images;

[0028]

[0029] in, For the kth j The predicted labels of the weak feature images after adding noise to a handwritten digit image;

[0030] For the kth j The ground-truth digital labels for handwritten digit images;

[0031] Each interference handwritten digit image is defined as follows:

[0032]

[0033] k j ∈[1,M]

[0034] in, is the jth interference handwritten digital image, i.e. the kth j A digital image of the user's handwriting, represents the pixel at row x and column y of the i-th interfering handwritten digit image, For the kth j The predicted labels of the weak feature images after adding noise to a handwritten digital image, is the true digital label corresponding to the jth interfering handwritten digital image, and M represents the number of interfering handwritten digital images;

[0035] The interference handwritten digital image generation module outputs each interference handwritten digital image to the generator;

[0036] The generator combines each interfering handwritten digital image, each interfering handwritten digital image's d-dimensional random vector Generate each generator handwritten digital image, and output each generator handwritten digital image to the discriminator and weakness feature extraction module respectively;

[0037] The discriminator outputs the probability that each interfering handwritten digital image is true, defined as

[0038] The probability that the discriminator outputs the generator's handwritten digital image is real is defined as

[0039] The weakness feature extraction module is composed of a feature extraction convolutional neural network;

[0040] The weakness feature extraction module extracts the features of the generator's handwritten digital image and outputs the kth j The predicted label of the noised weak point feature image of a handwritten digital image is The probability of Obtaining a weakness feature image of each interfering handwritten digital image, and outputting the weakness feature image of each interfering handwritten digital image to the noise amplification module;

[0041] The noise amplification module performs Laplace noise processing on the weakness feature image of each interfering handwritten digital image to obtain a noisy weakness feature image of each handwritten digital image, and outputs the noisy weakness feature image of each handwritten digital image to the label convolutional neural network;

[0042] The label convolutional neural network outputs the kth j The probability of the true digital label of the weak feature image after adding noise to a handwritten digital image is

[0043] Preferably, the adversarial network loss function model in step 2 is specifically defined as follows:

[0044] L all =αl cnn1 +βL g +loss z +loss cnn +L p

[0045]

[0046]

[0047]

[0048]

[0049]

[0050] Among them, L all is the total loss function, α is the weight of the feature extraction convolutional neural network, β is the weight of the generator, l cnn1 is the loss function of the feature extraction convolutional neural network, L p is the loss function of the discriminator, L g is the loss function of the generator, loss z is the random vector inverse operation loss function, loss cnn is the loss function of the label convolutional neural network, Output k for the discriminator j The probability that the interference handwritten digit image is true, Output generator k for the discriminator j The probability that a handwritten digit image is real, Output k j The predicted label of the noised weak point feature image of a handwritten digital image is The probability of For the kth j The qth value in the d-dimensional random vector of the interference handwritten digital image, Output kth label convolutional neural network j The probability of the true digital label of the weak feature image after adding noise to a handwritten digital image is

[0051] The advantage of the present invention is that it uses a generative adversarial network to use a special inverse mapping to generate interference images, instead of simply preprocessing the image, and trains the label convolutional neural network on weaknesses to obtain a higher-precision network, recognize handwritten digital images with sloppy handwriting and stains, and improve the accuracy of handwritten digital image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 : A method flow chart of a specific embodiment of the present invention.

[0053] Figure 2 : Basic framework diagram of the generative adversarial network according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] In specific implementation, the method proposed in the technical solution of the present invention can be automatically run by those skilled in the art using computer software technology. System devices that implement the method, such as computer-readable storage media that store the corresponding computer program of the technical solution of the present invention and computer equipment that runs the corresponding computer program, should also be within the scope of protection of the present invention.

[0056] The technical solution of the system of the embodiment of the present invention is: a generative adversarial network handwritten image digital label discrimination system, comprising:

[0057] Handwritten text collection system and host computer;

[0058] The handwritten text collection system is connected to the host computer;

[0059] The handwritten text collection system is used to collect a user's handwritten digital image and mark the handwritten digital image with a correct digital label according to the handwritten digital image.

[0060] The handwritten text recognition system is PP-OCRv3;

[0061] The host computer is selected as a server;

[0062] The following combination Figures 1 to 2 The technical solution of the method of the embodiment of the present invention is a generative adversarial network handwritten image digital label discrimination method, which is specifically as follows:

[0063] Step 1: The handwritten text collection system obtains each handwritten digital image written by different users, as well as the true digital label of each handwritten digital image.

[0064] Each handwritten digital image in step 1 is defined as:

[0065] WD i ={wd i (x,y)|x∈[1,U],y∈[1,V]}

[0066] i∈[1,N]

[0067] Among them, WD i represents the i-th user handwritten digital image, wd i(x, y) represents the pixel at the xth row and yth column of the i-th handwritten digital image, U = 28 represents the number of rows of the i-th handwritten digital image, V = 28 represents the number of columns of the i-th handwritten digital image; N = 20000 represents the number of handwritten digital images;

[0068] The true digital label of each handwritten digital image is defined as:

[0069] {Tlb i},Tlb i ∈[0,9]

[0070] Among them, Tlb i is the true digital label of the i-th handwritten digital image;

[0071] Step 2: Construct a generative adversarial network for interfering handwritten digit images. Pass each handwritten digit image through the interfering handwritten digit image generative adversarial network to predict the predicted labels of the noisy weak feature images of each handwritten digit image. Combined with the real digital labels of each handwritten digit image, the adversarial network loss function model is constructed. The optimized interfering handwritten digit image generative adversarial network is obtained through RMSprop iterative optimization training.

[0072] The interference handwritten digital image generative adversarial network described in step 2 is composed of a label convolutional neural network, an interference handwritten digital image generation module, a generator, a discriminator, a weakness feature extraction module, and a noise amplification module;

[0073] The label convolutional neural network, interference handwritten digital image generation module, generator, and discriminator are connected in sequence;

[0074] The generator, weakness feature extraction module, noise amplification module, and label convolutional neural network are connected in sequence;

[0075] Inputting each handwritten digital image into the interference handwritten digital image generation module;

[0076] The noise-added weakness feature image of each handwritten digit image is input into the label convolutional neural network, and the label convolutional neural network outputs the predicted label of the noise-added weakness feature image of each handwritten digit image;

[0077] Outputting the predicted labels of the noised weak point feature images of each handwritten digit image to the interference handwritten digit image generation module;

[0078] In the initial iterative optimization training of RMSprop, each handwritten digit image is used as the noised weakness feature image of each handwritten digit image;

[0079] The noise-added weakness feature image of each handwritten digital image is generated by the noise amplification module;

[0080] The interference handwritten digital image generation module selects Plb from multiple handwritten digital images. i ≠Tlb i The handwritten digital image is defined as an interference handwritten digital image, and an interference data set is constructed through multiple interference data images;

[0081]

[0082] in, For the kth j The predicted labels of the weak feature images after adding noise to a handwritten digit image;

[0083] For the kth j The ground-truth digital labels for handwritten digit images;

[0084] Each interference handwritten digit image is defined as follows:

[0085]

[0086] k j ∈[1,M]

[0087] in, is the jth interference handwritten digital image, i.e. the kth j A digital image of the user's handwriting, represents the pixel at row x and column y of the i-th interfering handwritten digit image, For the kth j The predicted labels of the weak feature images after adding noise to a handwritten digital image, is the true digital label corresponding to the jth interfering handwritten digital image, and M represents the number of interfering handwritten digital images;

[0088] The interference handwritten digital image generation module outputs each interference handwritten digital image to the generator;

[0089] The generator combines each interfering handwritten digital image, each interfering handwritten digital image's d=100-dimensional random vector Generate each generator handwritten digital image, and output each generator handwritten digital image to the discriminator and weakness feature extraction module respectively;

[0090] The discriminator outputs the probability that each interfering handwritten digital image is true, defined as

[0091] The probability that the discriminator outputs the generator's handwritten digital image is real is defined as

[0092] The weakness feature extraction module is composed of a feature extraction convolutional neural network;

[0093] The weakness feature extraction module extracts the features of the generator's handwritten digital image and outputs the kth j The predicted label of the noised weak point feature image of a handwritten digital image is The probability of Obtaining a weakness feature image of each interfering handwritten digital image, and outputting the weakness feature image of each interfering handwritten digital image to the noise amplification module;

[0094] The noise amplification module performs Laplace noise processing on the weakness feature image of each interfering handwritten digital image to obtain a noisy weakness feature image of each handwritten digital image, and outputs the noisy weakness feature image of each handwritten digital image to the label convolutional neural network;

[0095] The label convolutional neural network outputs the kth j The probability of the true digital label of the weak feature image after adding noise to a handwritten digital image is

[0096] The adversarial network loss function model described in step 2 is specifically defined as follows:

[0097] L all =αl cnn1 +βL g +loss z +loss cnn +L p

[0098]

[0099]

[0100]

[0101]

[0102]

[0103] Among them, L all is the total loss function, α is the weight of the feature extraction convolutional neural network, β is the weight of the generator, l cnn1 is the loss function of the feature extraction convolutional neural network, L p is the loss function of the discriminator, L g is the loss function of the generator, loss z is the random vector inverse operation loss function, loss cnn is the loss function of the label convolutional neural network, Output k for the discriminator j The probability that the interference handwritten digit image is true, Output generator k for the discriminator j The probability that a handwritten digit image is real, Output k j The predicted label of the noised weak point feature image of a handwritten digital image is The probability of For the kth j The qth value in the d-dimensional random vector of the interference handwritten digital image, Output kth label convolutional neural network j The probability of the true digital label of the weak feature image after adding noise to a handwritten digital image is

[0104] Step 3: Collect real-time handwritten digital images through the handwritten text collection system, and output the real-time handwritten digital images to the host computer. The host computer obtains the predicted label of the real-time handwritten digital image through the label convolutional neural network prediction of the optimized interference handwritten digital image generative adversarial network.

[0105] It should be understood that parts not elaborated in detail in this specification belong to the prior art.

[0106] Although the terms "handwriting recognition system" and "host computer" are frequently used herein, the possibility of using other terms is not excluded. These terms are used only to more conveniently describe the essence of the present invention. Interpreting them as any additional limitation is contrary to the spirit of the present invention.

[0107] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A handwritten digit recognition method based on an anti-interference convolutional neural network for a handwriting system, characterized in that: The handwriting system comprises: Handwritten text collection system and host computer; The handwritten text collection system is connected to the host computer; The handwritten text collection system is used to collect the user's handwritten digital image and mark the handwritten digital image with a correct digital label according to the handwritten digital image; The anti-interference convolutional neural network handwritten digit recognition method of the handwriting system comprises the following steps: Step 1: The handwritten text collection system obtains each handwritten digital image written by different users and the real digital label of each handwritten digital image; Step 2: Construct a generative adversarial network for interfering handwritten digit images. Pass each handwritten digit image through the interfering handwritten digit image generative adversarial network to predict the predicted labels of the noisy weak feature images of each handwritten digit image. Combined with the real digital labels of each handwritten digit image, the adversarial network loss function model is constructed. The optimized interfering handwritten digit image generative adversarial network is obtained through RMSprop iterative optimization training. Step 3: The real-time handwritten digital image is collected by the handwritten text collection system, and the real-time handwritten digital image is output to the host computer. The host computer predicts the label convolutional neural network of the optimized interference handwritten digital image generative adversarial network to obtain the predicted label of the real-time handwritten digital image; The interference handwritten digital image generative adversarial network described in step 2 is composed of a label convolutional neural network, an interference handwritten digital image generation module, a generator, a discriminator, a weakness feature extraction module, and a noise amplification module; The label convolutional neural network, interference handwritten digital image generation module, generator, and discriminator are connected in sequence; The generator, weakness feature extraction module, noise amplification module, and label convolutional neural network are connected in sequence; Inputting each handwritten digital image into the interference handwritten digital image generation module; The noise-added weakness feature image of each handwritten digit image is input into the label convolutional neural network, and the label convolutional neural network outputs the predicted label of the noise-added weakness feature image of each handwritten digit image; Outputting the predicted labels of the noised weak point feature images of each handwritten digit image to the interference handwritten digit image generation module; In the initial iterative optimization training of RMSprop, each handwritten digit image is used as the noised weakness feature image of each handwritten digit image; The noise-added weakness feature image of each handwritten digital image is generated by the noise amplification module; The interference handwritten digital image generation module selects Plb from multiple handwritten digital images. i ≠Tlb i The handwritten digital image is defined as an interference handwritten digital image, and an interference data set is constructed through multiple interference data images; in, For the kth j The predicted labels of the weak feature images after adding noise to a handwritten digit image; For the kth j The ground-truth digital labels for handwritten digit images; Each interference handwritten digit image is defined as follows: in, is the jth interference handwritten digital image, i.e. the kth j A digital image of the user's handwriting, represents the pixel at the xth row and yth column of the i-th interfering handwritten digital image, and M represents the number of interfering handwritten digital images; The interference handwritten digital image generation module outputs each interference handwritten digital image to the generator; The generator combines each interfering handwritten digital image, each interfering handwritten digital image's d-dimensional random vector Generate each generator handwritten digital image, and output each generator handwritten digital image to the discriminator and weakness feature extraction module respectively; The discriminator outputs the probability that each interfering handwritten digital image is true, defined as The probability that the discriminator outputs the generator's handwritten digital image is real is defined as The weakness feature extraction module is composed of a feature extraction convolutional neural network; The weakness feature extraction module extracts the features of the generator's handwritten digital image and outputs the kth j The predicted label of the noised weak point feature image of a handwritten digital image is The probability of Obtaining a weakness feature image of each interfering handwritten digital image, and outputting the weakness feature image of each interfering handwritten digital image to the noise amplification module; The noise amplification module performs Laplace noise processing on the weakness feature image of each interfering handwritten digital image to obtain a noisy weakness feature image of each interfering handwritten digital image, and outputs the noisy weakness feature image of each interfering handwritten digital image to the label convolutional neural network; The label convolutional neural network outputs the kth j The probability of the true digital label of the weak feature image after adding noise to a handwritten digital image is 2. The anti-interference convolutional neural network handwritten digit recognition method for a handwriting system according to claim 1, characterized in that: Each handwritten digital image in step 1 is defined as: WD i ={wd i (x,y)|x∈[1,U],y∈[1,V]} i∈[1,N] Among them, WD i represents the i-th user handwritten digital image, wd i (x,y) represents the pixel at the xth row and yth column of the i-th handwritten digital image, U represents the number of rows of the i-th handwritten digital image, V represents the number of columns of the i-th handwritten digital image; N represents the number of handwritten digital images; The true digital label of each handwritten digital image is defined as: <h2 style=";text-align:left;direction:ltr">{Tlb<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr">},Tlb<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> ∈[0,9] Among them, Tlb i is the true digital label of the i-th handwritten digital image.

3. The interference-resistant convolutional neural network handwritten digit recognition method for a handwriting system according to claim 2, characterized in that: The adversarial network loss function model described in step 2 is specifically defined as follows: L all =αl cnn1 +βL g +loss z +loss cnn +L p Among them, L all is the total loss function, α is the weight of the feature extraction convolutional neural network, β is the weight of the generator, l cnn1 is the loss function of the feature extraction convolutional neural network, L p is the loss function of the discriminator, L g is the loss function of the generator, loss z is the random vector inverse operation loss function, loss cnn is the loss function of the label convolutional neural network, Output k for the discriminator j The probability that the interference handwritten digit image is true, Output generator k for the discriminator j The probability that a handwritten digit image is real, Output k j The predicted label of the noised weak point feature image of a handwritten digital image is The probability of For the kth j The qth value in the d-dimensional random vector of the interference handwritten digital image, Output kth label convolutional neural network j The probability of the true digit label of the weak feature image after adding noise to a handwritten digit image.

Images