A digital recognition method based on local image features
By introducing multiple local feature extraction subnets and fully connected layers into the digital recognition network, the filter matrix is used to automatically reduce feature dimensions and extract image features, which solves the problems of low accuracy of digital recognition and cumbersome feature extraction in complex scenarios. It achieves high accuracy and wide application effects.
Patent Information
- Application Number
- CN202210442389.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-25
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-04-25
AI Technical Summary
The prior art has low accuracy in digital recognition in complex scenarios, cumbersome feature extraction process and large information loss, insufficient generalization ability of recognition models, making it difficult to adapt to applications in different scenarios.
Using a digital recognition method based on local features of the image, by building a digital recognition network model including multiple local feature extraction subnets, the filter matrix is used to automatically reduce the feature dimension and extract image features, combined with the full connection layer and the softmax output layer, and adopt the self-directed The adaptive moment estimation method is used to iteratively train to improve the recognition accuracy.
It effectively improves the accuracy of digital recognition, simplifies the process of reducing feature dimensions, enhances the ability to extract local feature of images, improves the generalization ability and application scenarios of the model, and reduces network training parameters.
Smart Images

Figure CN114842484B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image processing, and relates to an image digital recognition method, and in particular to a digital recognition method based on local features of an image, which can be used in the fields of house number recognition, handwritten digital recognition, express delivery collection and sorting, etc. Background Art
[0002] With the rapid development of electronic information, computer input has become more and more common, but handwritten or printed characters are still an irreplaceable way for people to transmit information. As a link between handwritten or printed numbers and computer input, digital recognition is widely used in finance, education, postal services and other fields, and its practicality is increasingly valued by people.
[0003] Traditional digital recognition is an issue of interest to the optical character recognition community. That is, optical technology is first used to convert the characters in paper documents into black and white dot matrix images, and then multiple conversions are performed through computer recognition software to become a format that humans can understand. At present, online recognition technology in digital recognition is relatively mature, while offline recognition technology needs to be further improved. In particular, recognizing numbers in real scenes is still a relatively difficult problem, and it is not as easy to distinguish between foreground and background as recognizing text. Environmental factors (such as shadows, noise and occlusion) and image acquisition factors (such as resolution, motion and focus blur) further complicate the problem of digital character recognition in street scene images, greatly reducing the accuracy of digital recognition.
[0004] The development of deep learning technology has provided new ideas for improving the above problems and has made breakthrough progress in the past few years. With the rapid growth of the amount of recognition data, the ability to process data and the requirements for recognition accuracy have become higher, and offline digital recognition deserves further study. For example, the patent application with application publication number CN112906829A and titled "A method and device for constructing a digital recognition model based on the Mnist data set" discloses a method for constructing a digital recognition model based on the Mnist data set. The method first trains the LeNet convolutional neural network model according to the Mnist data set, introduces a spatial pyramid pooling layer structure in the last convolution layer of the LeNet pre-trained model, and obtains a LeNet model with fixed output features by adaptively adjusting the pooling parameters. Each of its convolutional layers is introduced into batch normalization to obtain a LeNet model with output suitable for target task classification. The method attempts to solve the impact of images of different input sizes on recognition accuracy by referencing spatial pyramid pooling in the existing LeNet model. However, this method still has shortcomings: the image features and background used in this method are single, which is easy to cause distortion in image digital recognition in complex scenes, and the generalization of the recognition model in different scenes needs to be improved.
[0005] The patent application with application publication number CN110674822A and titled “A method for handwritten digit recognition based on feature dimensionality reduction” discloses a method for digit recognition based on feature dimensionality reduction, which performs grayscale and binarization preprocessing on the acquired handwritten digit image and extracts features, performs dimensionality reduction processing on the extracted features to obtain low-dimensional digital features, and then performs feature recognition processing through a convolutional neural network constructed with four convolution-pooling layers to obtain the recognition results of handwritten digits. This method mainly performs feature dimensionality reduction processing through a kernel principal component analysis model to reduce the feature dimension of handwritten digit features, thereby reducing redundant variables in the process of handwritten digit calculation and recognition, thereby reducing the difficulty and complexity of feature recognition, and improving the efficiency of handwritten digit recognition. However, the method has a cumbersome process for reducing the feature dimension, information loss will occur during the feature extraction process, the network model has many training parameters and few application scenarios for recognition. Summary of the invention
[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and propose a method for digital recognition based on local features of an image, aiming to improve the accuracy of digital recognition and reduce the parameters of network training.
[0007] To achieve the above object, the technical solution adopted by the present invention includes the following steps:
[0008] (1) Obtain training sample set and test sample set:
[0009] Get S RGB images containing A digital categories, preprocess each RGB image, then one-hot encode the digital categories in the preprocessed images, and use s1 RGB images and their corresponding labels as training sample sets, and use s2 RGB images and their corresponding labels as test sample sets, where 2≤A≤10, S≥90000, s1≥S / 2, s1+s2=S;
[0010] (2) Build a digital recognition network model H:
[0011] A digital recognition network model is constructed, which includes an input layer, K feature extraction networks, a fully connected layer and a softmax output layer connected in sequence; the feature extraction network includes a convolutional layer, a local feature extraction network and a 1×1 convolutional layer connected in sequence; the local feature extraction network is composed of a first local feature extraction subnetwork and a second local feature extraction subnetwork arranged in parallel in an alternating manner, the first local feature extraction subnetwork includes a minimum layer and a maximum layer connected in sequence, the second local feature extraction subnetwork includes a maximum layer and a minimum layer connected in sequence, and the number of the first local feature extraction subnetwork and the second local feature extraction subnetwork are both N, where K≥2 and N≥2;
[0012] (3) Iteratively train the digital recognition network model H:
[0013] (3a) The number of initialization iterations is t, the maximum number of iterations is T, T ≥ 100, and the current digit recognition network model is H t , and let t = 0, H = H t ;
[0014] (3b) Q training samples randomly selected from the training sample set are used as the current digit recognition network model H t The K feature extraction networks extract features from each training sample and obtain a feature map set F = {f1,f2,...,f q ,...,f Q}; The fully connected layer transforms each feature map f q The J pixel values of the training sample are mapped to A pixel values; the softmax output layer maps the A pixel values to the predicted probability that each training sample belongs to A categories through the softmax function, where f q Represents the feature map of size a×b×c corresponding to the qth training sample, q∈Q, Q≥64, J≥64;
[0015] (3c) Using the cross entropy loss function, the current digit recognition network H is calculated by the predicted probability of each training sample belonging to the category and the sample category label. t The loss value L t ; Adopt the adaptive moment estimation method Adam optimization algorithm, through the loss value L t For the digital recognition network H t The weights of the convolutional layer and the fully connected layer ω t and the deviation θ t , the maximum layer filter matrix W in the local feature extraction network D , minimum layer filter matrix W E Update to obtain the digital recognition network after this iteration;
[0016] (3d) Determine whether t≥T holds true. If so, obtain the trained digital recognition network model H'. Otherwise, set t=t+1 and execute step (3b);
[0017] (4) Obtain digital recognition results:
[0018] The test sample set is used as the input of the trained digit recognition network model H' to obtain the predicted probability that each test sample belongs to A categories, and the category with the largest probability is used as the digit recognition result of each test sample.
[0019] Compared with the prior art, the present invention has the following advantages:
[0020] (1) The local feature extraction network in the digital recognition network model constructed by the present invention includes multiple local feature extraction sub-networks. The local feature extraction sub-networks use a filter matrix to automatically reduce the image feature dimension, reduce the difficulty and complexity of feature recognition, and thus effectively extract image features. This overcomes the problem of the cumbersome process of reducing the feature dimension in the prior art, making the process of reducing the feature dimension simple in the present invention, and effectively improving the accuracy of digital recognition.
[0021] (2) In the process of extracting the features of each sample, the local feature extraction network of the present invention uses the filter matrix as a "probe" to collect image information to enhance the ability to extract local features of the image, thereby overcoming the problem of information loss caused by the use of convolutional layers and pooling layers in the prior art, and avoiding the impact of excessive loss of useful information on the ability to extract features, thereby effectively improving the accuracy of digital recognition.
[0022] (3) The digital recognition network constructed by the present invention uses a sample set with complex background information during training, which overcomes the problem of single image features and single background in the prior art. The model used can automatically process images in complex scenes, and the generalization ability of digital recognition is high, which makes the present invention have a wider range of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 It is a flow chart of the implementation of the present invention;
[0024] Figure 2 It is a structural schematic diagram of the digital identification network of the present invention. DETAILED DESCRIPTION
[0025] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0026] Reference Figure 1 , the present invention comprises the following steps:
[0027] Step 1) Obtain training sample set and test sample set:
[0028] Obtain S 32×32 RGB images containing A digital categories, preprocess each RGB image, and for ease of identification, change the original label 10 corresponding to the number 0 in the preprocessed sample set to 0, and perform one-hot encoding on all digital categories: convert the length of each label into a vector with a length equal to the number of categories, where the position of the vector is 0 except for the position of the category to which it belongs, and use s1 RGB images and their corresponding labels as training sample sets, and use s2 RGB images and their corresponding labels as test sample sets, where 2≤A≤10, S≥90000, s1≥S / 2, s1+s2=S; In this embodiment, 73257 images and their corresponding labels are selected from the 10 categories of the SVHN data set as training sample sets, and 26032 images and their corresponding labels are selected as test sample sets;
[0029] The specific idea of preprocessing each RGB image is as follows: first, grayscale each RGB image, and take the weighted average of its own and other pixel values in the neighborhood for each pixel in each grayscale image I(x, y) after grayscale processing, that is, effectively remove noise through Gaussian filtering; then, normalize the grayscale image U(x, y) after Gaussian filtering, and convert the integer value from 0 to 255 into a decimal value in the range of 0 to 1, and obtain the preprocessed image:
[0030] I(x,y)=0.3*R(x,y)+0.59*G(x,y)+0.11*B(x,y)
[0031]
[0032] Among them, R(x,y), G(x,y), B(x,y) are the R, G, and B channel components at the image point coordinates (x,y), and σ is the standard deviation;
[0033] Step 2) Build a digital recognition network model H:
[0034] A digital recognition network model is constructed, which includes an input layer, K feature extraction networks, a fully connected layer and a softmax output layer connected in sequence; the feature extraction network includes a convolutional layer, a local feature extraction network and a 1×1 convolutional layer connected in sequence; the local feature extraction network is composed of a first local feature extraction subnetwork and a second local feature extraction subnetwork arranged in parallel in an alternating manner, the first local feature extraction subnetwork includes a minimum layer and a maximum layer connected in sequence, so that the feature map is first operated through the minimum layer and then through the maximum layer, and the boundary in the feature map is smoothed through the filter matrix to eliminate small noise and reduce the feature dimension; the second local feature extraction subnetwork includes a maximum layer connected in sequence Large layer and minimum layer, realize that the feature map is first operated through the maximum layer and then through the minimum layer, fill the holes of the feature map through the filter matrix, connect the neighbors, and the two local feature extraction sub-networks can retain image details, greatly enhance the effect of image detail processing, and automatically reduce the feature dimension of the image, reduce the influence of noise and other unimportant features on feature extraction; the first local feature extraction sub-network and the second local feature extraction sub-network eliminate interference noise and enhance the ability to extract local features by alternately constructing alternating order filters in image processing. The number of the first local feature extraction sub-network and the second local feature extraction sub-network are both N, where K≥2 and N≥2;
[0035] Reference Figure 2 In this embodiment, the number of feature extraction networks K included in the digital recognition network model H is 2, the number of the first local feature extraction subnetwork and the number of the second local feature extraction subnetwork N is 4, and the parameters of each component of the digital recognition network model H are:
[0036] The convolution kernel size of the convolution layer in the first feature extraction network is 5×5, and the number is 8. The convolution kernel size of the convolution layer in the second feature extraction network is 3×3, and the number is 32. The convolution kernel step size of the convolution layer in all feature extraction networks is set to 1, and the valid filling method is used. The activation function is the relu function; the relu function formula is as follows:
[0037] v(x)=max(0,x)
[0038] Among them, x represents the output data after the input data is operated by the current layer of the network, and v(x) is the data after activation;
[0039] The number of filter matrices of the minimum layer and the maximum layer in the first local feature extraction subnetwork and the second local feature extraction subnetwork are both 8, the size is 3×3, the step size is set to 1, and the same filling method is used. Since the minimum layer and the maximum layer in the local feature extraction network are nonlinear operations, there is no need to select an activation function. The output of each parallel local feature extraction subnetwork is linearly connected to increase the diversity of features. The convolution kernel size of the 1×1 convolution layer is 1×1, the number is 32, the step size is set to 1, and the same filling method is used. The activation function is the relu function; in order to prevent overfitting, the number of nodes connected to the fully connected layer is 256, and random inactivation dropout is used to discard the pixel values of the fully connected layer with a probability of 0.5; the number of pixel values in the softmax output layer is set to 10, and the softmax function is used to obtain the normalized A categories. The softmax function formula is as follows:
[0040]
[0041] Among them, A represents the total number of digital categories, d is any one of the categories, is the index of the input for this category, g(x) d The softmax function uses exponential normalization to widen the distribution gap between input values that are close in distribution, making it less likely that the output category label will be misjudged.
[0042] Step 3) Train the digital recognition network model:
[0043] (3a) The number of initialization iterations is t, the maximum number of iterations is T, T ≥ 100, and the current digit recognition network model is H t , and let t = 0, H = H t ;
[0044] (3b) Q training samples randomly selected from the training sample set are used as the current digit recognition network model H t The input is used to speed up the update rate of weights, effectively reducing the network training time and the memory occupied by network operation; K feature extraction networks extract features from each training sample to obtain a feature map set F = {f1, f2, ..., f q ,...,f Q}; The fully connected layer transforms each feature map f q The J pixel values of the training sample are mapped to A pixel values; the softmax output layer maps the A pixel values to the predicted probability that each training sample belongs to A categories through the softmax function, where f qIndicates the feature map of size a×b×c corresponding to the qth training sample, q∈Q, Q≥64, J≥64; in this embodiment, Q is selected as 64 training samples as the current digital recognition network model H t The number of pixel values J in the fully connected layer is 256, and the number category A is 10;
[0045] In each feature extraction network, the operation ideas in the minimum layer and the maximum layer contained in the local feature extraction network are as follows: the minimum layer operation is to subtract the pixel value at the image point coordinate point from the value of the filter matrix, and then take the smallest result as the output result, that is, replace the multiplication in the convolution layer operation with subtraction, and replace the addition with the minimum pixel value; the maximum layer operation is to add the pixel value at the image point coordinate point and the value of the filter matrix, and then take the largest result as the output result, that is, replace the multiplication in the convolution layer operation with addition, and replace the addition with the maximum pixel value. The filter matrix can be regarded as the convolution kernel in the convolution layer operation;
[0046] (3c) Using the cross entropy loss function, the current digit recognition network H is calculated by the predicted probability of each training sample belonging to the category and the sample category label. t The loss value L t ; Adopt the adaptive moment estimation method Adam optimization algorithm, set the learning rate to 0.001, and pass the loss value L t For the digital recognition network H t The weights ω of the convolutional layer and the fully connected layer t and the deviation θ t , the maximum layer filter matrix W in the local feature extraction network D , minimum layer filter matrix W E Update to obtain the digital recognition network after this iteration;
[0047] Current Digital Recognition Network H t The loss value L t , the calculation formula is:
[0048]
[0049] Where A represents the total number of digital categories, p'(d) represents the sample category label, and p(d) represents the predicted probability that each training sample belongs to the category;
[0050] (3d) Determine whether t≥T holds true. If so, obtain the trained digital recognition network model H'. Otherwise, set t=t+1 and execute step (3b);
[0051] (4) Obtain digital recognition results:
[0052] The test sample set is used as the input of the trained digit recognition network model H' to obtain the predicted probability that each test sample belongs to A categories, and the category with the largest probability is used as the digit recognition result of each test sample. In this embodiment, the preprocessed 26032 test sample sets are input into the trained digit recognition network model H', and the extracted digit features are input into the softmax output layer through the feature extraction network through the fully connected layer to obtain the label probability of the digit, and the category with the largest probability is used as the digit recognition result.
[0053] The technical effects of the present invention are described below in conjunction with simulation experiments.
[0054] 1. Simulation experiment conditions:
[0055] The hardware test platform for the simulation experiment is: CPU is Intel Core i7-9700, main frequency is 3.00GHz, memory is 8GB, GPU is NVIDIA GeForce GTX 1050Ti; software platform is: Windows 10 64-bit operating system, python3.7.
[0056] The simulation experiment selected a digital SVHN data set of street scenes in complex natural scenes. The training set includes 73,257 numbers, the test set includes 26,032 numbers, and there are 531,131 additional numbers, all of which are 32×32 color images. The present invention trains the 73,257 training sample sets in the SVHN data set through a network, uses the trained network to test and output the 26,032 test sample sets, and obtains the digital recognition accuracy and the training network parameters. The results are shown in Table 1.
[0057] 2. Analysis of simulation results:
[0058] Table 1
[0059] Digital recognition accuracy Number of network parameters The present invention 98.5% 263114
[0060] The simulation results show that after simple image preprocessing, the accuracy of digital recognition is improved to 98.5% through network training. The network training parameters are only 263,114, which greatly reduces the computational cost after an order of magnitude reduction, automatically reduces the feature dimension, enhances the network's ability to extract local features, and effectively improves the accuracy of digital recognition. This is the result obtained on the street view data set in complex natural scenes used by the present invention, so the network model of the present invention can also be applied to digital recognition in other scenes, such as handwritten digital recognition tasks.
Claims
1. A digital recognition method based on local image features, characterized in that: The steps include: (1) Obtain training sample set and test sample set: Obtain S RGB images containing A digital categories, preprocess each RGB image, and then one-hot encode the digital categories in the preprocessed images. Use s1 RGB images and their corresponding labels as the training sample set, and use s2 RGB images and their corresponding labels as the test sample set, where 2≤A≤10, S≥90000, s1≥S / 2, and s1+s2=S; (2) Build a digital recognition network model H: Build a digital recognition network model including an input layer, K feature extraction networks, a fully connected layer and a softmax output layer connected in sequence; the feature extraction network includes a convolutional layer, a local feature extraction network and a 1×1 convolutional layer connected in sequence; the local feature extraction network is composed of a first local feature extraction subnetwork and a second local feature extraction subnetwork arranged in parallel, the first local feature extraction subnetwork includes a minimum layer and a maximum layer connected in sequence, the second local feature extraction subnetwork includes a maximum layer and a minimum layer connected in sequence, the number of the first local feature extraction subnetwork and the second local feature extraction subnetwork are both N, wherein the minimum layer operation is to replace the multiplication in the convolutional layer operation with subtraction, and the subtraction is replaced by taking the minimum pixel value; the maximum layer operation is to replace the multiplication in the convolutional layer operation with addition, and the addition is replaced by taking the maximum pixel value, K ≥ 2, N ≥ 2; (3) Iteratively train the digital recognition network model H: (3a) The number of initial iterations is t, the maximum number of iterations is T, T ≥ 100, and the current digit recognition network model is H t , and let t = 0, H = H t ; (3b) Q training samples randomly selected from the training sample set are used as the current digit recognition network model H t The K feature extraction networks extract features from each training sample and obtain the feature map set F = {f1,f2,...,fq,...,f Q }; The fully connected layer transforms each feature map f q The J pixel values of the training set are mapped to A pixel values; the softmax output layer maps the A pixel values to the predicted probability that each training sample belongs to A categories through the softmax function, where f q Represents the feature map of size a×b×c corresponding to the qth training sample, q∈Q, Q≥64, J≥64; (3c) Using the cross entropy loss function, the current digit recognition network H is calculated by the predicted probability of each training sample belonging to the category and the sample category label. t The loss value L t ; Adopting the adaptive moment estimation method Adam optimization algorithm, through the loss value L t For digital recognition network H t The weights ω of the convolutional layer and the fully connected layer t and the deviation θ t , the maximum layer filter matrix W in the local feature extraction network D , minimum layer filter matrix W E Update and obtain the digital recognition network after this iteration; (3d) Determine whether t≥T holds true. If so, obtain the trained digit recognition network model H'. Otherwise, set t=t+1 and execute step (3b); (4) Obtain digital recognition results: The test sample set is used as the input of the trained digit recognition network H' to obtain the predicted probability that each test sample belongs to A categories, and the category with the largest probability is used as the digit recognition result of each test sample.
2. The digital recognition method based on local image features according to claim 1, characterized in that: The preprocessing of each RGB image described in step (1) is specifically as follows: grayscale processing is performed on each RGB image, and each grayscale image I(x, y) of the grayscale processing is Gaussian filtered, and then the grayscale image U(x, y) after Gaussian filtering is normalized to obtain the preprocessed image: I(x,y)=0.3*R(x,y)+0.59*G(x,y)+0.11*B(x,y) Where R(x,y), G(x,y), and B(x,y) are the RGB channel components at the image point coordinate (x,y), and σ is the standard deviation.
3. The digital recognition method based on local image features according to claim 1, characterized in that: The digital recognition network model H described in step (2) includes the following: the number of feature extraction networks K=2, the number of the first local feature extraction subnetwork and the number of the second local feature extraction subnetwork N=4, and the parameters of the components of the digital recognition network model H are: The convolution kernel size of the convolution layer in the first feature extraction network is 5×5 and the number is 8. The convolution kernel size of the convolution layer in the second feature extraction network is 3×3 and the number is 32. Among them, the convolution kernel step size of the convolution layer in all feature extraction networks is set to 1, the valid padding method is used, and the activation function is the relu function; the number of filter matrices of the minimum layer and the maximum layer in the first local feature extraction subnetwork and the second local feature extraction subnetwork is 8, the size is 3×3, the step size is set to 1, and the same padding method is used; the convolution kernel size of the 1×1 convolution layer is 1× 1, the number is 32, the activation function is the relu function, the step size is set to 1, and the same padding method is used; the convolution kernel size of the 1×1 convolution layer is 1×1, the number is 32, the activation function is the relu function, the step size is set to 1, and the same padding method is used; the number of nodes connected to the fully connected layer is 256, and random inactivation dropout is used to discard the pixel values of the fully connected layer with a probability of 0.5; the number of pixel values in the softmax output layer is set to 10, and the softmax function is used to obtain the normalized A categories.
4. The digital recognition method based on local image features according to claim 1, characterized in that: The calculation of the current digit recognition network H described in step (3c) t The loss value L t , the calculation formula is: Where A represents the total number of digital categories, p'(d) represents the sample category label, and p(d) represents the predicted probability of each training sample belonging to the category.
