A method for recognizing mathematical expressions of graphic verification codes

Abstract

The invention relates to a method for recognizing mathematical expressions of graphic verification codes, comprising the following steps: S1, preparing sample images to be trained, extracting feature data of the sample images, wherein the HOG feature of the image, i.e., histogram of directional gradients, is used as feature data; S2, training the feature data based on an SVM classifier to generate a model file; S3, performing sample recognition based on the model file, wherein training and recognition are performed based on an SVM support vector machine of OPENCV.

Description

Technical Field

[0001] The invention belongs to the technical field of image recognition, and in particular relates to a method for recognizing mathematical expressions of graphic verification codes. Background Art

[0002] In the development process of many projects, it is necessary to access a third-party platform to complete data interaction. When the program performs automatic login operations, it needs to be able to automatically identify the text and arithmetic symbols in the graphic verification code and automatically calculate the results to facilitate the next step of business.

[0003] Traditional open source graphic text recognition, due to differences in sample images, results in models and algorithms that are unable to complete graphic verification code recognition, or the recognition rate is too low to meet program usage requirements. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method for recognizing mathematical expressions of graphic verification codes.

[0005] The present invention is achieved through the following technical solutions:

[0006] A method for recognizing mathematical expressions of graphic verification codes, characterized by comprising the steps of:

[0007] S1. Prepare a sample image to be trained and extract feature data of the sample image, wherein the HOG feature of the image, i.e., histogram of directional gradients, is used as feature data;

[0008] S2, training the feature data based on the SVM classifier to generate a model file;

[0009] S3, performing sample identification based on the model file;

[0010] Among them, the SVM support vector machine based on OPENCV is used for training and recognition.

[0011] Furthermore, the sample images include numbers 0-9, four operators of addition, subtraction, multiplication and division, a total of 14 types, 15 images of each type, a total of 210 images; among which, the identification labels of the 0-9 types are set to 0-9, and the identification labels of addition, subtraction, multiplication and division are set to 10, 11, 12, 13 respectively.

[0012] Furthermore, in the step S1, the feature data extraction includes the steps of:

[0013] 1.1. Convert the sample image into a grayscale image a;

[0014] 1.2. Perform denoising operation on a and output image b;

[0015] 1.3. Stretch image b to 128*128 size;

[0016] 1.4. Create a HOG descriptor, calculate the feature points of the stretched image b, and store the results in a one-dimensional array descriptors;

[0017] 2.5. Traverse the array descriptors and write the data into the corresponding Train_Mat matrix.

[0018] Furthermore, the Train_Mat matrix is ​​a two-dimensional matrix, each column of the Train_Mat matrix represents the HOG feature of an image, and each row is a sample image to be trained.

[0019] Furthermore, the identification label represented by each image is stored in the label dataset Labels_Mat.

[0020] Furthermore, in step S2, after the feature data extraction is completed, the Train_Mat matrix and Labels_Mat are put into SVM classifier training to generate a model file.

[0021] Furthermore, in step S3, the sample identification based on the model file includes the following steps:

[0022] 3.1. Load the sample image a, remove the black border in the original image a, and output image b.

[0023] 3.2. Remove the background noise from image b, adjust the color levels of b, set its maximum value to 100 and the minimum value to 50, and output image c.

[0024] 3.3. Extract the contour of c, filter out the valid contour containing the complete text, and obtain the contour rectangle image d;

[0025] 3.4. Traverse all contours, take their minimum circumscribed rectangle r, and determine whether the width of r exceeds the set threshold. If it is greater than the threshold, it means that r contains two or more characters. You need to manually split r into two connected rectangles r1 and r2, and store r1 and r2 in linked list e in turn; if it is less than the threshold, directly store r in linked list e;

[0026] 3.5. Traverse the linked list e, cut out the rectangular area images g in the linked list e from the image c one by one, perform sample recognition, and return the recognition result f;

[0027] 3.6. f>=0 indicates that it is recognized as valid data. When f>=0 and f<=9, it indicates that the result is a number; when f>=10 and f<=13, it indicates that the result is an operator.

[0028] 3.7. Store the recognition result in a string format. The result before the operator is returned is the left value of the arithmetic expression, and the result after the operator is recognized is the right value of the arithmetic expression.

[0029] 3.8. After the traversal is completed, determine whether the operator is recognized, calculate the mathematical expression based on the operator, the value on the left side of the expression, and the value on the right side of the expression, and finally output the calculation result.

[0030] Furthermore, in the step 3.3, the contour extraction includes: traversing the contours of c, obtaining the minimum circumscribed rectangle r of each contour, determining whether the width of r is consistent with the width of image c, and if so, deleting the contour; and determining whether r intersects with the minimum circumscribed rectangle of other contours, and if so, indicating that the contour is an internal contour of the text, it is regarded as an invalid text contour and deleted.

[0031] Furthermore, in step 3.5, the sample identification includes the steps of:

[0032] 3.5.1. Stretch the image g to 128*128 size;

[0033] 3.5.2. Create a HOG descriptor, calculate the feature points of image b, and store the results in a one-dimensional array descriptors.

[0034] 3.5.3. Create a sample test two-dimensional matrix Test_mat, the row of which is 1 and the column is the length of the array descriptors. Store the array descriptors into Test_mat in sequence.

[0035] 3.5.4. Create an SVM classifier and load the trained model file.

[0036] 3.5.5. Use the SVM classifier to verify Test_mat and return the recognition result f.

[0037] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a method for recognizing a mathematical expression of a graphic verification code are implemented.

[0038] Compared with the prior art, the present invention has the following advantages: the algorithm adopts SVM support vector machine based on OPENCV for training and recognition, and adopts HOG feature of the image, i.e. directional gradient histogram, as feature data, and can reliably complete the recognition of graphic verification code with high recognition efficiency, meeting the requirements of program use. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will be further described in detail below with reference to the accompanying drawings;

[0040] Figure 1 This is the recognition effect diagram of the present invention. The recognition results of Figure (g) are: 8, 10, 1, 5. Among them, 10 is represented by the "+" sign in the operator, and the final recognition result is: 8+15, and 23 is returned. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In a specific embodiment, the technology of automatically recognizing characters by computer is an important field of pattern recognition application. The recognition process includes steps such as sample image feature extraction, feature data training, model file generation, and test sample. This algorithm uses SVM support vector machine based on OPENCV for training and recognition, and uses the HOG feature of the image, i.e., directional gradient histogram, as feature data. The directional gradient histogram feature is a feature descriptor used for object detection in computer vision and image processing. It constitutes features by calculating and counting local image areas and gradient histograms. The present invention includes model training and sample recognition.

[0043] 1. Model training, including steps:

[0044] 1. Prepare the images to be trained, including the numbers 0 to 9, four operators: addition, subtraction, multiplication, and division, a total of 14 types, 15 images of each type, a total of 210 images.

[0045] 2. Extract the hog features of each image respectively, and write its feature data into the two-dimensional matrix Train_Mat. Each column of Train_Mat represents the hog features of an image, and each row is an image to be trained. At the same time, the recognition content represented by each image (the image labels of type 0-9 are 0-9, and the image labels of addition, subtraction, multiplication, and division are set to 10, 11, 12, and 13 respectively) is stored in the label data set Labels_Mat.

[0046] The feature extraction step includes:

[0047] 2.1. Convert the image into a grayscale image a.

[0048] 2.2. Perform denoising operation on image a and output image b.

[0049] 2.3. Stretch b to 128*128 size.

[0050] 2.4. Create a HOG descriptor, calculate the feature points of image b, and store the results in a one-dimensional array descriptors.

[0051] 2.5. Traverse the descriptors array and write the data into the corresponding Train_Mat matrix.

[0052] 3. Create an SVM classification trainer, set its type to classification, and use the linear kernel function.

[0053] 4. After data extraction is completed, Train_Mat and Labels_Mat are put into SVM classifier training to generate model files.

[0054] 2. Sample testing, including steps:

[0055] 1. Load the original image a, remove the black border in the original image, and output image b.

[0056] 2. Remove the background noise of image b, adjust the color levels of b, set its maximum value to 100 and the minimum value to 50, and output image c.

[0057] 3. Extract the contour of c and obtain the contour rectangle image d. Filter the valid contours that contain the complete text. Traverse the contours and obtain the minimum enclosing rectangle r of each contour. Determine whether the width of r is consistent with the image width. If so, delete the contour. At the same time, determine whether r intersects with the minimum enclosing rectangle of other contours. If so, it means that the contour is the internal contour of the text and is considered an invalid text contour and deleted.

[0058] 4. Traverse all contours and take their minimum circumscribed rectangle r. Determine whether the width of r exceeds the set threshold. When it is greater than the threshold, it means that r contains two or more characters. You need to manually split r into two connected rectangles r1 and r2, and store r1 and r2 in linked list e in turn. When it is less than the threshold, store r directly in linked list e. Since the height of the image contour r of the "-" sign in the operator is too small, and the image contour of the operator "÷" is composed of multiple small contours, its minimum circumscribed rectangle cannot represent the complete operator. Therefore, when the contour is obtained, it is necessary to forcibly expand the height of r to the height of the minimum circumscribed rectangle of the first character contour to ensure that the rectangle contains the complete operator image.

[0059] 5. Traverse the linked list e, intercept the image g of the rectangular area in the linked list e from the image c one by one, perform sample recognition, and return the recognition result f.

[0060] The sample identification steps are as follows:

[0061] 5.1. Stretch g to 128*128 size.

[0062] 5.2. Create a HOG descriptor, calculate the feature points of image b, and store the results in a one-dimensional array descriptors.

[0063] 5.3. Create a sample test two-dimensional matrix Test_mat, the matrix row is 1, the column is the length of descriptors, and store the descriptors data into Test_mat in sequence.

[0064] 5.4. Create an SVM classifier and load the trained model file.

[0065] 5.5. Use the SVM classifier to verify Test_mat and return the recognition result f.

[0066] 6. f>=0 indicates that it is recognized as valid data. When f>=0 and f<=9, it indicates that the result is a number. When f>=10 and f<=13, it indicates that the result is an arithmetic operator.

[0067] 7. The recognition result is stored in the form of a string. The result before the operator is returned is the left value of the arithmetic expression, and the result after the operator is recognized is the right value of the arithmetic expression.

[0068] 8. After the traversal is completed, determine whether the operator is recognized, calculate the mathematical expression based on the operator, the value on the left side of the expression, and the value on the right side of the expression, and finally return the calculation result.

[0069] The present invention also provides a computer-readable storage medium on which a computer program is stored, wherein when the program is executed by a processor, the steps of the method for recognizing a mathematical expression of a graphic verification code are implemented.

[0070] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for recognizing mathematical expressions of graphic verification codes when executing the program.

[0071] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention also fall within the scope of protection of the present invention.

Claims

1. A method for recognizing mathematical expressions of graphic verification codes, characterized in that: Includes steps: S1. Prepare a sample image to be trained and extract feature data of the sample image, wherein the HOG feature of the image, i.e., histogram of directional gradients, is used as feature data; S2, training the feature data based on the SVM classifier to generate a model file; S3, performing sample identification based on the model file; Among them, the SVM support vector machine based on OPENCV is used for training and recognition; In the step S3, the sample identification based on the model file includes the following steps: 3.

1. Load the sample image a, remove the black border in the original image a, and output image b; 3.

2. Remove the background noise of image b, adjust the color level of b, set its maximum value to 100 and the minimum value to 50, and output image c; 3.

3. Extract the contour of c, filter out the valid contour containing the complete text, and obtain the contour rectangle image d; 3.

4. Traverse all contours, take their minimum circumscribed rectangle r, and determine whether the width of r exceeds the set threshold. When it is greater than the threshold, it means that r contains two or more characters, and it is necessary to manually split r into two connected rectangles r1 and r2, and store r1 and r2 in the linked list e in turn; when it is less than the threshold, directly store r in the linked list e; Among them, when the contour is obtained, it is necessary to forcibly expand the height of r to the height of the minimum circumscribed rectangle of the first character contour to ensure that the rectangle contains the complete operator image; 3.

5. Traverse the linked list e, cut out the rectangular area images g in the linked list e from the image c one by one, perform sample recognition, and return the recognition result f; 3.6, f >= 0 indicates that it is recognized as valid data. When f>=0 and f<=9, it indicates that the result is a number; when f>=10 and f<= 13, it indicates that the result is an operator; 3.

7. Store the recognition result in a string format. The result before the operator is returned is the left value of the arithmetic expression, and the result after the operator is recognized is the right value of the arithmetic expression. 3.

8. After the traversal is completed, determine whether the operator is recognized, calculate the mathematical expression based on the operator, the value on the left side of the expression, and the value on the right side of the expression, and finally output the calculation result; The sample images include numbers 0-9, four operators of addition, subtraction, multiplication and division, a total of 14 types, 15 images of each type, a total of 210 images; among them, the identification labels of 0-9 types are set to 0-9, and the identification labels of addition, subtraction, multiplication and division are set to 10, 11, 12, 13 respectively.

2. The method for recognizing mathematical expressions of graphic verification codes according to claim 1, characterized in that: In step S1, the feature data extraction includes the steps of: 1.

1. Convert the sample image into a grayscale image a; 1.

2. Perform denoising operation on a and output image b; 1.

3. Stretch image b to 128×128 size; 1.

4. Create a HOG descriptor, calculate the feature points of the stretched image b, and store the results in a one-dimensional array descriptors; 2.

5. Traverse the array descriptors and write the data into the corresponding Train_Mat matrix.

3. The method for recognizing a mathematical expression of a graphical verification code according to claim 2, characterized in that: The Train_Mat matrix is ​​a two-dimensional matrix. Each column of the Train_Mat matrix represents the HOG feature of an image, and each row is a sample image to be trained.

4. The method for recognizing a mathematical expression of a graphic verification code according to claim 3, characterized in that: The identification label represented by each image is stored in the label dataset Labels_Mat.

5. The method for recognizing a mathematical expression of a graphical verification code according to claim 4, characterized in that: In step S2, after the feature data extraction is completed, the Train_Mat matrix and Labels_Mat are put into SVM classifier training to generate a model file.

6. The method for recognizing mathematical expressions of graphic verification codes according to claim 1, characterized in that: In step 3.3, the contour extraction includes: traversing the contour of c, obtaining the minimum circumscribed rectangle r of each contour, judging whether the width of r is consistent with the width of image c, if so, deleting the contour; and judging whether r intersects with the minimum circumscribed rectangle of other contours, if so, indicating that the contour is the internal contour of the text, it is regarded as an invalid text contour and deleted.

7. The method for recognizing mathematical expressions of graphic verification codes according to claim 1, characterized in that: In step 3.5, the sample identification includes the steps of: 3.5.

1. Stretch the image g to 128×128 size; 3.5.

2. Create HOG descriptor, calculate feature points of image b, and store the results in a one-dimensional array descriptors; 3.5.3、Create a sample test two-dimensional matrix Test_mat, the matrix row is 1, the column is the length of the array descriptors, and store the array descriptors into Test_mat in sequence; 3.5.

4. Create an SVM classifier and load the trained model file; 3.5.

5. Use the SVM classifier to verify Test_mat and return the recognition result f.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for recognizing mathematical expressions of graphic verification codes according to any one of claims 1 to 7 are implemented.

Images