A method for recognizing slanted Chinese character click verification codes

Abstract

The present invention discloses a method for recognizing inclined Chinese character click verification codes. The YOLOX target detection model is used to predict the text position box, and the text position box is used as the input parameter for foreground segmentation to segment the Chinese character foreground. The segmented foreground image is binarized, and the minimum circumscribed rectangle is drawn. The slope and angle of the circumscribed rectangle are calculated, and three preferred correction angles are given. The original image is rotated to obtain three corrected images with different angles. The corrected images are input into the text recognition model, and the recognition probabilities within the candidate Chinese character range are compared. The corrected image corresponding to the maximum probability is the final corrected result. The best matching result of the text position box and the candidate Chinese characters is obtained by circularly searching for the maximum value in the finally generated probability matrix and deleting the corresponding rows and columns. The present invention corrects the inclined Chinese characters in the verification code and improves the recognition accuracy of the Chinese character click verification code.

Description

Technical Field

[0001] This application belongs to the technical field of verification code recognition. More specifically, it relates to a method for recognizing slanted Chinese character click verification codes. Background Art

[0002] Verification codes are an important means for websites to prevent malicious attacks. There are various forms of verification codes, and the Chinese character click verification code is an excellent design among them. Compared with traditional input character verification codes, the Chinese character click verification code has high security and is very user-friendly.

[0003] While effectively preventing malicious attacks, the Chinese character click verification code also has a great impact on non-malicious automated programs, forcing the separation of originally automated processes and reducing work efficiency. At the same time, in order to evaluate the security of the Chinese character click verification code, it is often necessary to design corresponding verification code recognition methods for countermeasure testing.

[0004] Existing click verification code recognition technologies are only applicable to regular non-slanted texts and have not proposed an effective solution for recognizing slanted Chinese character click verification codes.

[0005] Therefore, a method for recognizing slanted Chinese character click verification codes is proposed to improve the recognition accuracy of Chinese character click verification codes and provide a reference solution for verification code recognition testing and non-malicious automated programs. Summary of the Invention

[0006] The purpose of this application is to address the above issues by proposing a method for recognizing slanted Chinese character click verification codes. This method can effectively correct the slant of Chinese characters within a certain angle range and make the best use of the known conditions in the verification code title to effectively improve the accuracy of text recognition.

[0007] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0008] A method for recognizing slanted Chinese character click verification codes includes:

[0009] Obtain a verification code image and perform object detection to obtain the position rectangular frames of all Chinese characters in the image;

[0010] Use the obtained rectangular frames as the basis for adaptive foreground and background segmentation to perform foreground and background segmentation. Add the foregrounds containing Chinese characters obtained from the foreground and background segmentation to obtain a verification code image containing only the foreground of Chinese characters;

[0011] Crop the verification code image containing only the foreground of Chinese characters according to the rectangular frame positions, crop out the sub-images corresponding to each Chinese character, convert the sub-images into binary images, detect the contours of the binary images, and obtain the corresponding minimum circumscribed rectangular frames;

[0012] For the circumscribed rectangle of each sub - figure, calculate the slopes of two adjacent sides, convert them into inclination angles, and correct the angles of the sub - figures according to the obtained inclination angles respectively. Use the original sub - figure and the two corrected sub - figures as the sub - figures to be recognized.

[0013] For the three sub - figures to be recognized of each Chinese character, use the text recognition model respectively to obtain the probability that each sub - figure to be recognized is recognized as each candidate Chinese character. Select the sub - figure to be recognized corresponding to the maximum probability among them. The sub - figures to be recognized selected for all Chinese characters in the verification code picture and the probabilities that they are recognized as each candidate Chinese character form a probability matrix.

[0014] In the probability matrix, use the method of repeatedly finding the maximum value and deleting the corresponding row and column to obtain the best matching result of the text position rectangle and the candidate Chinese character.

[0015] Furthermore, for performing object detection to obtain the position rectangles of all Chinese characters in the picture, YOLOX is used for object detection.

[0016] Furthermore, using the obtained rectangle as the basis for adaptive foreground - background segmentation to perform foreground - background segmentation, the GrabCut foreground - background segmentation algorithm in OpenCV is adopted.

[0017] Furthermore, converting the sub - figure into a binary image, detecting the contours of the binary image and obtaining the corresponding circumscribed rectangle, the contour detection method in OpenCV is adopted.

[0018] Furthermore, using the text recognition model respectively for recognition, and the text recognition model adopted is paddleOCR.

[0019] Furthermore, using the obtained rectangle as the basis for adaptive foreground - background segmentation to perform foreground - background segmentation, adding the foregrounds containing Chinese characters obtained by foreground - background segmentation, and obtaining a verification code picture containing only the foreground of Chinese characters. It also includes:

[0020] If the proportion of the area of the segmented foreground is less than the preferred probability of 10%, it is determined that the Chinese - character foreground is mis - taken as the background and removed. In this case, abandon the segmentation result and use the original picture as the verification code picture containing the Chinese - character foreground.

[0021] This application proposes a method for recognizing tilted Chinese - character click - to - select verification codes. Compared with the existing click - to - select verification code recognition methods, it corrects the tilted Chinese characters in the verification code and improves the recognition accuracy of Chinese - character click - to - select verification codes. Description of the Drawings

[0022] Figure 1 It is the flow chart of the method for recognizing tilted Chinese - character click - to - select verification codes of this application.

[0023] Figure 2This is the foreground image of Chinese characters after the division of this application.

[0024] Figure 3 This is the figure of the binary image contour outlined by the minimum bounding rectangle of this application.

[0025] Figure 4 This is the schematic diagram of the inclination angle of Chinese characters in this application.

[0026] Figure 5 This is the list of candidate syndrome subgraph recognition result probabilities.

[0027] Figure 6 This is the list of final recognition results. Detailed implementation manners

[0028] In order to make the objectives, technical solutions and advantages of this application clearer, the following further details this application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0029] In one embodiment, as Figure 1 shown, a method for recognizing an inclined Chinese character click verification code is proposed, including:

[0030] Step S1, obtain the verification code picture and perform object detection to obtain the position rectangles of all Chinese characters in the picture.

[0031] When the verification code needs to be verified, an inclined Chinese character click interface will pop up, prompting the user to click the Chinese characters on the interface in the prompted order, and submit the clicked verification code picture after the user clicks.

[0032] Then, perform object detection on the verification code picture, for example, use YOLOX to perform object detection, and obtain the rectangular frame position information of all inclined Chinese characters in the verification code picture.

[0033] In this embodiment, N is used to represent the number of all Chinese characters in the verification code picture. Thus, through object detection, the rectangular frame position vectors of N Chinese characters are obtained: box = [box1, box2,..., box N , where the i-th rectangular frame is represented as: box i = [x i1 , y i1 , x i2 , y i2 , (x i1 , y i1 ) represents the upper left vertex coordinates of the rectangular frame, and (x i2 , y i2 ) represents the lower right vertex coordinates of the rectangular frame.

[0034] Step S2: Use the obtained rectangular box as the basis for adaptive foreground and background segmentation to perform foreground and background segmentation. Add the foregrounds containing Chinese characters obtained from the foreground and background segmentation to obtain a verification code image containing only the foreground of Chinese characters.

[0035] In this embodiment, the GrabCut foreground and background segmentation algorithm in OpenCV is used. It loops N times, and for each position box of the Chinese character rectangular box in turn i perform foreground and background segmentation, that is, use the rectangular box position box i as the input parameter of the algorithm for foreground and background style. After segmentation, retain the foreground of Chinese characters, denoted as grabcutImg i , and add the foregrounds after N times of segmentation to obtain a verification code image grabcutImg with only N foregrounds of Chinese characters:

[0036]

[0037] In a specific embodiment, the obtained verification code image containing the foreground of tilted Chinese characters is as Figure 2 shown.

[0038] Step S3: Crop the verification code image containing only the foreground of Chinese characters according to the rectangular box position, crop out the sub-images corresponding to each Chinese character, convert the sub-images into binary images, detect the contours of the binary images and obtain the corresponding minimum circumscribed rectangular boxes.

[0039] In this embodiment, for the verification code image containing the foreground of tilted Chinese characters, crop out the sub-images corresponding to each Chinese character rectangular box boxImg = [boxImg1, boxImg2,..., boxImg N , convert it into a binary image, and then use the OpenCV contour detection method to obtain the contours of the binary image.

[0040] Then use the minimum circumscribed rectangle to enclose the contour points. As Figure 3 shown, obtain the four-point coordinates rect = [rect1, rect2,..., rect N of the circumscribed rectangle. The i-th circumscribed rectangle is expressed as: rect i = [[x i1 , y i1 , [x i2 , y i2 , [x i3 , y i3 , [x i4 , y i4 , [x i1 , y i1 , [x i2 , y i2 , [x i3 , y i3,[x i4 ,y i4 represent the coordinates of the four vertices respectively.

[0041] Step S4: For the circumscribed rectangle of each sub - graph, calculate the slopes of two adjacent sides, convert them into inclination angles, correct the angles of the sub - graphs respectively according to the obtained inclination angles, and use the original sub - graph and the two corrected sub - graphs as the sub - graphs to be recognized.

[0042] In this embodiment, sub - graphs with three different angles angle = [angle1, angle2, angle3] are used as the sub - graphs to be recognized.

[0043] Specifically, as Figure 4 shown, find the left and bottom two points among the four points, calculate the slope k = - 1 / ((y1 - y2) / (x1 - x2)) of the line where they are located, and then convert it into an angle angle1 = arctan(k)*180 / π. Similarly, the angle corresponding to the line where the right and bottom two points are located is angle2, and there is a relationship angle1 = angle2 + 90°. Then rotate the boxImg by the specified angle respectively to obtain two sub - graphs to be recognized.

[0044] The third sub - graph to be recognized is the original sub - graph, that is, angle3 = 0°.

[0045] Step S5: For the three sub - graphs to be recognized of each Chinese character, use the text recognition model to recognize them respectively, obtain the probability that each sub - graph to be recognized is recognized as each candidate Chinese character, select the sub - graph to be recognized corresponding to the maximum probability, and form a probability matrix with the sub - graphs to be recognized selected from all Chinese characters in the verification code picture and their probabilities of being recognized as each candidate Chinese character.

[0046] In this embodiment, it is assumed that the verification code prompt includes M (M <= N) Chinese characters word = [word1, word2,..., word M as candidate Chinese characters.

[0047] For each rectangular box, after obtaining the sub - graphs to be recognized with three different angles, input them into the text recognition model paddleOCR for recognition respectively, and obtain the probabilities corresponding to the M candidate Chinese characters from the model output:

[0048] answer = [answer1, answer2, answer3] (answer i = [probability1, probability2,..., probability M ).

[0049] For example, there are three candidate Chinese characters: meat, sandwich, and cake. In the probability table of the three candidate Chinese characters, choose the answer with the highest probability:

[0050] bestAnswer=answer i ,max[max(answer),max(answer2),max(answer3)]∈answer i .

[0051] Because it has the highest probability of corresponding to a character in the candidate Chinese characters, such as Figure 5 As shown, the maximum probability is marked in bold and the probability array bestAnswer is underlined. In this way, each boxImg corresponds to an optimal correction angle and a probability array bestAnswer of a candidate Chinese character, and N boxImgs correspond to an N*M probability matrix mat.

[0052] Select a maximum probability in mat, and the row and column (i, j) where it is located determines the corresponding small image boxImg i and candidate Chinese characters word j , then delete the i-th row and j-th column from mat, repeat M times to get the best matching result.

[0053] like Figure 6 As shown, the maximum probability in the first round is 0.94733 in the second row and second column, which means that the second picture corresponds to the second candidate Chinese character. Then the second row and second column are deleted. The maximum probability in the second round is 0.85409 in the third row and first column, which means that the third picture corresponds to the first candidate Chinese character. Then the third row and first column are deleted. In the third round, only 0.00072 in the first row and second column remains, which means that the first picture corresponds to the second candidate Chinese character.

[0054] According to the matching results obtained, the order of the Chinese character words in the candidate Chinese characters is determined to determine the order of the Chinese character rectangular box positions that need to be selected.

[0055] In this embodiment, "GrabCut" is an interactive image segmentation algorithm that automatically segments the image by marking the foreground with a rectangular box. For scenes with complex backgrounds, the segmentation may not perform as expected, with the foreground of Chinese characters mistakenly being removed as background. The foreground area percentage of the segmented image can be used to determine whether this occurs. In this case, GrabCut segmentation is abandoned and the original image is retained.

[0056] For example, if the segmented foreground area accounts for less than the preferred probability of 10%, it is determined that the Chinese character foreground is mistakenly regarded as the background and is removed. In this case, the segmentation result is discarded and the original image is used as the verification code image containing the Chinese character foreground.

[0057] In this embodiment, the "binary image" refers to first converting a three-channel image into a single-channel grayscale image, and then setting all non-zero pixels to 255.

[0058] In this embodiment, "contour detection" and "minimum bounding rectangle" are existing methods in OpenCV, but the connection between the two methods requires manual summarization of multi-level contour points.

[0059] The above-described embodiments merely represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An inclined Chinese character click verification code recognition method, characterized in that, The described method for recognizing slanted Chinese character click verification codes includes: Obtain the verification code image and perform object detection to obtain the position rectangular frames of all Chinese characters in the image; Use the obtained rectangular frames as the basis for adaptive foreground and background segmentation to perform foreground and background segmentation. Add the foregrounds containing Chinese characters obtained from the foreground and background segmentation to obtain a verification code image containing only the foreground of Chinese characters; Crop the verification code image containing only the foreground of Chinese characters according to the position of the rectangular frame, crop out the sub-images corresponding to each Chinese character, convert the sub-images into binary images, detect the contours of the binary images and obtain the corresponding minimum bounding rectangular frames; For the bounding rectangular frames of each sub-image, calculate the slopes of two adjacent sides, convert them into tilt angles, and correct the angles of the sub-images respectively according to the obtained tilt angles. Use the original sub-image and the two corrected sub-images as the sub-images to be recognized; For the three sub-images to be recognized of each Chinese character, use a text recognition model to perform recognition respectively to obtain the probabilities of each sub-image to be recognized as each candidate Chinese character. Select the sub-image to be recognized corresponding to the maximum probability. The sub-images to be recognized selected for all Chinese characters in the verification code image and their probabilities of being recognized as each candidate Chinese character form a probability matrix; In the probability matrix, use the method of repeatedly finding the maximum value and deleting the corresponding row and column to obtain the best matching results of the text position rectangular frames and candidate Chinese characters.

2. The tilted Chinese character click verification code recognition method according to claim 1, wherein For the step of performing object detection to obtain the position rectangular frames of all Chinese characters in the image, YOLOX is used for object detection.

3. The inclined Chinese character click verification code recognition method according to claim 1, characterized in that For the step of using the obtained rectangular frames as the basis for adaptive foreground and background segmentation to perform foreground and background segmentation, the GrabCut foreground and background segmentation algorithm in OpenCV is used.

4. The method for identifying the tilted Chinese character click verification code according to claim 1, wherein For the step of converting the sub-image into a binary image, detecting the contours of the binary image and obtaining the corresponding bounding rectangular frame, the OpenCV contour detection method is used.

5. The method for identifying the inclined Chinese character click verification code according to claim 1, wherein For the step of using a text recognition model to perform recognition respectively, the text recognition model used is paddleOCR.

6. The inclined Chinese character click verification code recognition method according to claim 1, wherein For the step of using the obtained rectangular frames as the basis for adaptive foreground and background segmentation to perform foreground and background segmentation, adding the foregrounds containing Chinese characters obtained from the foreground and background segmentation to obtain a verification code image containing only the foreground of Chinese characters, it further includes: If the proportion of the foreground area segmented is less than the preferred probability of 10%, it is determined that the Chinese character foreground is misrecognized as the background and removed. In this case, abandon the segmentation result and use the original image as the verification code image containing the foreground of Chinese characters.

Images