Slider Captcha recognition method and system based on RPA and LBP features

Abstract

The present application relates to the technical field of Internet, and more particularly to a slider verification code recognition method and system based on RPA and LBP features. The method comprises the following steps: S1, obtaining a slider gap picture and a background picture by using an RPA element screenshot component; S2, calculating the coordinate position of the slider gap on the background picture based on LBP features; S3, restoring the slider to the leftmost original position, and cracking the slider verification code according to the coordinate position of the slider gap on the background picture and the coordinate distance in the X-axis direction. The present application has the characteristics of being able to efficiently describe the texture of an image, being less affected by the brightness of the image, having higher robustness, small calculation amount, fast operation speed, and being more conducive to deployment.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, specifically to a slider CAPTCHA recognition method and system based on RPA and LBP features. Background Technology

[0002] With the development of internet applications and the widespread use of web pages, CAPTCHAs are commonly used for human-computer verification to prevent data access by robots or web crawlers. Among them, slider CAPTCHAs are widely used in web pages and application systems due to their strong interactivity, simple user operation, and high difficulty in cracking.

[0003] With the widespread application of RPA (Robotic Process Automation) technology, employees can be assisted in handling a large number of rule-based, repetitive workflow tasks. In these scenarios, automatic recognition of slider CAPTCHAs is required.

[0004] Existing methods for automatically recognizing slider CAPTCHAs can be broadly categorized into two types. One type is based on traditional color block recognition or template matching. The general idea behind this type is to first obtain the slider image and then find the corresponding region in the larger background image using color blocks, edges, or template matching. However, this method often has low accuracy and is easily affected by background interference, thus limiting its application scenarios. The other type uses neural networks. This type of method detects the outline of the slider notch and then finds the optimal target as the final result. This type of method has high computational cost, slow recognition speed, and large neural network models, making direct deployment impractical. Neural network training is data-driven, requiring data collection and manual annotation, resulting in high development costs and weak generalization ability for new scenarios.

[0005] Therefore, it is very important to design a slider CAPTCHA recognition method and system based on RPA and LBP features that can efficiently describe the texture of an image, is not affected by the brightness of the image, has higher robustness, low computational cost, fast operation speed, and is more conducive to deployment. Summary of the Invention

[0006] This invention aims to overcome the problems of existing automatic slider CAPTCHA recognition methods, such as low accuracy, susceptibility to background interference, high computational cost, slow recognition speed, and difficulty in direct deployment. It provides a slider CAPTCHA recognition method and system based on RPA and LBP features that can efficiently describe the texture of images, is unaffected by the brightness of the image, has higher robustness, lower computational cost, faster processing speed, and is more conducive to deployment.

[0007] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0008] A slider CAPTCHA recognition method based on RPA and LBP features, where RPA stands for Robotic Process Automation and LBP stands for Local Binary Pattern, includes the following steps;

[0009] S1, use the RPA element screenshot component to obtain the slider gap image and background image;

[0010] S2, based on LBP features, calculate the coordinate position of the slider notch on the background image;

[0011] S3 restores the slider to its original leftmost position. Based on the coordinates of the slider notch on the background image, slide the slider a certain distance along the X-axis to crack the slider CAPTCHA.

[0012] Preferably, step S1 includes the following steps:

[0013] S11, Enter the target and select the verification code image on the webpage as the area element;

[0014] S12, when the slider is not moving, use the RPA element screenshot component to take a screenshot of the slider verification code area and obtain the background image;

[0015] S13, move the slider to the far right, and use the RPA element screenshot component again to take a screenshot of the interface when the slider is moved to the far right.

[0016] Preferably, step S1 further includes the following steps:

[0017] S14, Subtract the pixel value of the background image from the screenshot of the interface when the slider is moved to the far right to obtain the area with the largest pixel difference and use it as the position area before and after the slider notch;

[0018] S15, cut out the background image, select the left side area as the slider area, and obtain the slider notch image.

[0019] Preferably, step S2 includes the following steps:

[0020] S21, Calculate the LBP features of the slider notch image, as follows:

[0021] The LBP operator is defined as follows: within a 3*3 window, using the center pixel value as a threshold, the grayscale values ​​of the eight adjacent pixels are compared with the center pixel value. If the surrounding pixel values ​​are greater than the center pixel value, the corresponding pixel is marked as 1; otherwise, it is marked as 0. The comparison of the eight points in the 3*3 neighborhood generates an 8-bit binary number. This 8-bit binary number is converted into a decimal number, i.e., the LBP code, to obtain the LBP value of the corresponding center pixel of the window. The LBP value is then used to reflect the texture information of the corresponding region.

[0022] S22, crop the leftmost slider area of ​​the background image to obtain the remaining area image and calculate the LBP features of the remaining area image.

[0023] Preferably, step S2 further includes the following steps:

[0024] S23, on the LBP feature map of the remaining region image, a sliding window is used to slide, and the LBP feature map within the sliding window is extracted sequentially.

[0025] The size of the sliding window is the same as the size of the slider notch diagram;

[0026] S24. Calculate the difference between the LBP feature map in all extracted sliding windows and the LBP feature map of the slider gap map, and select the result with the smallest difference as the final slider gap position.

[0027] Preferably, in step S23, a coordinate system is established with the top left corner of the remaining area image as the origin, the horizontal direction as the X-axis, and the vertical direction as the Y-axis.

[0028] Preferably, in step S24, the difference is calculated as follows:

[0029] ;

[0030] in, represents the LBP feature map within the sliding window and the LBP feature map of the slider gap, respectively, and p represents the coordinate point on the LBP feature map.

[0031] This invention also provides a slider CAPTCHA recognition system based on RPA and LBP features, including;

[0032] The image acquisition module is used to capture the slider notch and background image using the RPA element screenshot component;

[0033] The coordinate calculation module is used to calculate the coordinate position of the slider notch on the background image based on LBP features;

[0034] The CAPTCHA cracking module is used to restore the slider to its original leftmost position and crack the slider CAPTCHA by sliding the slider notch at the coordinates of the notch on the background image and the distance along the X-axis.

[0035] Compared with the prior art, the beneficial effects of this invention are: (1) This invention uses RPA screenshot and image difference calculation method to obtain the small image of the CAPTCHA slider gap and the large image of the background, which solves the problem that the CAPTCHA image cannot be directly downloaded in some usage scenarios; (2) This invention uses LBP features as the feature representation of the image, which can more effectively represent the texture features of the image, is not affected by factors such as color, has strong robustness and stable effect; (3) The solution of this invention has a small amount of computation, is fast, and can run on PC devices with low configuration. Compared with the neural network model solution, it is easier to deploy and use. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of an interface screenshot when the slider is not moved in this invention;

[0037] Figure 2 This is a schematic diagram of the interface screenshot when the slider is moved to the far right in this invention;

[0038] Figure 3 This is a schematic diagram of the positional area before and after the slider notch in this invention;

[0039] Figure 4 This is a schematic diagram of obtaining a slider notch image in this invention;

[0040] Figure 5 This is a schematic diagram of the LBP calculation principle in this invention;

[0041] Figure 6 This is a schematic diagram illustrating the calculation of the LBP feature of the slider notch map in this invention;

[0042] Figure 7 This is a schematic diagram illustrating the calculation of LBP features for the remaining region image in this invention.

[0043] Figure 8 This is a schematic diagram of using a sliding window to match the most similar slider gap map (LBP) feature in this invention.

[0044] Figure 9 This is a flowchart illustrating a practical application of the slider CAPTCHA recognition method based on RPA and LBP features provided in this embodiment of the invention. Detailed Implementation

[0045] To more clearly illustrate the embodiments of the present invention, specific implementation methods will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0046] This invention provides a slider CAPTCHA recognition method based on RPA and LBP features, comprising the following steps;

[0047] S1, use the RPA element screenshot component to obtain the slider gap image and background image;

[0048] S2, based on LBP features, calculate the coordinate position of the slider notch on the background image;

[0049] S3 restores the slider to its original leftmost position. Based on the coordinates of the slider notch on the background image, slide the slider a certain distance along the X-axis to crack the slider CAPTCHA.

[0050] In step S1, due to limitations of some applications or anti-scraping technologies of some web pages, there are situations where slider verification code images cannot be downloaded directly. This invention uses an RPA element screenshot component to take screenshots, and then combines them with image processing algorithms to solve this problem.

[0051] The RPA element screenshot component used in this invention can be easily configured by simply dragging and dropping it onto the panel and configuring its properties. These properties include the input target, save path, filename, and output variables. After capturing a screenshot of a specified element, the RPA element screenshot component saves the result to the set screenshot save path.

[0052] Specifically, step S1 includes the following steps:

[0053] S11, Enter the target and select the image on the webpage that needs to be verified by the slider code as the area element. Use the RPA component, which will automatically detect the element block and select the verification code image area as the target area.

[0054] S12, when the slider is not moving, use the RPA element screenshot component to take a screenshot of the slider's CAPTCHA area and obtain the background image, such as... Figure 1 As shown, the slider's movement is 0 at this point;

[0055] S13, move the slider to the far right (note that the mouse button should not be released at this time), and use the RPA element screenshot component again to take a screenshot of the interface when the slider is moved to the far right, as shown below. Figure 2 As shown, the slider moves approximately equal to the width of the background image.

[0056] The purpose of sliding the slider to the far right is to determine the slider position before and after sliding by calculating the pixel change area of ​​the two images.

[0057] S14, Figure 1 and Figure 2 Subtracting the pixel values ​​of two images yields the region with the largest pixel difference, which is then used as the area before and after the slider notch. Figure 3 As shown in the diagram. The black area represents two images with a pixel difference of 0, while the white area represents two images with a drastic change in pixel count. The area with the pixel change is the area before and after the slider is moved.

[0058] S15, select the left-hand position area as the slider area, Figure 1 Perform cutout on the image to obtain the slider notch image, such as... Figure 4 As shown, the cut-out image is the slider image block.

[0059] Specifically, step S2 includes the following steps:

[0060] S21, Calculate the LBP features of the slider notch image. The visualization effect of the LBP features is as follows: Figure 6 As shown in the figure, LBP features can fully display the texture features of image patches;

[0061] like Figure 5 As shown, the LBP operator is defined as follows: within a 3*3 window, using the center pixel value as a threshold, the grayscale values ​​of the eight adjacent pixels are compared with the center pixel value. If the surrounding pixel values ​​are greater than the center pixel value, the corresponding pixel is marked as 1; otherwise, it is marked as 0. The comparison of the eight points in the 3*3 neighborhood generates an 8-bit binary number, which is then converted into a decimal number, i.e., the LBP code. There are 256 possible LBP codes, ultimately yielding the LBP value of the corresponding center pixel. This LBP value is then used to reflect the texture information of the corresponding region.

[0062] Local Binary Pattern (LBP) is an operator used to describe local features of an image. LBP features have significant advantages such as gray-level invariance and rotation invariance.

[0063] S22, cut off Figure 1 The leftmost slider area captures the remaining region image and calculates its LBP features. The visualization results of these features are shown below. Figure 7 As shown.

[0064] S23, on the LBP feature map of the remaining region image, a sliding window is used ( Figure 8 The bounding box in the window is slid across the window, and the LBP feature maps within the sliding window are extracted sequentially.

[0065] The size of the sliding window is the same as the size of the slider notch diagram;

[0066] Figure 8 In this context, a coordinate system is established with the top left corner of the remaining area image as the origin, the horizontal direction as the X-axis, and the vertical direction as the Y-axis.

[0067] S24, calculate the difference between the LBP feature map of all extracted sliding windows and the LBP feature map of the slider gap map, and select the result with the smallest difference as the final slider gap position. Figure 8 (The box located in the middle).

[0068] The specific method for calculating the difference is as follows:

[0069] ;

[0070] in, represents the LBP feature map within the sliding window and the LBP feature map of the slider gap, respectively, and p represents the coordinate point on the LBP feature map.

[0071] After obtaining the optimal slider notch position, restore the slider to its original leftmost position, and then slide it horizontally along the x-axis to crack the CAPTCHA.

[0072] This invention also provides a slider CAPTCHA recognition system based on RPA and LBP features, including;

[0073] The image acquisition module is used to capture the slider notch and background image using the RPA element screenshot component;

[0074] The coordinate calculation module is used to calculate the coordinate position of the slider notch on the background image based on LBP features;

[0075] The CAPTCHA cracking module is used to restore the slider to its original leftmost position and crack the slider CAPTCHA by sliding the slider notch at the coordinates of the notch on the background image and the distance along the X-axis.

[0076] Figure 9 This is a flowchart illustrating the specific steps to crack a slider CAPTCHA. The present invention, in conjunction with specific embodiments, provides a slider CAPTCHA recognition method based on RPA and LBP features, including the following steps:

[0077] 1. Use RPA software to take a screenshot of the slider CAPTCHA area to obtain the first slider CAPTCHA image;

[0078] 2. Using RPA software, move the slider to the far right and take another screenshot of the slider verification code area to obtain the second slider verification code image. Note that the mouse should not be released at this time.

[0079] 3. Calculate the difference between the first and second slider CAPTCHA images. The difference between the two images represents the position of the slider before and after movement. Select the position of the left slider before movement and cut out the image from the first slider CAPTCHA image to obtain the small image of the slider notch.

[0080] 4. Calculate the LBP features of the slider notch mini-image obtained in step 3 to obtain the LBP feature map of the slider notch mini-image;

[0081] 5. Crop the slider area before the slider moves from the first slider CAPTCHA image to obtain the third slider CAPTCHA image, and calculate the LBP feature map of the third slider CAPTCHA image;

[0082] 6. Using a sliding window of the same size as the slider notch image, slide it sequentially on the LBP feature map of the third slider CAPTCHA image, and calculate the difference between the feature map in the window and the LBP feature map of the slider notch image, and find the feature region with the smallest difference.

[0083] 7. The x-axis coordinate of the area obtained in step 6 is the final moving distance d. Move the mouse back to its original position and then move it d pixels to the right to crack the slider CAPTCHA.

[0084] The innovative aspects of this invention are as follows:

[0085] 1. This invention solves the problem of CAPTCHA images being unable to be downloaded in certain scenarios. This invention uses RPA software to capture element screenshots and combines them with image processing methods to obtain a small image of the slider notch and a large background image, providing a WYSIWYG (What You See Is What You Get) solution without requiring downloading the CAPTCHA image.

[0086] 2. This invention uses LBP features, resulting in high recognition accuracy. The choice of LBP features as the feature matching method in this invention not only reduces computational cost but also achieves higher accuracy and robustness compared to general template matching methods.

[0087] 3. This invention designs a complete CAPTCHA solution that is easy to deploy and use. The entire solution has low computational load, high speed, and is easier to use in scenarios such as intranets and low-configuration devices.

[0088] The above description is merely a detailed explanation of preferred embodiments and principles of the present invention. For those skilled in the art, there may be changes in specific implementation methods based on the ideas provided by the present invention, and these changes should also be considered within the scope of protection of the present invention.

Claims

1. A slider CAPTCHA recognition method based on RPA and LBP features, where RPA stands for Robotic Process Automation and LBP stands for Local Binary Pattern Recognition, characterized in that... Includes the following steps; S1, use the RPA element screenshot component to obtain the slider gap image and background image; S2, based on LBP features, calculate the coordinate position of the slider notch on the background image; S3, restore the slider to its original leftmost position, and based on the coordinates of the slider notch on the background image, slide the slider a certain distance along the X-axis to crack the slider verification code; Step S1 includes the following steps: S11, Enter the target and select the verification code image on the webpage as the area element; S12, when the slider is not moving, use the RPA element screenshot component to take a screenshot of the slider verification code area and obtain the background image; S13, move the slider to the far right, and use the RPA element screenshot component again to take a screenshot of the interface when the slider is moved to the far right. Step S1 also includes the following steps: S14, Subtract the pixel value of the background image from the screenshot of the interface when the slider is moved to the far right to obtain the area with the largest pixel difference and use it as the position area before and after the slider notch; S15, cut out the background image, select the left side area as the slider area, and obtain the slider notch image; Step S2 includes the following steps: S21, Calculate the LBP features of the slider notch image, as follows: The LBP operator is defined as follows: within a 3*3 window, using the center pixel value as a threshold, the grayscale values ​​of the 8 adjacent pixels are compared with the center pixel value. If the surrounding pixel values ​​are greater than the center pixel value, the corresponding pixel is marked as 1; otherwise, it is marked as 0. The comparison of the 8 points in the 3*3 neighborhood generates an 8-bit binary number. The 8-bit binary number is converted into a decimal number to obtain the LBP value of the corresponding center pixel of the window. The LBP value is used to reflect the texture information of the corresponding area. S22, crop the leftmost slider area of ​​the background image to obtain the remaining area image and calculate the LBP features of the remaining area image; S23, on the LBP feature map of the remaining region image, a sliding window is used to slide, and the LBP feature map within the sliding window is extracted sequentially. The size of the sliding window is the same as the size of the slider notch diagram; S24. Calculate the difference between the LBP feature map in all extracted sliding windows and the LBP feature map of the slider gap map, and select the result with the smallest difference as the final slider gap position.

2. The slider CAPTCHA recognition method based on RPA and LBP features according to claim 1, characterized in that, In step S23, a coordinate system is established with the top left corner of the remaining area image as the origin, the horizontal direction as the X-axis, and the vertical direction as the Y-axis.

3. The slider CAPTCHA recognition method based on RPA and LBP features according to claim 2, characterized in that, In step S24, the difference is calculated as follows: ； in, represents the LBP feature map within the sliding window and the LBP feature map of the slider gap, respectively, and p represents the coordinate point on the LBP feature map.

4. A slider CAPTCHA recognition system based on RPA and LBP features, used to implement the slider CAPTCHA recognition method based on RPA and LBP features as described in any one of claims 1-3, characterized in that, The slider CAPTCHA recognition system based on RPA and LBP features includes: The image acquisition module is used to capture the slider notch and background image using the RPA element screenshot component; The coordinate calculation module is used to calculate the coordinate position of the slider notch on the background image based on LBP features; The CAPTCHA cracking module is used to restore the slider to its original leftmost position and crack the slider CAPTCHA by sliding the slider notch at the coordinates of the notch on the background image and the distance along the X-axis.

Images