Image verification code test method and device, equipment and storage medium

Abstract

The invention discloses an image verification code testing method and device, equipment and a storage medium. The method is characterized by comprising the steps of collecting an image verification code of a to-be-tested page; identifying the image verification code through a pre-trained character identification neural network model, and determining text verification information, verification operation object information and to-be-verified object pixel points; generating an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified; and automatically testing the image verification code according to the image verification script, and determining a test result of the image verification code. The verification data of the image verification code can be effectively extracted, and the test efficiency and the test accuracy of the image verification code are improved.

Description

Technical Field

[0001] The present invention relates to the field of image verification codes, and in particular to a testing method, device, equipment and storage medium for image verification codes. Background Art

[0002] An image verification code can be a verification method for distinguishing between users and automated software. An image verification code requires a user to complete a specific task to verify whether he or she is a real user. Usually, the image verification code is displayed in the form of an image on the user's display device, requiring the user to identify and click on the corresponding characters to complete the verification of the image verification code. The image verification code accepts the user's click operation, identifies the user's click operation and determines whether the user is a real user or a fake user. The test of the image verification code is often performed by manually locating the characters in the image verification code, and then writing a corresponding test script to test the image verification code. Although the prior art can automatically write test scripts, for the image verification code, the image verification code data and location and other information still need to be manually set by the tester, and the test efficiency is not improved. Summary of the invention

[0003] The present invention provides a method, device, equipment and storage medium for testing an image verification code, so as to solve the technical problem that the image verification code cannot be fully automatically tested in the prior art.

[0004] According to one aspect of the present invention, a method for testing an image verification code is provided, comprising:

[0005] Collect the image verification code of the page to be tested;

[0006] The image verification code is recognized by a pre-trained character recognition neural network model to determine text verification information, verification operation object information and pixel points of the object to be verified;

[0007] Generate an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified;

[0008] The image verification code is automatically tested according to the image verification script to determine a test result of the image verification code.

[0009] According to another aspect of the present invention, there is provided a testing device for an image verification code, comprising:

[0010] A data collection module is used to collect the image verification code of the page to be tested;

[0011] A data extraction module is used to recognize the image verification code through a pre-trained character recognition neural network model to determine text verification information, verification operation object information and pixel points of the object to be verified;

[0012] A test script generation module, used to generate an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified;

[0013] The automated testing module is used to automatically test the image verification code according to the image verification script to determine the test result of the image verification code.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image verification code testing method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image verification code testing method described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention can automatically identify the image verification code of the page to be tested, identify the text verification information and the verification operation object information used for the test, and can locate the information of the object to be verified and the pixel points located in the device in detail, and automatically generate an image verification script based on the text verification information and the verification operation object information, and perform automatic testing based on the image verification script to obtain corresponding test results; it can automatically identify the data required for the test through a character recognition neural network model, effectively improve the degree of automation of the automatic testing of the image verification code, and can improve the test efficiency for the automatic testing of the image verification code, and effectively reduce the test time of the tester; it solves the technical problem that the image verification code cannot be fully automatically tested in the prior art, and improves the test efficiency and test accuracy of the image verification code.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a method for testing an image verification code is provided for an embodiment of the present invention;

[0023] Figure 2 A flowchart of another method for testing an image verification code provided by an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of the structure of a testing device for an image verification code provided by an embodiment of the present invention;

[0025] Figure 4 A schematic structural diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Figure 1A flowchart of a method for testing an image verification code is provided for an embodiment of the present invention. This embodiment can be applied to the case where a test device performs an automated test on the image verification code function displayed by the test device. The method can be executed by a test device for an image verification code. The test device for an image verification code can be implemented in the form of hardware and / or software. The test device for an image verification code can be configured in an electronic device. Figure 1 As shown, the method includes:

[0029] S110, collecting the image verification code of the page to be tested.

[0030] Among them, the page to be tested can be an image verification code page displayed by the display device of the test device; the page to be tested can be displayed by a window page or a complete page, and the image verification code is displayed in a pavement manner in the window page; or the image verification code is displayed in the complete page.

[0031] Among them, the image verification code can be a form of verification for determining whether the visitor of a webpage or application is a human. The image verification code mainly includes text verification information and a region to be verified in the page to be tested; the text verification information can be information prompting the user to complete the image verification; for example, the text verification information can be "Please complete the security verification, if you want to drag the slider to the right to fill the puzzle", "Please click the Chinese characters in order: A, B, C", "Please connect the pattern in numerical order", "Please enter the letters, words and numbers in the above picture, distinguishing or not distinguishing upper and lower case". The region to be verified can be an object that the user needs to operate and / or an object and region operated by the user. It should be noted that in different types of image verification codes, the area to be verified is different. For example, in the above example "Please complete the security verification and drag the slider right to fill the puzzle", the area to be verified is the slider dragged by the user and the picture that the slider needs to be spliced ​​together. The slider dragged by the user is the object that the user needs to operate; the puzzle corresponding to the slider and the picture that needs to be spliced ​​together are the objects and areas operated by the user; in the above example "Please click on the Chinese characters in order: A, B, C", the area to be verified is the picture containing the Chinese characters "A, B, C", and the objects that the user needs to operate are the Chinese characters "A, B, C".

[0032] Optionally, the page to be tested is usually run in a computer terminal, and the page to be tested is displayed to the user through the screen display device of the computer terminal. When collecting the image verification code of the page to be tested, the page to be tested can be directly captured by an automated screenshot tool, and the screenshot size of the automated screenshot tool can be set to ensure that the captured images are the same; or the image verification code can be directly downloaded by requesting the page to be tested.

[0033] S120, identifying the image verification code through a pre-trained character recognition neural network model to determine text verification information, verification operation object information and pixel points of the object to be verified.

[0034] The verification operation object information may be the object information that needs verification operation in the image verification code, the verification operation object information includes the area to be verified and the object to be verified, the area to be verified may be the area used for verification in the image verification code, the object to be verified may be the object for the image verification code to implement the verification operation, and the pixel points of the object to be verified may be the pixel point information constituting the object to be verified in the image verification code. For example, taking the image verification code as "Please click on the Chinese characters in order: Jia, Yi, Bing" as an example, the area to be verified may be the image area containing the Chinese characters "Jia, Yi, Bing" in the image verification code, the object to be verified may be the Chinese characters "Jia, Yi, Bing", and the pixel points of the object to be verified may be the pixel points constituting the Chinese characters "Jia, Yi, Bing" in the picture.

[0035] Among them, the character recognition neural network model can be a neural network model pre-trained for extracting character information from an image verification code. It should be noted that the character recognition neural network model can be obtained by training a pre-established full convolutional network neural network model. Optionally, the specific training process is as follows: by collecting image verification codes as training images, scaling the training images to a fixed size, and generating text score graphs and text geometry parameters corresponding to each training image as labels for each training image, generating training sets, test sets, and validation sets, training by inputting the training set into the full convolutional network neural network model, and testing and validating through the test set and validation set, setting the loss functions to score graph loss and binary graph loss, respectively, and selecting Adam (Adaptive Moment Estimation) optimizer as the training optimizer. Finally, the character recognition neural network model is obtained.

[0036] Specifically, the image verification code is input into a pre-trained character recognition neural network model, and the image verification code is automatically recognized by the character recognition neural network model to obtain text verification information, verification operation object information and pixel points of the object to be verified related to the image verification code.

[0037] S130: Generate an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified.

[0038] The image verification script may be a test script that can automatically execute verification of the image verification code. Exemplarily, the text verification information, the verification operation object information and the pixel points of the object to be verified can be read through natural language processing technology, and the corresponding image verification script can be written based on the read text verification information, the verification operation object information and the pixel points of the object to be verified.

[0039] Optionally, a test script is written according to the text verification information, the verification operation object information and the pixels of the object to be verified to generate an image verification script for the test image verification code.

[0040] S140: Automatically test the image verification code according to the image verification script to determine a test result of the image verification code.

[0041] Among them, the test result can be a successful image verification code test or an image verification code test recognition; it should be noted that the test result of the image verification script is related to the expected result set in advance by the tester. According to different expected results set in advance by the tester, the test result will also be different.

[0042] Specifically, the image verification script is executed through the automated testing tool, the image verification code of the page to be tested is automatically tested based on the operation instructions in the image verification script, and the response result of the page to be tested is captured, and then the test result of the image verification code is generated according to the response result.

[0043] The technical solution of the embodiment of the present invention can automatically identify the image verification code of the page to be tested, identify the text verification information and the verification operation object information used for the test, and can locate the information of the object to be verified and the pixel points located in the device in detail, and automatically generate an image verification script based on the text verification information and the verification operation object information, and perform automatic testing based on the image verification script to obtain corresponding test results; it can automatically identify the data required for the test through a character recognition neural network model, effectively improve the degree of automation of the automatic testing of the image verification code, and can improve the test efficiency for the automatic testing of the image verification code, and effectively reduce the test time of the tester; it solves the technical problem that the image verification code cannot be fully automatically tested in the prior art, and improves the test efficiency and test accuracy of the image verification code.

[0044] Figure 2 This is a flowchart of another method for testing an image verification code provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is a specific method for identifying the image verification code by using a pre-trained character recognition neural network model. Figure 2 As shown, the method includes:

[0045] Optionally, the character recognition neural network model includes a text extraction network and a verification range recognition module; the text extraction module is used to extract text information in the image verification code, and the verification range recognition module is used to identify pixel points of the object to be verified in the image verification code.

[0046] S210: Collect the image verification code of the page to be tested.

[0047] S220, performing text region recognition and text recognition on the image verification code through the text extraction network to determine the text verification information and verification operation object information.

[0048] Optionally, the text extraction network includes a convolutional network, a feature pyramid network and an output network; the convolutional network includes multiple convolutional layers and a maximum pooling layer, the convolutional layer is used to extract the features of the image verification code, the maximum pooling layer is used to reduce the dimension of the features, the convolutional network gradually extracts the features of the image verification code, and outputs the features of multiple image verification codes; the feature pyramid network is constructed by FPN (Feature Pyramid Networks, feature pyramid), in an embodiment of the present invention, the feature pyramid network is based on the convolutional network, through top-down upsampling, and through a lateral connection, the upsampled features are added element by element to the corresponding features to obtain a fused multi-scale feature, the output network is composed of a fully connected network, and feature recognition is performed through two different convolutional layers of the fully connected network, and a text score map and a geometric shape parameter are output respectively. Wherein, the text score map is in the range of [0, 1], indicating the probability that each pixel in the image verification code is text; the geometric shape parameter can be the geometric shape parameter of the text area in the image verification code. Exemplarily, if the probability value in the text score map is 1, the pixel is considered to be a pixel of text, and if the probability value in the text score map is 0, the pixel is considered to be the background image of the image verification code. The text geometry parameters can be the pixel coordinates of the four corners of a rectangle in any direction.

[0049] Optionally, the text extraction network can recognize the text in the image verification code, divide the range of the text in the image verification code, and generate a text score map and text geometry parameters of the text, thereby determining the text verification information and verification operation object information.

[0050] Optionally, in another optional embodiment of the present invention, the performing text region recognition and text recognition on the image verification code by the text extraction network to determine the text verification information and verification operation object information includes:

[0051] The image verification code is subjected to feature extraction through the convolutional network to obtain at least one convolutional feature; all the convolutional features are subjected to upsampling and lateral connection through the feature pyramid network to perform feature fusion and determine multi-scale features; the multi-scale features are subjected to convolutional feature processing through the output network to obtain a text score map and text geometric shape parameters; the text verification information and verification operation object information are determined according to the text score map and the text geometric shape parameters.

[0052] Among them, the convolution feature can be a convolution feature map extracted based on the image verification code; the multi-scale feature can be a feature map obtained by upsampling the convolution feature map and fusion of the convolution feature map element by element based on the feature pyramid network. For example, if the image input to the convolution network is an input 512x512x3 image, then after five convolution feature extractions, convolution feature maps of 256x256x64, 128x128x128, 64x64x256, 32x32x512 and 16x16x512 are obtained, and upsampling is performed 4 times through the feature pyramid network: 16x16x512 is upsampled to 32x32x256, and it is added element by element to the convolution feature map of 32x32x256 to obtain a 32x32x512 fused feature map; 32x32x256 is upsampled to 64x The 64x64x128 is upsampled to 128x128x64 and added element by element with the convolution feature map of 64x64x128 to obtain a fused feature map of 64x64x256; the 64x64x128 is upsampled to 128x128x64 and added element by element with the convolution feature map of 128x128x64 to obtain a fused feature map of 128x128x128; the 128x128x64 is upsampled to 256x256x32 and added element by element with the convolution feature map of 256x256x32 to obtain a fused feature map of 256x256x64, and all the fused feature maps are used as multi-scale features.

[0053] Optionally, the output network is a fully connected network, the full connection includes two different convolutional layers, the two different convolutional layers of the fully connected network perform feature recognition, and output text score maps and geometric shape parameters respectively.

[0054] Optionally, after obtaining the text score map and text geometry parameters, the area corresponding to the prompt text information in the image verification code is identified based on the text score map and the text geometry parameters to obtain the text verification information, and the text and text geometry parameters of other areas in the image verification code except the text verification information are used as verification operation object information.

[0055] S230: Performing area recognition on the verification operation object information through the verification range recognition network to determine the pixel points of the object to be verified.

[0056] Optionally, the verification range recognition network can be used to perform differentiable binarization calculations on the verification operation object information, and can convert the text score map into a binary text score map based on the text score map of the verification operation object information; based on the binary text score map, the pixel points of the object to be verified are determined in combination with the text geometry parameters of the verification operation object information.

[0057] Optionally, in another optional embodiment of the present invention, the performing area recognition on the verification operation object information through the verification range recognition network to determine the pixel points of the object to be verified includes:

[0058] The text score map corresponding to the area to be verified is differentiably binarized by the verification range recognition network to obtain a binary map to be verified;

[0059] The text pixels of the binary image to be verified are extracted according to the text geometric shape parameters of the object to be verified by non-maximum suppression to determine the pixels of the object to be verified.

[0060] The binary image to be verified may be a binary image obtained by performing differentiable binarization on the text score image corresponding to the area to be verified.

[0061] Optionally, the process of performing differentiable binarization on the text score map corresponding to the area to be verified is as follows: for each pixel of the text score map, the average value of the local pixel values ​​around the pixel is calculated as the threshold of the pixel, and the pixel average value corresponding to each pixel is obtained. The pixel average value is calculated by a differentiable binarization function and then binarized to obtain a binary map to be verified. Exemplarily, the differentiable binarization function can be a sigmoid function.

[0062] Among them, non-maximum suppression is (Non-Maximum Suppression, NMS).

[0063] Specifically, the text score map corresponding to the area to be verified is differentiably binarized through the verification range recognition network to obtain the binary map to be verified, and the foreground areas corresponding to all connected objects to be verified are extracted based on the binary map to be verified. Text pixel points are extracted in the binary map to be verified based on the text geometric shape parameters of the object to be verified and the foreground area corresponding to the object to be verified through non-maximum suppression to determine the pixel points of the object to be verified.

[0064] S240: Generate an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified.

[0065] Optionally, in another optional embodiment of the present invention, the step of generating an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified includes:

[0066] Acquire the screen coordinate system of the page to be tested, map the pixel points of the object to be verified to the screen coordinate system, and determine the coordinate set of the object to be verified; perform text extraction on the text verification information to determine the verification requirement information; generate the image verification script according to the verification requirement information, the verification operation object information and the coordinate set of the object to be verified.

[0067] The screen coordinate system may be a coordinate system of a display device that displays the page to be tested. It should be noted that the screen coordinate system may be established based on the screen width and height by directly acquiring the screen width and height of the display device.

[0068] Among them, the coordinate set of the object to be verified can be the coordinates of the pixel points of the object to be verified in the screen coordinate system. Optionally, according to the position of the pixel points of the object to be verified in the image verification code, the pixel points of the object to be verified are mapped from the image coordinate system corresponding to the image verification code to the coordinate points corresponding to the screen coordinate system, and the coordinates of the coordinate points are recorded to form the coordinate set of the object to be verified. Exemplarily, in an Android device, the coordinates of the coordinate points can be directly recorded to form the coordinate set of the object to be verified; in an IOS device, the coordinates of the recorded coordinate points need to be converted into a coordinate system to obtain the coordinate set of the object to be verified.

[0069] The verification requirement information may be information describing the verification method of the image verification code, and the verification requirement information is a verification requirement statement composed of text in the text verification information.

[0070] Specifically, the screen coordinate system of the page to be tested is obtained, the pixel points of the object to be verified are mapped to the screen coordinate system, the coordinate set of the object to be verified is determined, text is extracted from the text verification information, the verification requirement information is determined, and an image verification script is generated according to the verification requirement information, the verification operation object information and the coordinate set of the object to be verified.

[0071] Optionally, in another optional embodiment of the present invention, generating the image verification script according to the verification requirement information, the verification operation object information and the coordinate set of the object to be verified includes: loading a pre-connected script generation model, performing semantic parsing on the verification requirement information through the script generation model, and determining the verification operation mode;

[0072] The image verification script is generated by the script generation model according to the verification operation mode, the verification operation object information and the coordinate set of the object to be verified.

[0073] The script generation model may be a pre-trained large language model (LLM) for generating test scripts.

[0074] Among them, the verification operation method can be the verification operation method obtained by understanding the semantic information of the verification requirement information. Exemplarily, the verification operation method can be dragging a slider or sequentially clicking on the Chinese characters "A, B, C".

[0075] Optionally, by preparing a large amount of text data and test scripts as training data, then training a large language model to obtain a script generation model, and setting a connection interface for the script generation model, connecting to the script generation model through the connection interface.

[0076] Specifically, after connecting to the script generation model, load the pre-connected script generation model, perform semantic parsing on the verification requirement information through the script generation model to determine the verification operation method, and generate an image verification script according to the verification operation method, verification operation object information, and set of coordinates of the object to be verified through the script generation model.

[0077] S250. Automatically test the image verification code according to the image verification script, and determine the test result of the image verification code.

[0078] Optionally, in another optional embodiment of the present invention, the automatically testing the image verification code according to the image verification script and determining the test result of the image verification code includes:

[0079] Performing verification simulation on the image verification code according to the image verification script through a preset automated testing tool to obtain the response result of the page to be tested; comparing the response result with the test expected result to determine the test result.

[0080] Among them, the preset automated testing tool can be a software tool that is preset to be able to automatically execute test scripts; Exemplarily, the automated testing tool can be Selenium, WebDriverIO, Cypress, Puppeteer, etc.

[0081] Among them, the test expected result can be the test requirements input by the tester to the script generation model before generating an image test script through the script generation model, and the script generation model outputs the corresponding test expected result based on the test requirements. Exemplarily, the test requirement is default set to a test that passes the verification. The image verification script can be: whether sequentially clicking on the Chinese characters "A, B, C" can pass the verification, then the test expected result can be passing the verification; when the test requirement is changed to a test that does not pass the verification, the image verification script can be whether sequentially clicking on the Chinese characters "A, A, A, B, C" can pass the verification, then the test expected result can be not passing the verification.

[0082] Specifically, the image verification code is verified and simulated according to the image verification script through a preset automated testing tool to obtain the response result of the page to be tested, and the response result is compared with the expected test result to determine the test result. Among them, the response result can be the feedback information based on the verification operation after the page to be tested receives the image verification script to perform the verification operation. Exemplarily, if the verification operation passes the image verification code of the page to be tested, the feedback information is that the verification is passed; if the verification operation does not pass the image verification code of the page to be tested, the feedback information is that the verification cannot be passed.

[0083] Optionally, an embodiment of the present invention also provides an automatic test report generation function, which can automatically analyze and compare the image verification script, test results and the image verification code of the page to be tested to generate a test report; the automatic test report generation function can also perform analysis and comparison based on multiple test results, and automatically generate a test result comparison chart, which is convenient for testers and other personnel to view the reasons for result failure or test success.

[0084] The technical solution of the embodiment of the present invention can automatically identify the data required for the test through the character recognition neural network model, effectively improve the degree of automation of the automatic testing of the image verification code, and can improve the test efficiency for the automatic testing of the image verification code, effectively reducing the test time of the tester; it solves the technical problem that the image verification code cannot be fully automatically tested in the prior art, and improves the test efficiency and test accuracy of the image verification code.

[0085] Figure 3 A schematic diagram of the structure of a test device for an image verification code provided by an embodiment of the present invention. Figure 3 As shown, the device includes: a data acquisition module 310, a data extraction module 320, a test script generation module 330 and an automated test module 340; wherein,

[0086] The data collection module 310 is used to collect the image verification code of the page to be tested;

[0087] The data extraction module 320 is used to recognize the image verification code through a pre-trained character recognition neural network model to determine the text verification information, the verification operation object information and the pixel points of the object to be verified;

[0088] A test script generation module 330, used to generate an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified;

[0089] The automated testing module 340 is configured to automatically test the image verification code according to the image verification script to determine a test result of the image verification code.

[0090] The technical solution of the embodiment of the present invention can automatically identify the image verification code of the page to be tested, identify the text verification information and the verification operation object information used for the test, and can locate the information of the object to be verified and the pixel points located in the device in detail, and automatically generate an image verification script based on the text verification information and the verification operation object information, and perform automatic testing based on the image verification script to obtain corresponding test results; it can automatically identify the data required for the test through a character recognition neural network model, effectively improve the degree of automation of the automatic testing of the image verification code, and can improve the test efficiency for the automatic testing of the image verification code, and effectively reduce the test time of the tester; it solves the technical problem that the image verification code cannot be fully automatically tested in the prior art, and improves the test efficiency and test accuracy of the image verification code.

[0091] Optionally, the data extraction module is specifically used for:

[0092] Performing text region recognition and text recognition on the image verification code through the text extraction network to determine the text verification information and verification operation object information;

[0093] The verification range recognition network performs region recognition on the verification operation object information to determine the pixel points of the object to be verified.

[0094] Optionally, the data extraction module is further used to:

[0095] Extracting features of the image verification code through the convolutional network to obtain at least one convolutional feature;

[0096] Upsampling and horizontally connecting all the convolutional features through the feature pyramid network to perform feature fusion and determine multi-scale features;

[0097] Performing convolution feature processing on the multi-scale features through an output network to obtain a text score map and text geometric shape parameters;

[0098] The text verification information and verification operation object information are determined according to the text score map and the text geometric shape parameters.

[0099] Optionally, the data extraction module is further used to:

[0100] The text score map corresponding to the area to be verified is differentiably binarized by the verification range recognition network to obtain a binary map to be verified;

[0101] The text pixels of the binary image to be verified are extracted according to the text geometric shape parameters of the object to be verified by non-maximum suppression to determine the pixels of the object to be verified.

[0102] Optionally, the test script generation module is specifically used to:

[0103] Acquire the screen coordinate system of the page to be tested, map the pixel points of the object to be verified to the screen coordinate system, and determine the coordinate set of the object to be verified;

[0104] Extracting text from the text verification information to determine verification requirement information;

[0105] The image verification script is generated according to the verification requirement information, the verification operation object information and the coordinate set of the object to be verified.

[0106] Optionally, the test script generation module is further used to:

[0107] Loading a pre-connected script generation model, performing semantic analysis on the verification requirement information through the script generation model, and determining a verification operation mode;

[0108] The image verification script is generated by the script generation model according to the verification operation mode, the verification operation object information and the coordinate set of the object to be verified.

[0109] Optionally, the automated testing module is specifically used for:

[0110] Performing verification simulation on the image verification code according to the image verification script through a preset automated testing tool to obtain a response result of the page to be tested;

[0111] The response result is compared with the expected test result to determine the test result.

[0112] The image verification code testing device provided in the embodiment of the present invention can execute the image verification code testing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0113] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their modes are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0114] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0115] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer grid such as the Internet and / or various telecommunication grids.

[0116] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the test method of the image verification code.

[0117] In some embodiments, the test method of the image verification code may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the test method of the image verification code described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the test method of the image verification code in any other appropriate manner (e.g., by means of firmware).

[0118] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the modes / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0120] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0122] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a grid browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication grid). Examples of communication grids include: a local area network (LAN), a wide area network (WAN), a blockchain grid, and the Internet.

[0123] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication grid. The client and server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0124] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0125] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the image verification code testing method provided in any embodiment of the present invention are implemented. The method includes:

[0126] Collect the image verification code of the page to be tested;

[0127] The image verification code is recognized by a pre-trained character recognition neural network model to determine text verification information, verification operation object information and pixel points of the object to be verified;

[0128] Generate an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified;

[0129] The image verification code is automatically tested according to the image verification script to determine a test result of the image verification code.

[0130] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0131] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0132] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0133] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of grid, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] It should be understood by those skilled in the art that the above modules or steps of the present invention can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a grid composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0135] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0136] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for testing an image verification code, characterized in that: include: Collect the image verification code of the page to be tested; The image verification code is recognized by a pre-trained character recognition neural network model to determine text verification information, verification operation object information and pixel points of the object to be verified; Generate an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified; The image verification code is automatically tested according to the image verification script to determine a test result of the image verification code.

2. The method according to claim 1, characterized in that The character recognition neural network model includes a text extraction network and a verification range recognition module; The image verification code is recognized by a pre-trained character recognition neural network model to determine the text verification information and the pixel points of the object to be verified, including: Performing text region recognition and text recognition on the image verification code through the text extraction network to determine the text verification information and verification operation object information; The verification range recognition network performs region recognition on the verification operation object information to determine the pixel points of the object to be verified.

3. The method according to claim 2, characterized in that The text extraction network includes a convolutional network, a feature pyramid network and an output network; The step of performing text region recognition and text recognition on the image verification code through the text extraction network to determine the text verification information and verification operation object information includes: Extracting features of the image verification code through the convolutional network to obtain at least one convolutional feature; Upsampling and horizontally connecting all the convolutional features through the feature pyramid network to perform feature fusion and determine multi-scale features; Performing convolution feature processing on the multi-scale features through an output network to obtain a text score map and text geometric shape parameters; The text verification information and verification operation object information are determined according to the text score map and the text geometric shape parameters.

4. The method according to claim 3, characterized in that The verification operation object information includes a region to be verified and an object to be verified; the verification operation object information is region identified by the verification range identification network to determine the pixel points of the object to be verified; including: The text score map corresponding to the area to be verified is differentiably binarized by the verification range recognition network to obtain a binary map to be verified; The text pixels of the binary image to be verified are extracted according to the text geometric shape parameters of the object to be verified by non-maximum suppression to determine the pixels of the object to be verified.

5. The method according to claim 1, characterized in that The step of generating an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified includes: Acquire the screen coordinate system of the page to be tested, map the pixel points of the object to be verified to the screen coordinate system, and determine the coordinate set of the object to be verified; Extracting text from the text verification information to determine verification requirement information; The image verification script is generated according to the verification requirement information, the verification operation object information and the coordinate set of the object to be verified.

6. The method according to claim 5, characterized in that The step of generating the image verification script according to the verification requirement information, the verification operation object information and the coordinate set of the object to be verified includes: Loading a pre-connected script generation model, performing semantic analysis on the verification requirement information through the script generation model, and determining a verification operation mode; The image verification script is generated by the script generation model according to the verification operation mode, the verification operation object information and the coordinate set of the object to be verified.

7. The method according to claim 1, characterized in that The automatically testing the image verification code according to the image verification script to determine the test result of the image verification code includes: Performing verification simulation on the image verification code according to the image verification script through a preset automated testing tool to obtain a response result of the page to be tested; The response result is compared with the expected test result to determine the test result.

8. A testing device for an image verification code, characterized in that: include: A data collection module is used to collect the image verification code of the page to be tested; A data extraction module is used to recognize the image verification code through a pre-trained character recognition neural network model to determine text verification information, verification operation object information and pixel points of the object to be verified; A test script generation module, used to generate an image verification script according to the text verification information, the verification operation object information and the pixel points of the object to be verified; The automated testing module is used to automatically test the image verification code according to the image verification script to determine the test result of the image verification code.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the image verification code testing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image verification code testing method according to any one of claims 1 to 7 when executed.

Images