Character-level self-attention network model and character-type verification code security detection method

Abstract

The application discloses a character-level self-attention network model and a character-type verification code security detection method, and the model comprises the following steps: a stacked multi-scale convolution main network can preliminarily extract features of an input character-type verification code picture and output a feature vector and a downsampling operation; a character-level self-attention Transformer network is connected with an output end of the stacked multi-scale convolution main network, an encoder of the character-level self-attention Transformer network is used for representing the feature vector output by the stacked convolution main network, and a decoder is used for obtaining a prediction probability output of a character sequence of the character-type verification code picture according to a right-shift character sequence corresponding to the input character-type verification code picture and an output of the encoder. The model realizes more efficient and robust representation of the character sequence by learning the correlation between adjacent characters, has good generalization performance on a small training data set, and greatly reduces the time and manpower required for recognizing the verification code.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and network security, and in particular to a character-level self-attention network model and a character-type CAPTCHA security detection method. Background Technology

[0002] Character-based CAPTCHAs are images containing complex background noise and distorted characters. Humans can easily recognize the characters, but machines struggle to do so. These CAPTCHAs are widely used for security protection in various networks. To objectively evaluate the security performance of character-based CAPTCHAs, current technologies primarily design end-to-end deep neural networks to attempt to recognize them, and then measure the security of the CAPTCHA by its recognition rate.

[0003] Currently, the main methods for recognizing character-based CAPTCHAs based on deep network models are:

[0004] (1) Recognition method based on convolutional neural network. See comparative literature 1: Wang, J., et al. "CAPTCHA recognition based on deep convolutional neural network". Mathematical Biosciences and Engineering 2019.

[0005] (2) Recognition method based on recurrent neural network-attention mechanism. See comparative literature 2: Yang, Z., et al. “An End-to-End Attack on Text CAPTCHAs”. IEEE Transactions on InformationForensics and Security 2020.

[0006] Existing recognition methods require the collection and annotation of a large number of character-based CAPTCHA images for training deep network models. Often, tens of thousands, or even hundreds of thousands of character-based CAPTCHA images are needed to train the model for effective recognition. However, the frequent updates and volatile styles of character-based CAPTCHA images make the time and manpower required by this method unacceptable, severely limiting its applicability.

[0007] In view of this, the present invention is hereby proposed. Summary of the Invention

[0008] The purpose of this invention is to provide a character-based CAPTCHA security detection method based on a character-level self-attention network model, which can train an accurate character-based CAPTCHA recognition model based on a small number of samples, thereby solving the above-mentioned technical problems existing in the prior art.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] This invention provides a character-level self-attention network model, comprising:

[0011] Stacked multi-scale convolutional backbone networks and character-level self-attention Transformer networks; among them...

[0012] The input of the stacked multi-scale convolutional backbone network is the image of the character verification code that is being recognized. The stacked multi-scale convolutional backbone network can perform preliminary feature extraction on the input character verification code image and output a feature vector, as well as perform downsampling operations.

[0013] The character-level self-attention Transformer network is connected to the output of the stacked multi-scale convolutional backbone network. The character-level self-attention Transformer network consists of an encoder and a decoder. The encoder is used to represent the feature vector output by the stacked convolutional backbone network. The decoder takes the right-shifted character sequence, which is processed by shifting the character sequence corresponding to the identified CAPTCHA image one position to the right, as input. At the same time, based on the output of the encoder, it outputs the predicted probability of the character sequence of the character CAPTCHA image.

[0014] This invention also provides a character-level self-attention network model for character-based CAPTCHA security detection, employing the character-level self-attention network model described in this invention, and includes the following steps:

[0015] Collect 600 to 1000 character-based CAPTCHA images from the target website. Label the character sequence of each image through manual recognition to obtain a training dataset. Then train the character-level self-attention network model until the parameters of the character-level self-attention network model converge, thus completing the training of the character-level self-attention network model.

[0016] Using a pre-trained character-level self-attention network model, character-based CAPTCHA images that require security checks are detected, and the security of the detected character-based CAPTCHA images is determined based on the detection results.

[0017] Compared with existing technologies, the character-based CAPTCHA security detection method based on a character-level self-attention network model provided by this invention has the following advantages:

[0018] By employing a network model composed of a stacked multi-scale convolutional backbone network and a character-level self-attention Transformer network, it can achieve good recognition performance on a small training dataset (referring to a training dataset containing no more than 1,000 character-type CAPTCHA images). In the security detection of character-type CAPTCHA images, it significantly reduces the time and manpower required to build the evaluation method, and effectively solves the defect of existing CAPTCHA security evaluation methods that require the collection and annotation of a large number of CAPTCHA images. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram illustrating the structure of a stacked multi-scale convolutional backbone network based on a character-level self-attention network model, as provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram illustrating the structure of a character-level self-attention Transformer network based on a character-level self-attention network model, as provided in an embodiment of the present invention.

[0022] Figure 3 The T model based on a character-level self-attention network provided in this embodiment of the invention e A schematic diagram of the encoder character-level self-attention calculation process when =2.

[0023] Figure 4 This is a schematic diagram illustrating the generalization performance of a character-level self-attention network model under different numbers of training samples, as provided in an embodiment of the present invention.

[0024] Figure 5 Different perceptual region thresholds T provided for embodiments of the present invention based on a character-level self-attention network model e Corresponding model recognition performance diagram;

[0025] Figure 6 This is a schematic diagram illustrating the CAPTCHA recognition results of EBAY, Sohu, NetEase, and Baidu based on a character-level self-attention network model, as provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the specific content of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, which do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0027] First, the following explanations are provided for the terms that may be used in this article:

[0028] The term "and / or" means that either or both can be achieved simultaneously. For example, X and / or Y means that it includes both "X" or "Y" as well as the three cases of "X and Y".

[0029] The terms “including,” “comprising,” “containing,” “having,” or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, “including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.)” should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0030] The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim.

[0031] Unless otherwise explicitly specified or limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this document according to the specific circumstances.

[0032] When concentration, temperature, pressure, size, or other parameters are expressed as numerical ranges, such ranges should be understood to specifically disclose all ranges formed by any pairing of upper limits, lower limits, or preferred values ​​within that range, regardless of whether the range is explicitly stated; for example, if the numerical range "2 to 8" is stated, then that range should be interpreted to include ranges such as "2 to 7", "2 to 6", "5 to 7", "3 to 4 and 6 to 7", "3 to 5 and 7", "2 and 5 to 7", etc. Unless otherwise stated, the numerical ranges described herein include both their endpoints and all integers and fractions within that range.

[0033] The terms “center,” “longitudinal,” “lateral,” “length,” “width,” “thickness,” “upper,” “lower,” “front,” “back,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” “outer,” “clockwise,” and “counterclockwise” indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience and simplification of description and do not imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this document.

[0034] The following is a detailed description of the character-based CAPTCHA security detection method based on a character-level self-attention network model provided by this invention. Contents not described in detail in the embodiments of this invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, they should be performed according to conventional conditions in the art or conditions recommended by the manufacturer. Reagents or instruments used in the embodiments of this invention, unless otherwise specified by the manufacturer, are all commercially available conventional products.

[0035] like Figure 1 , 2 As shown, this embodiment of the invention provides a character-level self-attention network model, including:

[0036] Stacked multi-scale convolutional backbone networks and character-level self-attention Transformer networks; among them...

[0037] The input of the stacked multi-scale convolutional backbone network is the image of the character verification code that is being recognized. The stacked multi-scale convolutional backbone network can perform preliminary feature extraction on the input character verification code image and output a feature vector, as well as perform downsampling operations.

[0038] The character-level self-attention Transformer network is connected to the output of the stacked multi-scale convolutional backbone network. The character-level self-attention Transformer network consists of an encoder and a decoder. The encoder is used to represent the primary feature vector output by the stacked convolutional backbone network. The decoder takes the right-shifted character sequence, which is processed by shifting the character sequence corresponding to the identified CAPTCHA image one position to the right, as input. At the same time, based on the output of the encoder, it outputs the predicted probability of the character sequence of the character CAPTCHA image.

[0039] See Figure 1 In the above model, the stacked multi-scale convolutional backbone network is a ResNet network composed of multiple stacked convolutional neural networks with multi-scale convolutions.

[0040] See Figure 1 In the above model, the multi-scale convolutional ResNet network includes:

[0041] 1×1 convolution, 5×5 convolution, 3×3 convolution and 2×2 max pooling layers; among which,

[0042] The 1×1 convolution represents mapping short-circuit connections, and the 3×3 and 5×5 convolutions extract character structure features within different neighborhood ranges, respectively.

[0043] The combined input of the 1×1 convolution, 5×5 convolution, and 3×3 convolution is connected as the input of the ResNet network with multi-scale convolution, and the combined output of the 1×1 convolution, 5×5 convolution, and 3×3 convolution is connected to the 2×2 max pooling layer.

[0044] The 2×2 max pooling layer downsamples the convolution output after combining it with a stride of 2 and a window size of 2×2.

[0045] In the ResNet network with multi-scale convolutions described above, the 5×5 convolution consists of two superimposed 3×3 convolutions.

[0046] In the ResNet network with multi-scale convolution mentioned above, convolution refers to a convolutional neural network.

[0047] The ResNet network with the above-described multi-scale convolutional structure, due to the combination of multiple convolutional neural networks, can adapt well to characters with different widths.

[0048] See Figure 2 The encoder of the character-level self-attention Transformer network consists of sequentially connected position encoding modules and multiple stacked encoder character-level self-attention layers, wherein...

[0049] The input of the position encoding module is connected to the output of the stacked multi-scale convolutional backbone network;

[0050] The multi-layered stacked encoder character-level self-attention layer includes:

[0051] The multi-head self-attention layer, character-level masking layer, merging and normalization layer, feedforward network layer, and merging and normalization layer are connected in sequence.

[0052] The decoder of the character-level self-attention Transformer network consists of a character label vector embedding layer, a position encoding module, a single-layer decoder character-level self-attention layer, a multi-layer stacked encoder-decoder self-attention layer, a linear transformation layer, and a softmax layer, connected in sequence.

[0053] The input end of the character label vector embedding layer is used to input the right-shifted character sequence after the character sequence corresponding to the recognized character verification code image is shifted one position to the right.

[0054] The single-layer decoder character-level self-attention layer includes:

[0055] The multi-head self-attention layer, the character-level masking layer, and the merging and normalization layer are connected in sequence.

[0056] The multi-head self-attention layer of the multi-layered stacked encoder-decoder includes:

[0057] The multi-head self-attention layer, the merging and normalization layer, the feedforward network layer, and the merging and normalization layer are connected in sequence.

[0058] The output of the encoder is connected to the input of the self-attention layer of the multi-layer stacked encoder-decoder of the decoder.

[0059] In the above model, the encoder of the character-level self-attention Transformer network represents the primary feature vector output by the stacked convolutional backbone network in the following manner:

[0060] The position encoding module of the encoder embeds position information into the primary feature vector using absolute position encoding, represented by the following formula (1), to obtain a vector f corresponding to different positions i. i The formula (1) is:

[0061]

[0062] In the formula (1), For feature vectors The vector corresponding to position i in the vector; The one-hot encoded vector [0, 0, ..., 0, 1, 0, ... 0] is defined with the i-th element being 1 and all other elements being 0. The weight parameter matrix is ​​used to perform a linear transformation on the one-hot encoded vector; f d h These are the feature length and the number of feature channels corresponding to the feature vector f, respectively;

[0063] The vector f corresponding to different positions i i The features are combined to form the position-encoded feature vector f'.

[0064] The multi-head self-attention layer of the encoder maps the position-encoded feature vector f′ to a query matrix Q, a key matrix K, and a value matrix V through linear transformations, respectively. The transpose of the query matrix Q and the key matrix K is then used as the basis for the mapping. T The correlation matrix R is obtained by multiplying the matrices, and the elements r in this matrix are... i，j The correlation between position i and position j is represented by the following formula (2):

[0065] r i，j =∑(q i ⊙k j (2);

[0066] In formula (2), ⊙ represents element-wise multiplication; q i This indicates the query to the i-th row in matrix Q; k j K represents the transpose of the bond matrix. T The j-th column;

[0067] The multi-head self-attention layer of the encoder uses a positional relationship mask, represented by formula (3), to filter the correlation matrix R. Formula (3) is:

[0068]

[0069] In equation (3), r i，j These are elements of the correlation matrix R; the threshold T e These are hyperparameters that restrict the receptive region in the multi-head self-attention layer of the encoder, causing the model to compute only the feature f at position i. i The feature set {f} within the character-level neighborhood i+Δi |-T e ≤Δi≤T e The correlation of}, the threshold T e The result is obtained by calculation using formula (4):

[0070]

[0071] In equation (4), N = 2 represents the number of stacked modules in the encoder; This represents the upper bound of the pixel width of all characters; This indicates the rounding operation;

[0072] The calculation and update formula for the multi-head self-attention layer of the encoder is as follows:

[0073]

[0074] In formula (5), the Attention function is the overall mapping function of the multi-head self-attention layer; the softmax function is the normalized exponential function; d h It is the number of channels in the feature vector;

[0075] The output of the multi-head self-attention layer of the encoder is processed by merging, normalization, and feedforward network layers to obtain the final encoder output.

[0076] In the encoder structure described above, the multi-head self-attention layer filters the correlation matrix R using a positional relationship mask, thereby transforming the receptive region of the multi-head self-attention layer from a global region to a specified local region. This is because there is only semantic association between adjacent characters in the character verification code. The local receptive region allows the model to learn the correlation characteristics between characters in a targeted manner, thereby improving the model's recognition accuracy and robustness.

[0077] In the above model, the decoder of the character-level self-attention Transformer network takes the right-shifted character sequence (one position to the right of the character sequence corresponding to the identified CAPTCHA image) as input, and simultaneously obtains the predicted probability output of the character sequence of the character-type CAPTCHA image based on the output of the encoder, including:

[0078] The character tag vector embedding layer and position encoding module of the decoder calculate the representation vector t′ of the right-shifted character tags according to formula (6), where formula (6) is:

[0079]

[0080] In formula (6), t i This is the label category number corresponding to the i-th position in the label sequence; The weight parameter matrix is ​​used for the one-hot encoding vector Onehot(t) i A linear transformation of ), where C is the number of label categories; W is the number of label categories. d Onehot(i) is the positional encoding of the tag sequence in the decoder; The weight parameter matrix is ​​used for the linear transformation of the one-hot encoded vector Onehot(i); c The length of the label sequence;

[0081] The multi-head self-attention layer of the decoder maps the representation vector t′ to the query matrix Q, the key matrix K, and the value matrix V through linear transformations, respectively. The multi-head self-attention layer filters the elements of the relevance matrix R using the following mask according to the following formula (7):

[0082]

[0083] Since the decoder ignores the sequence information after the current position, the character-level mask in formula (7) is used to limit the perception area with a range of 1;

[0084] The upper-level self-attention module in the decoder is an encoder-decoder self-attention layer, where the query matrix Q comes from the decoder, and the key matrix K and value matrix V come from the encoder. This self-attention layer uses the traditional self-attention formula for calculation, and the output is transformed by a linear transformation and mapped by a softmax layer to obtain the final character sequence prediction probability output.

[0085] In the above-mentioned decoder structure, the multi-layer self-attention layer filters the elements of the correlation matrix R with a character-level mask, thereby achieving character-level restriction of the perceptual region and improving the model's recognition performance.

[0086] This invention provides a character-level self-attention network model for character-based CAPTCHA security detection, characterized by employing the aforementioned character-level self-attention network model and including the following steps:

[0087] Collect 600 to 1000 character-based CAPTCHA images from the target website (which can be a website that requires verification of the security of character-based CAPTCHAs). Label the character sequence of each image through manual recognition to obtain a training dataset. Then train the character-level self-attention network model until the parameters of the character-level self-attention network model converge, thus completing the training of the character-level self-attention network model.

[0088] Using a pre-trained character-level self-attention network model, character-based CAPTCHA images that require security checks are detected, and the security of the detected character-based CAPTCHA images is determined based on the detection results.

[0089] Specifically, a CAPTCHA is considered successfully recognized only if every single character in the CAPTCHA is correctly identified. Existing research suggests that when the recognition accuracy of a CAPTCHA exceeds 1%, it is relatively vulnerable to attacks by automatic recognition algorithms. In such cases, the CAPTCHA is considered to have limited security and should be updated or replaced promptly.

[0090] In the above method, after the character-level self-attention network model is trained, a testing step is also included, comprising:

[0091] Using a pre-trained character-level self-attention network model, the model is used to identify a test dataset consisting of manually labeled character-type CAPTCHA images. The identification results are then compared with the manually labeled character-type CAPTCHA images in the test dataset to determine the recognition accuracy of the character-level self-attention network model.

[0092] In the above method, the test dataset contains 100 character-type CAPTCHA images.

[0093] In summary, the model and method of this invention, by employing a network model composed of a stacked multi-scale convolutional backbone network and a character-level self-attention Transformer network, can achieve good recognition performance on a small training dataset (referring to a training dataset containing no more than 1,000 character-type CAPTCHA images). In the security detection of character-type CAPTCHA images, it significantly reduces the time and manpower required to build an evaluation method.

[0094] To more clearly demonstrate the technical solution and its effects provided by the present invention, the following detailed description of the character-based self-attention network model-based character CAPTCHA security detection method provided by the present invention will be based on specific embodiments.

[0095] Example 1

[0096] This invention provides a character-level self-attention network model, a deep network model that demonstrates good recognition performance on character-based CAPTCHA images on small training datasets. This model specifically learns the correlation between adjacent characters, achieving a more efficient and robust representation of character sequences, thus exhibiting good recognition performance on small training datasets. It significantly reduces the time and manpower required to build evaluation methods. It effectively solves the problem of existing CAPTCHA security evaluation methods requiring the collection and annotation of a large number of CAPTCHA images.

[0097] like Figure 1 , 2 As shown, this character-level self-attention network model has multiple layered structures. The bottom layer network is a stacked multi-scale convolutional backbone network, which consists of multiple stacked convolutional layers and is used to extract preliminary features and perform downsampling operations on the recognized character-type CAPTCHA image. The upper layer network is a Transformer network based on character-level self-attention, which consists of an encoder and a decoder. The encoder is used to represent the feature vector output by the stacked multi-scale convolutional backbone network, and the decoder is used to predict the classification of the character sequence.

[0098] (1) The structure of stacked multi-scale convolutional backbone networks (see...) Figure 1 ):

[0099] A ResNet network with multi-scale convolutions is used as the backbone of the hierarchical model. 1×1 convolutions represent mapping short-circuit connections, while 3×3 and 5×5 convolutions extract character structure features from different neighborhood ranges. The 5×5 convolution is composed of two stacked 3×3 convolutions, which provides good adaptability to characters with varying widths. The convolutional outputs are downsampled after combination using a max-pooling layer with a stride of 2 and a window size of 2×2. For the input CAPTCHA image... The backbone network reduces its height dimension to 1 dimension through convolution and max pooling, outputting a feature vector. Where w represents the height and width of the input image, respectively. Since the image is a color image, the feature vector has 3 channels; l f d h These represent the width of the output feature vector and the number of feature channels, respectively.

[0100] (2) Structure of character-level self-attention Transformer network (see...) Figure 2 ):

[0101] Includes encoders and decoders;

[0102] The encoder consists of a position encoding module and multiple stacked encoder character-level self-attention layers connected in sequence. The input of the position encoding module is connected to the output of the stacked multi-scale convolutional backbone network.

[0103] The multi-layered stacked encoder character-level self-attention layer includes:

[0104] The multi-head self-attention layer, character-level masking layer, merging and normalization layer, feedforward network layer, and merging and normalization layer are connected in sequence.

[0105] The encoder encodes the feature vector f extracted by the stacked multi-scale convolutional backbone network using absolute position information according to the following formula (1) to obtain the encoded vector f′:

[0106]

[0107] In the formula (1), For feature vectors The vector corresponding to position i in the vector; The one-hot encoded vector [0, 0, ..., 0, 1, 0, ... 0] is defined with the i-th element being 1 and all other elements being 0. The weight parameter matrix is ​​used to perform a linear transformation on the one-hot encoded vector; f d hThese are the feature length and the number of feature channels corresponding to the feature vector f, respectively;

[0108] The vector f corresponding to different positions i i The feature vectors are combined to form the final location-encoded vector f′. Encoding the feature vectors using absolute location information ensures the consistency and continuity of the location information. The feature vector f′ is mapped to a query matrix Q, a key matrix K, and a value matrix V through linear transformations. The query matrix Q and the key matrix K are then used to... T The correlation matrix R is obtained by calculating the matrix product, and the elements r in the correlation matrix R are... i，j The correlation between position i and position j is represented by the following formula (2):

[0109] r i，j =∑(q i ⊙k j (2);

[0110] In formula (2), ⊙ represents element-wise multiplication, and q i This indicates the query to the i-th row in matrix Q; k j Represents the bond matrix K T The j-th column;

[0111] The encoder of this embodiment of the invention filters the elements of the correlation matrix R using a position-related mask according to the following formula (3), thereby transforming the perception range of self-attention from global to a specified local region (see...). Figure 3 ):

[0112]

[0113] In formula (3), the threshold T e These are character-level restrictions on the self-attention perception region, causing the model to compute only the feature f at position i. i The feature set {f} within the character-level neighborhood i+Δi |-T e ≤Δi≤T e The correlation of};

[0114] The threshold T e The result is obtained by calculation using formula (4):

[0115]

[0116] In the formula (3), N is the number of stacked multi-head self-attention layers in the encoder; Defines the upper bound of the pixel width of all characters; This indicates the rounding operation;

[0117] Based on the filtering results obtained above, the self-attention calculation and update function of the encoder is:

[0118]

[0119] The self-attention calculated by the above formula (5) is a character-level self-attention because the perception area of ​​the multi-head self-attention layer is restricted at the character level by the position mask. This allows the network model to learn the correlation between structural features between adjacent characters most effectively, such as the characteristics of sticky strokes and local aliasing between interfering character pairs, and achieves a significant improvement in generalization performance under effective training samples.

[0120] The decoder consists of a character tag vector embedding layer, a position encoding module, a single-layer decoder character-level self-attention layer, a multi-layer stacked encoder-decoder self-attention linear transformation layer, and a softmax layer, connected in sequence.

[0121] The input end of the character label vector embedding layer is used to input the right-shifted character sequence after the character sequence corresponding to the recognized character verification code image is shifted one position to the right.

[0122] The single-layer decoder character-level self-attention layer includes:

[0123] The multi-head self-attention layer, the character-level masking layer, and the merging and normalization layer are connected in sequence.

[0124] The multi-head self-attention layer of the multi-layered stacked encoder-decoder includes:

[0125] The multi-head self-attention layer, the merging and normalization layer, the feedforward network layer, and the merging and normalization layer are connected in sequence.

[0126] The output of the encoder is connected to the input of the self-attention layer of the multi-layer stacked encoder-decoder of the decoder.

[0127] The character label vector embedding layer and position encoding module of the decoder take the right-shifted character sequence obtained by shifting the character sequence corresponding to the identified character-type verification code image one bit to the right as input, and calculate the representation vector t′ corresponding to the verification code image through the following formula (6):

[0128]

[0129] In formula (6), W t Onehot(t i ) is; t i This is the label category number corresponding to the i-th position in the label sequence; The weight parameter matrix is ​​used for the one-hot encoding vector Onehot(t) iA linear transformation of ), where C is the number of label categories; W is the number of label categories. d Onehot(i) is the positional encoding of the tag sequence in the decoder; The weight parameter matrix is ​​used for the linear transformation of the one-hot encoded vector Onehot(i); c The length of the label sequence;

[0130] The single-layer decoder character-level self-attention layer maps the representation vector t′ to the query matrix Q, key matrix K, and value matrix V respectively through linear transformation. The single-layer decoder character-level self-attention layer filters the elements of the relevance matrix R with the following mask according to the following formula (7):

[0131]

[0132] The upper-level self-attention module in this decoder is a multi-layered stacked encoder-decoder self-attention layer, where the query matrix Q comes from the decoder, and the key matrix K and value matrix V come from the encoder. The output of this multi-layered stacked encoder-decoder self-attention layer, calculated using the traditional self-attention formula, is transformed by a linear transformation and mapped by a softmax layer to obtain the final character sequence prediction probability output.

[0133] The advantages of this invention are: the model incorporates a character-level self-attention mechanism in the encoder and decoder of the Transformer network. This mechanism determines the local perceptual range of self-attention based on the character width in the CAPTCHA image, enabling the model to specifically learn the correlation between adjacent characters, achieving a more efficient and robust representation of character sequences. It exhibits good generalization performance on small training datasets, significantly reducing the time and manpower required for CAPTCHA recognition. In practical deployments, the CAPTCHA recognition method based on this model only requires 600-1000 labeled images to effectively recognize the character CAPTCHAs in the corresponding character-based CAPTCHA images. This low-cost, high-precision recognition method can more objectively and accurately evaluate the security performance of character-based CAPTCHAs used by websites under the latest recognition technologies.

[0134] Example 2

[0135] This embodiment provides a method for detecting the security of character-based CAPTCHA images using a character-level self-attention network model from this invention, thereby achieving a more objective and accurate assessment of the security performance of character-based CAPTCHAs used by websites. The method includes the following steps:

[0136] (1) Select the size of the training dataset according to the security standards. If the security requirements of the website are high, it is recommended to collect 1100 character verification code images of the website (1000 of which are used for training and 100 for testing). Otherwise, you can collect only 700 character verification code images (600 of which are used for training and 100 for testing).

[0137] (2) Manually annotate the character sequences in the character-type verification code images (it is estimated that it will take 1 to 2 hours for one person to annotate);

[0138] (3) Build using computers or servers with high-performance computing capabilities (such as computers configured with GPUs). Figure 1 The neural network model shown in this embodiment of the invention is a character-level self-attention network model;

[0139] (4) Train the model using the training dataset. After the model parameters converge, use the test set to evaluate the recognition accuracy of the model. If the recognition accuracy reaches 1%, it is confirmed that the character verification code image of the website does not meet the security requirements.

[0140] For example, the method of this invention can be used to evaluate the CAPTCHAs of Baidu, NetEase, Sohu, and eBay, and the recognition accuracy results are as follows:

[0141]

[0142] The following experiments were conducted to verify the safety and generalization performance of the model of this invention:

[0143] Experiment 1: Performance evaluation of the benchmark dataset:

[0144] This experiment uses Sina Weibo's CAPTCHA as the benchmark dataset to evaluate the model's recognition performance under different training sample numbers. Sina Weibo's CAPTCHA possesses multiple security features, including distortion, rotation, character adhesion, and complex backgrounds. The CAPTCHA contains 25 classes of English characters, and the adhered characters form various interference character pairs, making it difficult for previous recognition methods to effectively identify them, thus limiting their performance.

[0145] The benchmark dataset contains 55,000 labeled CAPTCHA images. The experiment first randomly selects 5,000 images as a fixed validation set. Then, 200, 400, 600, 800, 1,000, 2,000, 4,000, 6,000, 8,000, 10,000, 20,000, and 50,000 images are used as the training dataset. The character accuracy and sequence accuracy of the network on the validation set images are evaluated sequentially with different training sample sizes. Experimental results... Figure 4 As shown.

[0146] Analysis of the experimental results reveals that when the number of training samples is small, the model's generalization performance changes significantly with the increase in the number of samples: when the model is trained using only 200 images, the accuracy of recognizing a single character is 0.1558, and the accuracy of recognizing sequences approaches zero. It is important to note that the CAPTCHA contains 25 different classes of English characters, with 4 characters per image. In the 200 training samples, each character appears only an average of 16 times. Considering the interference of factors such as distortion, deformation, character aliasing, and background noise, deep learning models often struggle to effectively learn relevant features. However, the method proposed in this patent achieves a character recognition accuracy far exceeding that of random prediction (0.1558 > 0.04), indicating that it has learned relatively effective character representations even with a small sample size.

[0147] When the training sample size increased to 400 images, the model's character accuracy rapidly improved to 0.6974, and the corresponding sequence recognition rate increased to 0.2739, significantly exceeding the widely accepted standard for CAPTCHA recognition tasks. This means that when the sequence accuracy is greater than 0.01, the attack algorithm can crack the CAPTCHA. When the training sample size was further increased to 600, 800, and 1,000 images, the sequence accuracy improved to 0.7322, 0.8152, and 0.8521, respectively. At this point, the recognition method based on the proposed model only required 1-2 attempts to crack the Sina CAPTCHA. Experimental data shows that when the training sample size exceeds 4,000 images, the model's generalization performance tends to saturate, and the recognition accuracy increases slowly with the increase in the number of samples.

[0148] Experiment 2: Threshold T of the sensing region e Regarding recognition performance:

[0149] In Experiment 2, the input image has a height h = 40 and a width w = 100. After convolution and downsampling operations in the backbone network, the length l of the feature vector is... f =25. Among the 25 categories of English characters included in Sina's verification code, the character with the largest horizontal width is 'W'. Considering the effects of rotation and deformation, the upper bound of the width of this character is estimated to be... Substituting into formula (4), we can obtain T. e ≈l f / 4, the model ultimately chose T. e =6 is used as the threshold for the receptive region. To verify the effectiveness of the self-attention mechanism based on character-level receptive regions in this chapter, this section uses different receptive region thresholds T. e The experiment used 600 images as the training dataset, and the recognition performance on the test set was as follows: Figure 5 As shown.

[0150] Among them, l f / 20 corresponds to T e=1, meaning the neighborhood contains only a single feature. In this case, the sequence recognition rate is 0.3077. The self-attention layer in the encoder is limited by the size of the perceptual range and can only calculate a limited correlation within a local region. As T... e As T increases, the perceptual range of the self-attention layer gradually expands. e Approaching l f At / 4, the model's character recognition rate and sequence recognition rate reach their maximum, and at this point, the perception range corresponds to the character-level region size, proving the effectiveness of the model structure.

[0151] When T e As the number of steps increases, the recognition rate begins to decline, especially when T... e =l f The model employs a global self-attention mechanism, meaning that features in the encoder are correlated with all other features. This phenomenon may be because the character-level receptive region allows the encoder to most effectively learn the correlations between structural features of adjacent characters, such as the overlapping strokes and local aliasing between interfering character pairs. As the receptive region expands further, the model fits the correlations between features at a distance during training, even though these features have no semantic connection, leading to a decrease in the model's generalization performance.

[0152] Experiment 3: Multi-type CAPTCHA Recognition

[0153] Experiment 3 further validates the comprehensive recognition performance of the character-level self-attention mechanism-based network model using character-based CAPTCHAs from various real-world websites. The CAPTCHA dataset includes images from four websites: eBay, NetEase, Baidu, and Sohu, with 8500 labeled samples in each category. eBay CAPTCHAs exhibit characters with larger rotation angles and some characters are highly overlapping; Sohu CAPTCHAs have line noise running throughout the image, connecting all characters into connected regions; Baidu and NetEase CAPTCHAs show significant random distortion of characters, resulting in severe character stroke overlap. These CAPTCHAs all contain difficult-to-distinguish interference character pairs, such as '4'-'1', 'N'-'M', 'u'-'b', etc. Testing the model's recognition performance on different datasets effectively evaluates its robustness to different security features.

[0154] Similar to previous experiments, 500 images were randomly selected as a fixed test set. Then, 600, 1,000, 2,000, 4,000, and 8,000 images were used as training datasets to evaluate the model's character recognition accuracy and sequence recognition accuracy on the test set images. The experimental results are as follows: Figure 6 As shown.

[0155] It can be observed that, using only 600 training samples, the model of this invention achieved a sequence recognition rate of over 65% on eBay, Sohu, and NetEase CAPTCHAs. When the number of training samples increased to 1,000, the sequence recognition rate for Baidu CAPTCHAs rapidly increased from 30.4% to 77.3%. Experimental results demonstrate that the character-level self-attention-based network model proposed in this invention can effectively recognize CAPTCHAs with different fonts and structures even with a small number of samples, exhibiting good robustness.

[0156] In practical deployments, the CAPTCHA recognition method based on the model proposed in this chapter only requires collecting and annotating 600-1,000 images to effectively crack the website's CAPTCHA. Taking Sina's CAPTCHA as an example, the training dataset consisting of 1,000 annotated samples requires approximately 3-4 hours of time, and the model's sequence recognition rate is 85.2%. This low-cost, high-precision recognition method indicates that websites need to reassess their current security practices.

[0157] In summary, the evaluation results of the detection method in this embodiment of the invention show that Baidu's character-based CAPTCHA image has better security performance compared to the character-based CAPTCHA images of the other three websites. However, even with 1000 training samples, the model still has a relatively high probability of successfully cracking Baidu's CAPTCHA in 1-2 recognition attempts. Based on the detection results, it is necessary for each website to update and improve the style of its CAPTCHA to ensure its security performance.

[0158] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A character-level self-attention network model, characterized in that, include: Stacked multi-scale convolutional backbone networks and character-level self-attention Transformer networks; among them... The input of the stacked multi-scale convolutional backbone network is the image of the character verification code that is being recognized. The stacked multi-scale convolutional backbone network can perform preliminary feature extraction on the input character verification code image and output a feature vector, as well as perform downsampling operations. The character-level self-attention Transformer network is connected to the output of the stacked multi-scale convolutional backbone network. The character-level self-attention Transformer network consists of an encoder and a decoder. The encoder is used to represent the primary feature vector output by the stacked convolutional backbone network. The decoder takes the right-shifted character sequence, which is processed by shifting the character sequence corresponding to the identified character CAPTCHA image one position to the right, as input. At the same time, based on the output of the encoder, it outputs the predicted probability of the character sequence of the character CAPTCHA image. The stacked multi-scale convolutional backbone network is a ResNet network consisting of multiple stacked convolutional neural networks with multi-scale convolutions. The encoder of the character-level self-attention Transformer network consists of sequentially connected position encoding modules and multiple stacked encoder character-level self-attention layers, wherein... The input of the position encoding module is connected to the output of the stacked multi-scale convolutional backbone network; The multi-layered stacked encoder character-level self-attention layer includes: The multi-head self-attention layer, character-level masking layer, merging and normalization layer, feedforward network layer, and merging and normalization layer are connected in sequence. The decoder of the character-level self-attention Transformer network consists of a character label vector embedding layer, a position encoding module, a single-layer decoder character-level self-attention layer, a multi-layer stacked encoder-decoder self-attention layer, a linear transformation layer, and a softmax layer, connected in sequence. The input end of the character label vector embedding layer is used to input the right-shifted character sequence after the character sequence corresponding to the recognized character verification code image is shifted one position to the right. The single-layer decoder character-level self-attention layer includes: The multi-head self-attention layer, the character-level masking layer, and the merging and normalization layer are connected in sequence. The multi-head self-attention layer of the multi-layered stacked encoder-decoder includes: The multi-head self-attention layer, the merging and normalization layer, the feedforward network layer, and the merging and normalization layer are connected in sequence. The output of the encoder is connected to the input of the self-attention layer of the multi-layer stacked encoder-decoder of the decoder.

2. The character-level self-attention network model according to claim 1, characterized in that, The multi-scale convolutional ResNet network includes: 1×1 convolution, 5×5 convolution, 3×3 convolution and 2×2 max pooling layers; among which, The 1×1 convolution represents mapping short-circuit connections, and the 3×3 and 5×5 convolutions extract character structure features within different neighborhood ranges, respectively. The combined input of the 1×1 convolution, 5×5 convolution, and 3×3 convolution is connected as the input of the ResNet network with multi-scale convolution, and the combined output of the 1×1 convolution, 5×5 convolution, and 3×3 convolution is connected to the 2×2 max pooling layer. The 2×2 max pooling layer downsamples the convolution output after combining it with a stride of 2 and a window size of 2×2.

3. The character-level self-attention network model according to claim 2, characterized in that, The 5×5 convolution consists of two superimposed 3×3 convolutions.

4. The character-level self-attention network model according to any one of claims 1 to 3, characterized in that, The encoder of the character-level self-attention Transformer network represents the primary feature vector output by the stacked convolutional backbone network in the following manner: The position encoding module of the encoder embeds position information into the primary feature vector using absolute position encoding represented by the following formula (1) to obtain the position information corresponding to different positions. vector The formula (1) is: (1)； In the formula (1), For feature vectors The middle corresponds to the position ; For the first A one-hot encoded vector with one element being 1 and the rest being 0. ; This is the weight parameter matrix, used to perform linear transformation on the one-hot encoded vector; , They are the feature vectors The corresponding feature length and number of feature channels; will correspond to different positions vector Combined into a position-encoded feature vector ; The multi-head self-attention layer of the encoder encodes the feature vector after the position. Each is mapped to a query matrix through a linear transformation. Key matrix Sum matrix Using the query matrix The transpose of the bond matrix and the bond matrix The correlation matrix is ​​obtained by calculating the matrix product. The elements in this matrix Indicates position With position The correlation between them is calculated using the following formula (2): (2)； In the formula (2), Represents element-wise product; Representing the query matrix The first in OK; The transpose of the key matrix The List; The multi-head self-attention layer of the encoder uses the positional relationship mask represented by formula (3) on the correlation matrix. For filtering, formula (3) is: (3)； In the aforementioned formula (3), It is a correlation matrix Element; threshold These are hyperparameters that limit the perceptual region in the multi-head self-attention layer of the encoder, allowing the model to compute only the position. Features of the place Feature set within the character-level neighborhood The correlation, the threshold The result is obtained by formula (4): (4)； In the aforementioned formula (4), Indicates the number of stacked modules in the encoder; This represents the upper bound of the pixel width of all characters; This indicates the rounding operation; The calculation and update formula for the multi-head self-attention layer of the encoder is as follows: (5)； In the formula (5), the Attention function is the overall mapping function of the multi-head self-attention layer; the softmax function is the normalized exponential function. It is the number of channels in the feature vector; The output of the multi-head self-attention layer of the encoder is processed by merging, normalization, and feedforward network layers to obtain the final encoder output.

5. The character-level self-attention network model according to claim 4, characterized in that, The decoder of the character-level self-attention Transformer network takes a right-shifted character sequence (one position to the right) of the character sequence corresponding to the identified character CAPTCHA image as input, and simultaneously, based on the encoder's output, derives a predicted probability output for the character sequence of the character CAPTCHA image, including: The character tag vector embedding layer and position encoding module of the decoder calculate the representation vector of the right-shifted character tag according to formula (6). The formula (6) is: (6)； In the formula (6), For the first tag in the sequence The tag category number corresponding to each position; The weight parameter matrix is ​​used for one-hot encoding vectors. Linear transformation; The number of tag categories; This is the position encoding of the tag sequence in the decoder; The weight parameter matrix is ​​used for one-hot encoding vectors. Linear transformation; The length of the label sequence; The multi-head self-attention layer of the decoder will use the representation vector Each is mapped to a query matrix through a linear transformation. Key matrix Sum matrix Using a query matrix Device matrix of the bond matrix The correlation matrix is ​​obtained by calculating the matrix product. The correlation matrix elements in Indicates position With position The correlation between the two is determined by the multi-head self-attention layer using the following formula (7) and the following mask on the correlation matrix. Filter the elements: (7) In the multi-layered stacked encoder-decoder self-attention layer of the decoder, the key matrix Sum matrix From the encoder, query matrix The output of the multi-layered encoder-decoder, derived from the attention layer, is linearly transformed by the linear transformation layer and then passed to the softmax layer to obtain the final symbol sequence prediction probability output.

6. A character-level self-attention network model for character-based CAPTCHA security detection, characterized in that, The method employs the character-level self-attention network model according to any one of claims 1 to 5, comprising the following steps: Collect 600 to 1000 character-based CAPTCHA images from the target website. Label the character sequence of each image through manual recognition to obtain a training dataset. Then train the character-level self-attention network model until the parameters of the character-level self-attention network model converge, thus completing the training of the character-level self-attention network model. Using a pre-trained character-level self-attention network model, character-based CAPTCHA images that require security checks are detected, and the security of the detected character-based CAPTCHA images is determined based on the detection results.

7. The character-based CAPTCHA security detection method based on a character-level self-attention network model according to claim 6, characterized in that, The training dataset contains 600 to 1000 character-based CAPTCHA images.

8. The character-based CAPTCHA security detection method based on a character-level self-attention network model according to claim 6 or 7, characterized in that, After the character-level self-attention network model is trained, a testing step is also included, including: Using a pre-trained character-level self-attention network model, the model is used to identify a test dataset consisting of manually labeled character-type CAPTCHA images. The identification results are then compared with the manually labeled character-type CAPTCHA images in the test dataset to determine the recognition accuracy of the character-level self-attention network model.

9. The character-based CAPTCHA security detection method based on a character-level self-attention network model according to claim 8, characterized in that, The test dataset contains 100 character-based CAPTCHA images.

Images