Verification code identification method and device based on large model, and related equipment
By training multimodal features based on a large model and optimizing the three-tower model, the problem of adapting CAPTCHA verification technology to diverse CAPTCHAs was solved, achieving improved accuracy and efficiency in cross-type recognition while reducing costs and risks.
Patent Information
- Application Number
- CN202511105371.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-07
- Publication Date
- 2025-11-21
AI Technical Summary
Existing CAPTCHA verification technologies are difficult to adapt to diverse CAPTCHA formats, resulting in high development costs, high maintenance complexity, and difficulty in dealing with attacks from malicious actors. Furthermore, traditional models lack generalization ability and cannot transfer learning across different types.
A large model-based approach is adopted to acquire multimodal features (visual, audio, and text) and train a three-tower model. The model parameters are optimized using a triplet loss function to construct a cross-type CAPTCHA recognition framework, including image, voice, and text CAPTCHAs.
It improves the recognition accuracy and success rate of cross-type CAPTCHAs, reduces development and maintenance costs, enhances adaptability to new types of CAPTCHAs, and reduces the risk of attacks by malicious actors.
Smart Images
Figure HDA0005538192540000011 
Figure HDA0005538192540000012
Abstract
Description
TECHNICAL FIELD
[0001] Embodiments of the present application relate to the technical field of information verification, and in particular to a verification code recognition method and device based on a large model and related equipment. BACKGROUND
[0002] In the digital era, as the core security mechanism for distinguishing between human and machine operations, the form of verification code has evolved from early character recognition to diversified interactive forms, covering dozens of types such as sliding puzzle, target clicking, image-text question answering, and behavior trajectory analysis. Different platforms often choose differentiated verification code schemes based on the balance between security needs and user experience: for example, financial websites often use high-precision sliders + biological trajectory verification, e-commerce platforms prefer to click on goods or semantic question answering, and lightweight applications may use simple image-text matching mode.
[0003] However, the current mainstream verification code verification technology still has significant adaptation limitations and is difficult to cope with this diversity challenge. Traditional schemes mostly use the "rule hard coding" mode: for each type of verification code, manually define feature extraction rules (such as the edge contour of the slider, the target coordinate range of the click) and verification logic (such as the drag speed threshold, the click accuracy rate standard). This mode needs to completely reconstruct the algorithm module when facing new types or variants of verification code - for example, when a platform changes the background image of the slider verification code from static to dynamic blur, or changes the target of the click from "click the car" to "click the inverted animal", the original model will have a sharp drop in accuracy due to the invalidation of the feature rules.
[0004] At the same time, the generalization ability defects of existing technologies further exacerbate the adaptation difficulty. Models trained based on a single task (such as algorithms that can only recognize the slider of a specific website) cannot transfer the common features of different types of verification codes (such as the difference in behavior dynamics between human and machine operations), resulting in the need for enterprises to deploy independent verification systems for each type of verification code, which not only increases development costs and maintenance complexity, but also makes it difficult to cope with the rapid iteration of black production attack methods (such as automated cracking tools for specific types of verification code).
[0005] Under this background, how to break through the dependence of traditional technology on specific types of verification code and build a general verification framework with cross-type adaptation ability has become a key issue for improving the efficiency of digital security protection. SUMMARY
[0006] Embodiments of the present application provide a verification code recognition method and device based on a large model and related equipment, aiming to solve the technical problem that traditional verification code verification technology is difficult to adapt to the diversity of current verification codes.
[0007] In a first aspect, embodiments of the present application provide a verification code recognition method based on a large model, comprising:
[0008] Obtain different types of sample verification codes, including image type verification codes, voice type verification codes and text type verification codes;
[0009] Preprocess different types of sample verification codes, and extract multi-modal features of the sample verification codes, including visual features, audio features and text features;
[0010] Obtain verification strategies and interference features of each sample verification code, and input each multi-modal feature and corresponding verification strategy and interference feature into an initial multi-modal large model for identification training, based on an output result of the initial multi-modal large model and a corresponding true verification code type identifier, loss calculation is performed according to a triplet loss function, a model loss is obtained, and the model loss is back propagated to optimize model parameters of the multi-modal large model, and an optimized multi-modal large model is obtained, the output result is a type identifier of the verification code, and the interference feature at least includes distortion of the verification code, background noise points and voice noise;
[0011] Obtain a target verification code required for current identification, and output the target verification code to the multi-modal large model for identification to obtain an identification result.
[0012] In a second aspect, an embodiment of the present application provides a verification code identification device based on a large model, which comprises:
[0013] The obtaining module is configured to obtain different types of sample verification codes, including image type verification codes, voice type verification codes and text type verification codes;
[0014] The preprocessing module is configured to preprocess different types of sample verification codes, and extract multi-modal features of the sample verification codes, including visual features, audio features and text features;
[0015] The model training module is configured to obtain verification strategies and interference features of each sample verification code, and input each multi-modal feature and corresponding verification strategy and interference feature into an initial multi-modal large model for identification training, based on an output result of the initial multi-modal large model and a corresponding true verification code type identifier, loss calculation is performed according to a triplet loss function, a model loss is obtained, and the model loss is back propagated to optimize model parameters of the multi-modal large model, and an optimized multi-modal large model is obtained, the output result is a type identifier of the verification code, and the interference feature at least includes distortion of the verification code, background noise points and voice noise;
[0016] The identification module is configured to obtain a target verification code required for current identification, and output the target verification code to the multi-modal large model for identification to obtain an identification result.
[0017] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the CAPTCHA recognition method based on a large model as described in the first aspect.
[0018] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the large-model-based CAPTCHA recognition method described in the first aspect.
[0019] This invention provides a method, apparatus, and related equipment for CAPTCHA recognition based on a large-scale model. The method acquires CAPTCHA samples of different types; preprocesses these samples to extract multimodal features; acquires the verification strategy and interference features for each CAPTCHA sample, and inputs each multimodal feature along with its corresponding verification strategy and interference features into an initial multimodal large-scale model for training. Based on the output of the initial multimodal large-scale model and the corresponding real CAPTCHA type identifier, a loss is calculated using a triplet loss function to obtain the model loss. Backpropagation is then performed based on the model loss to optimize the model parameters, resulting in an optimized multimodal large-scale model. Finally, the target CAPTCHA to be recognized is acquired, and the target CAPTCHA is output to the multimodal large-scale model for recognition, yielding the recognition result. This method trains a multimodal large-scale model based on different types of CAPTCHA samples, and improves the accuracy and success rate of CAPTCHA recognition across different types by using the multimodal large-scale model. Attached Figure Description
[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 A flowchart illustrating an embodiment of the CAPTCHA recognition method based on a large model provided by the present invention;
[0022] Figure 2 This is a schematic block diagram of a CAPTCHA recognition device based on a large model, provided in an embodiment of the present invention. Detailed Implementation
[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0025] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0026] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0027] Please see Figure 1 This is a flowchart illustrating a large-model-based CAPTCHA recognition method provided in an embodiment of the present invention. The method includes steps S110 to S140.
[0028] Step S110: Obtain sample verification codes of different types, including image verification codes, voice verification codes, and text verification codes;
[0029] In this embodiment, for login scenarios on mainstream global websites (e-commerce, finance, social media, etc.), a headless browser (such as Puppeteer) is used to simulate user interactions to trigger various CAPTCHAs. CAPTCHA loading is triggered by simulating login failures, IP switching, and other behaviors, and the complete interaction process (such as the initial position of the slider and the coordinate range of the selected target) is recorded. CAPTCHA types include graphic CAPTCHAs (containing character distortion, interference lines, and noise), slider CAPTCHAs (puzzle / gap matching), click CAPTCHAs (text / image target clicks), voice CAPTCHAs (containing noise / dialects), behavioral CAPTCHAs (mouse trajectory / operation habits), and SMS CAPTCHAs (dynamic characters), etc.
[0030] Step S120: Preprocess the sample verification codes of different types and extract the multimodal features of the sample verification codes, including visual features, audio features and text features;
[0031] In this embodiment, different types of sample CAPTCHAs are preprocessed. For image-based CAPTCHAs (such as graphic / slider / point-based CAPTCHAs), which require image recognition, a screenshot of the CAPTCHA area is taken, and then visual features (edge contours, color distribution, and texture features) are extracted. Image segmentation techniques are used to separate the target area (such as slider notches, point-based text, etc.) from interference elements (such as background noise in image CAPTCHAs, distortion in text CAPTCHAs, etc.) to obtain visual features. For voice-based CAPTCHAs, audio sampling is performed on the voice, and spectral features (Mel-frequency cepstral coefficients, MFCC) and duration features are extracted. Environmental noise is processed using noise reduction algorithms (such as spectral subtraction) to obtain audio features. For text / behavioral CAPTCHAs (such as SMS CAPTCHAs), the character sequence of SMS CAPTCHAs and the mouse trajectory coordinate sequence of behavioral CAPTCHAs are structured and encoded to generate text features.
[0032] Step S130: Obtain the verification strategy and interference features of each sample CAPTCHA, and input each multimodal feature and the corresponding verification strategy and interference features into the initial multimodal large model for recognition training. Based on the output result of the initial multimodal large model and the corresponding real CAPTCHA type identifier, calculate the loss according to the triplet loss function to obtain the model loss, and perform backpropagation based on the model loss to optimize the model parameters of the multimodal large model to obtain the optimized multimodal large model. The output result is the CAPTCHA type identifier.
[0033] In this embodiment, a multimodal large model with a three-tower model structure is used as the base. This model supports cross-modal input of text, images, and audio, and can uniformly handle the heterogeneous features of various CAPTCHAs. Combined with a dynamic routing mechanism, computational resources are automatically allocated for different types of CAPTCHAs (e.g., image CAPTCHAs emphasize the visual encoder, while voice CAPTCHAs emphasize the audio encoder). The three-tower model structure is a model structure used in recommendation systems and other fields. It adds a tower for feature crossing to the two-tower model, aiming to improve the model's expressive power while balancing model complexity and inference latency to a certain extent. The multimodal large model with a three-tower model structure includes a modality encoding tower, an interaction logic tower, and an adversarial feature tower. Specifically: the multimodal features corresponding to each sample CAPTCHA are input into the modality encoding tower for encoding to obtain the corresponding modality semantic vector, which includes visual semantic vector, audio semantic vector, and text semantic vector; the verification strategy corresponding to each sample CAPTCHA is input into the interaction logic tower for encoding to obtain the rule semantic vector; the interference features corresponding to each sample CAPTCHA are input into the adversarial feature tower for encoding to obtain the anti-interference vector; finally, the triplet loss function maximizes the feature similarity of similar CAPTCHAs (such as slider CAPTCHAs from different websites) and minimizes cross-type differences (such as slider and point-based CAPTCHAs). The triplet loss function constructs triplet samples of "anchor (a) -- positive example (p) -- negative example (n)," where:
[0034] Anchor point a: Randomly select a sample (such as the slider verification code of website 1 in type A), the corresponding rule semantic vector of the verification strategy, and the anti-interference vector of the interference feature;
[0035] Positive example p: The rule semantic vector and the anti-interference vector of the interference features of the verification strategy corresponding to other samples of the same type as the anchor (such as the slider CAPTCHA of website 2 in type A, ensuring "different websites of the same type");
[0036] Negative example n: The rule semantic vector and the anti-interference vector of the interference features of the verification strategy corresponding to the cross-type sample of the anchor point (such as the click verification code of website 1 in type B).
[0037] The formula for the triplet loss function L is as follows:
[0038] L(a,p,n)=max(0,d(a,p) 2 -d(a,n) 2 +m arg in),
[0039] Here, margin represents the difference, which is a custom value, such as 0.5; d() represents the Euclidean distance between two feature vectors. The smaller the Euclidean distance, the higher the similarity.
[0040] The parameters of the feature extractor are updated through backpropagation, minimizing the loss function L. The specific results are as follows:
[0041] When d(a,p) 2 -d(a,n) 2 When +m arg in > 0 (i.e., the distance between positive examples is greater than that between negative examples), the loss L is positive. The model will adjust the parameters to decrease d(a,p) (to increase similarity within the same class) and increase d(a,n) (to expand the difference across classes).
[0042] When d(a,p) 2 -d(a,n) 2 When +m arg in≤0 (i.e., the distance to positive examples is smaller than that to negative examples, and the distance to negative examples is larger), the loss L=0, and the model does not need to be adjusted.
[0043] In this embodiment, the verification strategies differ depending on the type of CAPTCHA. Currently, character / numeric CAPTCHAs often display distorted characters with superimposed interference lines / noise (letters, numbers, Chinese characters, etc.), and the corresponding verification strategy is to select the corresponding character or select characters according to the verification rules in sequence. Image-text selection CAPTCHAs require selecting a specified target (e.g., "click all cars," "click the upside-down animal," "click the area containing traffic lights"), and some involve the order of selection. Image puzzle / slider CAPTCHAs often display cut images (e.g., slider puzzles, rotating images), and the verification strategy is for the user to drag the slider to put the pieces back in place or rotate the image to the correct angle. There are also text-based question-and-answer CAPTCHAs, which typically pose simple mathematical calculations, semantic questions, or common sense questions (e.g., "1+5=?", "Is 'apple' a fruit or an appliance?", "Please write the abbreviation for 'Shanghai'," etc.), requiring the user to input the correct answer, and so on.
[0044] Step S140: Obtain the target verification code to be identified, and output the target verification code to the multimodal large model for identification to obtain the identification result.
[0045] In this embodiment, the target CAPTCHA to be identified is input into a pre-trained multimodal large-scale model. The feature similarity between the current CAPTCHA and historical samples is calculated, and the CAPTCHA type is determined based on the feature similarity. A corresponding verification strategy is then determined based on the CAPTCHA type, and the verification strategy is displayed to the user, guiding them to verify according to the strategy or to use the appropriate verification tool. For example:
[0046] For image-based CAPTCHAs:
[0047] By extracting multimodal features from the graphic verification code: OCR recognizes character regions (combined with visual features to filter out interference lines), and generates a set of character candidates (such as the fuzzy recognition of "8" and "B");
[0048] Semantic verification: The multimodal large model is used to reason about the character context logic (such as the length of the verification code, whether it contains special symbols, and whether the characters are distorted) and output the optimal character sequence (confidence ≥ 95%).
[0049] For slider CAPTCHAs:
[0050] Gap detection: Compare the visual feature differences (such as edge gradient changes) between the slider background image and the target image to locate the gap coordinates (error ≤ 2px);
[0051] Trajectory generation: Based on the temporal characteristics of historical successful trajectories (acceleration 0.1-0.3 m / s²). 2 (Maximum speed ≤ 1.2m / s) to generate a sliding path that conforms to human operating habits;
[0052] For clicking the verification code:
[0053] Target detection: The model identifies the semantic features of the target to be clicked (such as "Please click all animals") and locates the coordinates of the target area;
[0054] Sequence planning: If clicks need to be made in a logical order (e.g., "click in order of numbers 1-5"), generate a click sequence based on the text semantics;
[0055] For voice verification codes:
[0056] Speech-to-text: The model's audio encoder is called to convert speech into a character sequence, and the recognition error is corrected by the dialect / noise adaptation module (e.g., "si" is corrected to "4").
[0057] For behavioral verification codes:
[0058] Trajectory simulation: Learn the behavioral characteristics of normal user operations (such as the curvature of mouse movement and the number of pauses) and generate interaction trajectories that conform to the platform's judgment criteria;
[0059] Regarding SMS verification codes:
[0060] Interface adaptation: Read SMS content through the authorized interface and extract a 6-8 digit / letter sequence (combining regular expression matching and semantic filtering of spam messages).
[0061] This method acquires CAPTCHA samples of different types; preprocesses these samples to extract multimodal features; acquires the verification strategy and interference features for each CAPTCHA sample, and inputs each multimodal feature along with its corresponding verification strategy and interference features into an initial multimodal large-scale model for recognition training. Based on the output of the initial multimodal large-scale model and the corresponding real CAPTCHA type identifier, loss is calculated using a triplet loss function to obtain the model loss. Backpropagation is then performed based on the model loss to optimize the model parameters of the multimodal large-scale model, resulting in an optimized model. Finally, the target CAPTCHA to be recognized is acquired, and the target CAPTCHA is output to the multimodal large-scale model for recognition, yielding the recognition result. This method trains a multimodal large-scale model based on different types of CAPTCHA samples and uses the multimodal large-scale model to recognize CAPTCHAs, improving the accuracy and success rate of cross-type CAPTCHA recognition.
[0062] This invention also provides a large-model-based CAPTCHA recognition device, which is used to execute any of the aforementioned embodiments of the large-model-based CAPTCHA recognition method. Specifically, please refer to... Figure 2 , Figure 2 This is a schematic block diagram of a large-scale CAPTCHA recognition device 100 provided in an embodiment of the present invention. The large-scale CAPTCHA recognition device 100 can be configured in a server.
[0063] like Figure 2 As shown, the CAPTCHA recognition device 100 based on a large model includes an acquisition module 110, a preprocessing module 120, a model training module 130, and a recognition module 140.
[0064] The acquisition module 110 is used to acquire different types of sample verification codes, including image verification codes, voice verification codes and text verification codes;
[0065] The preprocessing module 120 is used to preprocess the sample verification codes of different types and extract the multimodal features of the sample verification codes, including visual features, audio features and text features;
[0066] The model training module 130 is used to acquire the verification strategy and interference features of each sample CAPTCHA, and input each multimodal feature and the corresponding verification strategy and interference features into an initial multimodal large model for recognition training. Based on the output result of the initial multimodal large model and the corresponding real CAPTCHA type identifier, the loss is calculated according to the triplet loss function to obtain the model loss, and backpropagation is performed according to the model loss to optimize the model parameters of the multimodal large model to obtain an optimized multimodal large model. The output result is the CAPTCHA type identifier.
[0067] The recognition module 140 is used to obtain the target verification code that needs to be recognized, and output the target verification code to the multimodal large model for recognition to obtain the recognition result.
[0068] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the large-model-based CAPTCHA recognition method described above.
[0069] In another embodiment of the invention, a computer-readable storage medium is provided. This computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the large-model-based CAPTCHA recognition method as described above.
[0070] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.
[0071] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Units with the same function may be grouped into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, or it may be an electrical, mechanical, or other form of connection.
[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.
[0073] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0074] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks.
[0075] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A CAPTCHA recognition method based on a large model, characterized in that, include: Obtain different types of sample verification codes, including image verification codes, voice verification codes, and text verification codes; The sample CAPTCHAs of different types are preprocessed to extract multimodal features of the sample CAPTCHAs, including visual features, audio features and text features; The verification strategy and interference features of each sample CAPTCHA are obtained, and each multimodal feature and its corresponding verification strategy and interference features are input into an initial multimodal large model for recognition training. Based on the output of the initial multimodal large model and the corresponding real CAPTCHA type identifier, the loss is calculated according to the triplet loss function to obtain the model loss. Backpropagation is performed based on the model loss to optimize the model parameters of the multimodal large model to obtain an optimized multimodal large model. The output is the CAPTCHA type identifier, and the interference features include at least CAPTCHA distortion, background noise, and speech noise. Obtain the target verification code that needs to be identified, and output the target verification code to the multimodal large model for identification to obtain the identification result.
2. The CAPTCHA recognition method based on a large model as described in claim 1, characterized in that, The extraction of multimodal features from the sample CAPTCHA includes: Visual features are extracted from the image-type CAPTCHA to obtain the visual features corresponding to the image-type CAPTCHA. The audio of the voice-based CAPTCHA is sampled to obtain the audio features corresponding to the voice-based CAPTCHA; The text-based CAPTCHA is structured and encoded to obtain the text features corresponding to the text-based CAPTCHA.
3. The CAPTCHA recognition method based on a large model as described in claim 1, characterized in that, The initial multimodal large model is a three-tower model structure, including a modality coding tower, an interaction logic tower, and an adversarial feature tower. The step of inputting each multimodal feature, along with its corresponding verification strategy and interference features, into the initial multimodal large model for recognition training includes: The multimodal features corresponding to each sample verification code are input into the modality coding tower for encoding to obtain the corresponding modality semantic vector, which includes visual semantic vector, audio semantic vector and text semantic vector; The verification strategy corresponding to each sample verification code is input into the interaction logic tower for encoding to obtain a rule semantic vector. The interference features corresponding to each sample CAPTCHA are input into the adversarial feature tower for encoding to obtain the anti-interference vector.
4. The CAPTCHA recognition method based on a large model as described in claim 1, characterized in that, The step of outputting the target verification code into the multimodal large model for recognition to obtain the recognition result includes: Based on the recognition results, a corresponding target verification strategy is determined, and the corresponding tool is invoked to perform CAPTCHA verification according to the target verification strategy.
5. A CAPTCHA recognition device based on a large model, characterized in that, include: The acquisition module is used to acquire different types of sample verification codes, including image verification codes, voice verification codes, and text verification codes; The preprocessing module is used to preprocess the sample CAPTCHAs of different types and extract the multimodal features of the sample CAPTCHAs, including visual features, audio features and text features; The model training module is used to acquire the verification strategy and interference features of each sample CAPTCHA, and input each multimodal feature and the corresponding verification strategy and interference features into an initial multimodal large model for recognition training. Based on the output of the initial multimodal large model and the corresponding real CAPTCHA type identifier, the loss is calculated according to the triplet loss function to obtain the model loss, and backpropagation is performed based on the model loss to optimize the model parameters of the multimodal large model to obtain an optimized multimodal large model. The output is the CAPTCHA type identifier, and the interference features include at least the distortion of the CAPTCHA, background noise, and speech noise. The recognition module is used to obtain the target verification code that needs to be recognized, and output the target verification code to the multimodal large model for recognition to obtain the recognition result.
6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the CAPTCHA recognition method based on a large model as described in any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the CAPTCHA recognition method based on a large model as described in any one of claims 1 to 4.