An anti-crawler method based on TLS fingerprint and JS obfuscation encryption

Abstract

This invention relates to the field of electronic data processing technology, and in particular to an anti-crawler method based on TLS fingerprinting and JS obfuscation encryption, comprising: step S1, constructing a TLS fingerprint blacklist; step S2, performing TLS fingerprint verification on the client; step S3, performing cookie verification on clients that pass TLS verification according to flag2; step S4, calculating the waiting time ΔT according to flag2; step S5, comparing the waiting time ΔT with a preset waiting time ΔT0, judging the validity of flag2 based on the comparison result, and correcting the cookie verification result based on the judgment result; step S6, obtaining param1 of clients whose cookie verification fails; step S7, verifying param1 and outputting the param1 verification result. This invention improves website data security.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic data processing, and in particular to an anti-crawler method based on TLS fingerprint and JS obfuscation encryption. Background Art

[0002] Web crawler technology is widely used to automatically collect information on the Internet, which is crucial for data analysis, market research, content aggregation, price comparison services and other fields. However, a large number of crawler accesses will consume the server resources of the website, affecting the access speed and experience of normal users, and sometimes even causing the server to crash. Some sensitive information or user data may be leaked due to improper access by crawlers, threatening personal privacy and data security. Traditional anti-crawling strategies based on static rules are difficult to effectively identify and prevent crawler behavior of automated browser tools such as Selenium and Puppeteer.

[0003] China Patent Publication No.: CN115396213A discloses a deep anti-crawling method, system, device and medium based on TLS and webrtc, the method comprising: obtaining a TCP handshake request carrying user information; generating a corresponding TLS fingerprint according to the TCP handshake request; comparing the TLS fingerprint and user information with the redis fingerprint library to determine whether it is a malicious access, if so, denying access; returning a JavaScript file for detecting the environment according to the TCP handshake request, obtaining the canvas information, font information, audio fingerprint and instant messaging address returned by the user; determining whether the instant messaging address is consistent with the IP information of the TCP handshake request, if not, denying access; obtaining the access frequency of the request that is completely consistent with the canvas information, font information, audio fingerprint or instant messaging address, comparing the access frequency with the preset access frequency, and denying access if the access frequency is greater than the preset access frequency. However, the invention does not establish a TLS fingerprint blacklist, and does not perform JS encryption, so the anti-crawling security is low, and it is easy to be cracked by crawlers, making it difficult to improve the security of the website. Summary of the invention

[0004] To this end, the present invention provides an anti-crawler method based on TLS fingerprint and JS obfuscation encryption, so as to overcome the problem in the prior art that websites are easily cracked by crawlers and the security of the websites cannot be improved.

[0005] To achieve the above object, the present invention provides an anti-crawler method based on TLS fingerprint and JS obfuscation encryption, comprising:

[0006] Step S1, constructing a TLS fingerprint blacklist;

[0007] Step S2, performing TLS fingerprint verification on the client according to the TLS fingerprint blacklist, and outputting the TLS fingerprint verification result;

[0008] Step S3, perform cookie verification on the client that has passed TLS verification according to flag2, and judge the cookie verification result;

[0009] Step S4, calculating the waiting time ΔT according to flag2;

[0010] Step S5, comparing the waiting time ΔT with the preset waiting time ΔT0, judging the validity of flag2 according to the comparison result, and correcting the cookie verification result according to the judgment result;

[0011] Step S6, obtaining param1 of the client that fails cookie verification;

[0012] Step S7, verify param1 and output the param1 verification result.

[0013] Further, in step S1, the method for constructing a TLS fingerprint blacklist includes:

[0014] Step S10, obtaining the data packet generated during the handshake between the client and TLS;

[0015] Step S11, extracting features from the data packet through a convolutional neural network model to obtain a feature data packet;

[0016] Step S12, analyzing the characteristic data packet to obtain a JA4 fingerprint;

[0017] Step S13, judging the JA4 fingerprint, and constructing a TLS fingerprint blacklist according to the judgment result.

[0018] Furthermore, in the step S10, the data packets in the handshake process between the client and TLS are obtained through a network packet capture tool; in the step S11, when the feature extraction of the data packet is performed through the convolutional neural network model, the preset data packet is divided into a 70% training set and a 30% test set, the training set is input into the convolutional neural network model for training, and the test set is input into the trained convolutional neural network model, and the parameters in the convolutional neural network model are optimized and iterated until the accuracy of the output result of the test set of the convolutional neural network model reaches 98%, the convolutional neural network model is output, and the data packet is input into the convolutional neural network model, and the convolutional neural network model is used to extract features of the data packet to obtain a feature data packet; in the step S12, the feature data packet is analyzed by the JA3 algorithm to obtain a JA4 fingerprint.

[0019] Furthermore, in step S13, when judging the JA4 fingerprint, the JA4 fingerprint is compared with the preset JA4 fingerprint in the fingerprint database, the JA4 fingerprint is judged according to the comparison result, and a TLS fingerprint blacklist is constructed according to the judgment result, wherein:

[0020] When there is a preset JA4 fingerprint that is consistent with the JA4 fingerprint in the fingerprint database, the JA4 fingerprint is determined to be a normal fingerprint and is not added to the TLS fingerprint blacklist;

[0021] When there is no preset JA4 fingerprint consistent with the JA4 fingerprint in the fingerprint database, the JA4 fingerprint is determined to be an abnormal fingerprint and the fingerprint is added to the TLS fingerprint blacklist.

[0022] Further, in step S2, when performing TLS fingerprint verification on the client according to the TLS fingerprint blacklist, the client fingerprint is compared with the TLS fingerprint blacklist, and the TLS fingerprint verification result is output according to the comparison result, wherein:

[0023] When the JA4 fingerprint that is consistent with the client fingerprint exists in the TLS fingerprint blacklist, the TLS fingerprint verification result is determined to be TLS fingerprint verification failure, and a 403 error code is returned to the client;

[0024] When there is no JA4 fingerprint that is consistent with the client fingerprint in the TLS fingerprint blacklist, the TLS fingerprint verification result is determined to be TLS fingerprint verification passed, and the output is a client that has passed TLS fingerprint verification.

[0025] Further, in step S3, when performing cookie verification on the client request that passes TLS fingerprint verification according to flag2, flag2 is decrypted by the key, and the cookie verification result is judged according to the decryption result, wherein:

[0026] When the key successfully decrypts flag2, the cookie verification is determined to be successful, and the decrypted flag2 is output;

[0027] When the key fails to decrypt flag2, the cookie verification is determined to have failed, and the output is the client that failed the cookie verification.

[0028] Furthermore, in step S4, when the waiting time ΔT is calculated according to flag2, the decrypted Unix timestamp T of flag2 is calculated with the current timestamp T0 to obtain the waiting time ΔT, and ΔT=T0-T is set.

[0029] Furthermore, in step S5, the waiting time ΔT is compared with the preset waiting time ΔT0, the validity of flag2 is judged according to the comparison result, and the result of the cookie verification is corrected according to the judgment result, wherein:

[0030] When ΔT≤ΔT0, flag2 is determined to be valid, the result of cookie verification is not modified, and the target page is returned to the client;

[0031] When ΔT>ΔT0, flag2 is determined to be invalid, and the cookie verification pass is corrected to the cookie verification fail.

[0032] Furthermore, in step S6, when obtaining param1 of the client that fails the cookie verification, flag1 is used as the AES key, and the current timestamp timestamp is encrypted by flag1 to obtain param1.

[0033] Furthermore, in step S7, when param1 is verified, the server decrypts param1 using a preset AES key, verifies param1 according to the decryption result, and outputs it, wherein:

[0034] When the preset AES key successfully decrypts param1, it is determined that param1 verification has passed, flag2 is encrypted using the preset AES key, and the target page is returned to the client;

[0035] When the preset AES key fails to decrypt param1, param1 is determined to have failed verification and a 403 error code is returned to the client.

[0036] Compared with the prior art, the beneficial effect of the present invention is that the method constructs a TLS fingerprint blacklist through step S1, and performs TLS fingerprint verification on the client according to the TLS fingerprint blacklist through step S2, and outputs the TLS fingerprint verification result, so as to identify and reject the client with known security vulnerabilities and malicious behaviors, thereby improving the security of the website, the method performs cookie verification on the client that passes the TLS verification according to flag2 through step S3, and judges the cookie verification result, and further performs security verification on the client, thereby further improving the security of the website, the method calculates the waiting time ΔT according to flag2 through step S4, and compares the waiting time ΔT with the preset waiting time ΔT0 through step S5, judges the validity of flag2 according to the comparison result, and corrects the cookie verification result according to the judgment result, so as to adjust the security policy according to the actual situation, thereby improving the flexibility and adaptability of the security mechanism, the method obtains param1 of the client that fails the cookie verification through step S6, and verifies param1 through step S7, and outputs the param1 verification result, so as to improve the security policy and analyze potential threats, thereby further improving the security of the website. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of the anti-crawler method based on TLS fingerprint and JS obfuscation encryption in this embodiment;

[0038] Figure 2 The following is a flow chart of the method for constructing a TLS fingerprint blacklist in this embodiment. DETAILED DESCRIPTION

[0039] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0040] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0041] See also Figure 1 As shown, it is a flow chart of the anti-crawler method based on TLS fingerprint and JS obfuscation encryption in this example, and the method includes:

[0042] Step S1, constructing a TLS fingerprint blacklist;

[0043] Step S2, performing TLS fingerprint verification on the client according to the TLS fingerprint blacklist, and outputting the TLS fingerprint verification result;

[0044] Step S3, perform cookie verification on the client that has passed TLS verification according to flag2, and judge the cookie verification result;

[0045] Step S4, calculating the waiting time ΔT according to flag2;

[0046] Step S5, comparing the waiting time ΔT with the preset waiting time ΔT0, judging the validity of flag2 according to the comparison result, and correcting the cookie verification result according to the judgment result;

[0047] Step S6, obtaining param1 of the client that fails cookie verification;

[0048] Step S7, verify param1 and output the param1 verification result.

[0049] Specifically, the method is applied to a network platform, and a multi-level and dynamic security protection mechanism is constructed by constructing a TLS fingerprint blacklist, TLS fingerprint verification, cookie verification, waiting time analysis and param1 verification, thereby improving the security and protection capabilities of the website. The method constructs a TLS fingerprint blacklist through step S1, and performs TLS fingerprint verification on the client according to the TLS fingerprint blacklist through step S2, and outputs the TLS fingerprint verification result to identify and reject clients with known security vulnerabilities and malicious behaviors, thereby improving the security of the website. The method performs cookie verification on the client that passes TLS verification according to flag2 through step S3, and judges the cookie verification result. The client is further security-verified, thereby further improving the security of the website. The method calculates the waiting time ΔT according to flag2 through step S4, and compares the waiting time ΔT with the preset waiting time ΔT0 through step S5. The validity of flag2 is judged according to the comparison result, and the cookie verification result is corrected according to the judgment result, so as to adjust the security policy according to the actual situation, thereby improving the flexibility and adaptability of the security mechanism. The method obtains param1 of the client that fails the cookie verification through step S6, and verifies param1 through step S7, and outputs the param1 verification result, so as to improve the security policy and analyze potential threats, thereby further improving the security of the website.

[0050] Specifically, in step S1, the TLS fingerprint blacklist refers to a JA4 fingerprint database containing TLS clients that are considered to have potential risks, abnormal behaviors, and illegal activities.

[0051] See also Figure 2 As shown, it is a flow chart of the method for constructing a TLS fingerprint blacklist in this example, and the method includes:

[0052] Step S10, obtaining the data packet generated during the handshake between the client and TLS;

[0053] Step S11, extracting features from the data packet through a convolutional neural network model to obtain a feature data packet;

[0054] Step S12, analyzing the characteristic data packet to obtain a JA4 fingerprint;

[0055] Step S13, judging the JA4 fingerprint, and constructing a TLS fingerprint blacklist according to the judgment result.

[0056] Specifically, the method for constructing a TLS fingerprint blacklist collects data packets generated during the handshake between the client and TLS, uses a convolutional neural network model to extract features, and then obtains JA4 fingerprints, and constructs a TLS fingerprint blacklist based on the judgment results, thereby improving the automation and accuracy of security protection.

[0057] Specifically, in step S10, a data packet generated during the handshake between the client and TLS is obtained through a network packet capture tool.

[0058] Specifically, the network packet capture tool refers to a software tool used to capture and analyze network traffic. This embodiment does not limit the type of network packet capture tool. Relevant technical personnel in this field can freely set it according to actual needs. It only needs to meet the need to obtain the data packets generated during the handshake process between the client and the TLS. For example, the network packet capture tool can be set to Wi reshark. The client refers to the party that initiates a connection request during the TLS handshake process, such as a web browser and a mobile application. The data packet refers to the data unit transmitted over the network between the client and the server when establishing a secure communication channel.

[0059] Specifically, in step S11, when extracting features from a data packet through a convolutional neural network model, the preset data packet is divided into a 70% training set and a 30% test set, the training set is input into the convolutional neural network model for training, and the test set is input into the trained convolutional neural network model, the parameters in the convolutional neural network model are optimized and iterated until the accuracy of the output result of the test set of the convolutional neural network model reaches 98%, the convolutional neural network model is output, and the data packet is input into the convolutional neural network model, and the convolutional neural network model extracts features from the data packet to obtain a feature data packet.

[0060] Specifically, the convolutional neural network model refers to a deep learning neural network model used to process data with a grid-like topological structure, the preset data packet refers to a data packet preset for training the convolutional neural network model, the training set refers to a data set used to train the convolutional neural network model, and the test set refers to a data set used to evaluate the performance of the convolutional neural network model.

[0061] Specifically, in step S12, the characteristic data packet is analyzed by using the JA3 algorithm to obtain a JA4 fingerprint.

[0062] Specifically, the JA3 algorithm refers to a method of generating a JA4 fingerprint by analyzing fields in a feature data packet, and the JA4 fingerprint refers to a string identifier used to identify and track a client.

[0063] Specifically, in step S13, when judging the JA4 fingerprint, the JA4 fingerprint is compared with the preset JA4 fingerprint in the fingerprint database, the JA4 fingerprint is judged according to the comparison result, and a TLS fingerprint blacklist is constructed according to the judgment result, wherein:

[0064] When there is a preset JA4 fingerprint that is consistent with the JA4 fingerprint in the fingerprint database, the JA4 fingerprint is determined to be a normal fingerprint and is not added to the TLS fingerprint blacklist;

[0065] When there is no preset JA4 fingerprint consistent with the JA4 fingerprint in the fingerprint database, the JA4 fingerprint is determined to be an abnormal fingerprint and the fingerprint is added to the TLS fingerprint blacklist.

[0066] Specifically, the fingerprint database refers to a database that stores fingerprint information of network protocols, the preset JA4 fingerprint refers to a pre-set and stored authorized JA4 fingerprint, the normal fingerprint refers to an authorized JA4 fingerprint, and the abnormal fingerprint refers to malware, crawlers and unauthorized JA4 fingerprints.

[0067] Specifically, in step S2, when performing TLS fingerprint verification on the client according to the TLS fingerprint blacklist, the client fingerprint is compared with the TLS fingerprint blacklist, and the TLS fingerprint verification result is output according to the comparison result, wherein:

[0068] When the JA4 fingerprint that is consistent with the client fingerprint exists in the TLS fingerprint blacklist, the TLS fingerprint verification result is determined to be TLS fingerprint verification failure, and a 403 error code is returned to the client;

[0069] When there is no JA4 fingerprint that is consistent with the client fingerprint in the TLS fingerprint blacklist, the TLS fingerprint verification result is determined to be TLS fingerprint verification passed, and the output is a client that has passed TLS fingerprint verification.

[0070] Specifically, the client fingerprint refers to a string identifier used to identify and distinguish different clients, and the 403 error code refers to a status code in the HTTP protocol, indicating that the server understands the client's request but refuses to execute the request.

[0071] Specifically, in step S3, when performing cookie verification on the client request that passes TLS fingerprint verification according to flag2, flag2 is decrypted by the key, and the cookie verification result is judged according to the decryption result, wherein:

[0072] When the key successfully decrypts flag2, the cookie verification is determined to be successful, and the decrypted flag2 is output;

[0073] When the key fails to decrypt flag2, the cookie verification is determined to have failed, and the output is the client that failed the cookie verification.

[0074] Specifically, the flag2 refers to the encrypted Unix timestamp. For example, the Unix timestamp of the date 2024-05-2816:22:49222 is expressed as 1716884569222, which is encrypted as U2FsdGVkX1948cTSufxtl6i5NBesGAKUu6V21lHhwzU=. The Unix timestamp refers to the number of seconds that have passed since midnight on January 1, 1970. The cookie verification refers to the technology used for user identity authentication and session management. The target page refers to the page that the user expects to visit. The key refers to the key used for encryption. and decryption parameters, the decryption refers to the process of using the key to restore the flag2 encrypted by the encryption algorithm before, the decrypted flag2 refers to the flag2 decrypted by the key, the key successfully decrypts flag2 means that the data obtained after the key decrypts flag2 is not damaged in format, length and content, the key fails to decrypt flag2 means that the data obtained after the key decrypts flag2 is damaged in format, length and content, the undamaged data refers to complete, accurate and readable data, and the damaged data refers to incomplete, inaccurate and unreadable data.

[0075] Specifically, in step S4, when calculating the waiting time ΔT according to flag2, the decrypted Unix timestamp T of flag2 and the current timestamp T0 are calculated to obtain the waiting time ΔT, and ΔT=T0-T is set.

[0076] Specifically, the current timestamp T0 refers to the total number of seconds from 00:00:00 on January 1, 1970 to the time when the waiting time ΔT is calculated.

[0077] Specifically, in step S5, the waiting time ΔT is compared with the preset waiting time ΔT0, the validity of flag2 is judged according to the comparison result, and the result of the cookie verification is corrected according to the judgment result, wherein:

[0078] When ΔT≤ΔT0, flag2 is determined to be valid, the result of cookie verification is not modified, and the target page is returned to the client;

[0079] When ΔT>ΔT0, flag2 is determined to be invalid, and the cookie verification pass is corrected to the cookie verification fail.

[0080] Specifically, the preset waiting time ΔT0 refers to a time interval preset to reflect the validity of flag2. This embodiment does not limit the value of the preset waiting time ΔT0. Relevant technical personnel in this field can freely set it according to actual needs. It only needs to meet the requirement of reflecting the validity of flag2. For example, the preset waiting time ΔT0 can be set to 120s.

[0081] Specifically, in step S6, when obtaining param1 of the client that fails the cookie verification, flag1 is used as the AES key, and the current timestamp timestamp is encrypted by flag1 to obtain param1.

[0082] Specifically, flag1 refers to a randomly generated 16-bit string containing numbers and letters, the AES key refers to a set of secret information used for the AES encryption algorithm for encrypting and decrypting data, the current timestamp timestamp refers to the current date and time, and param1 refers to the result obtained by encrypting the current timestamp timestamp using the AES key (flag1).

[0083] Specifically, in step S7, when param1 is verified, the server decrypts param1 using a preset AES key, verifies param1 according to the decryption result, and outputs it, wherein:

[0084] When the preset AES key successfully decrypts param1, it is determined that param1 verification has passed, flag2 is encrypted using the preset AES key, and the target page is returned to the client;

[0085] When the preset AES key fails to decrypt param1, param1 is determined to have failed verification and a 403 error code is returned to the client.

[0086] Specifically, the preset AES key refers to an AES key preset for verifying param1. This embodiment does not limit the specific content of the preset AES key. Relevant technical personnel in this field can freely set it according to actual needs, and only need to meet the needs of verifying param1. For example, the preset AES key can be set to flag1. The preset AES key successfully decrypts param1, which means that the data obtained after the preset AES key decrypts param1 is not damaged in format, length and content. The preset AES key fails to decrypt param1, which means that the data obtained after the preset AES key decrypts param1 is damaged in format, length and content.

[0087] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An anti-crawler method based on TLS fingerprint and JS obfuscation encryption, characterized in that: include: Step S1, constructing a TLS fingerprint blacklist; Step S2, performing TLS fingerprint verification on the client according to the TLS fingerprint blacklist, and outputting the TLS fingerprint verification result; Step S3, performing cookie verification on the client that has passed TLS verification according to flag2, and judging the cookie verification result; the flag2 refers to the encrypted Unix timestamp; Step S4, calculating the waiting time ΔT according to flag2; Step S5, comparing the waiting time ΔT with the preset waiting time ΔT0, judging the validity of flag2 according to the comparison result, and correcting the cookie verification result according to the judgment result; Step S6, obtaining param1 of the client that fails cookie verification; Step S7, verify param1 and output the param1 verification result; In step S3, when performing cookie verification on the client request that passes TLS fingerprint verification according to flag2, flag2 is decrypted by the key, and the cookie verification result is judged according to the decryption result, wherein: When the key successfully decrypts flag2, the cookie verification is determined to be successful, and the decrypted flag2 is output; When the key fails to decrypt flag2, the cookie verification is determined to have failed, and the output is the client that failed the cookie verification; In step S4, when calculating the waiting time ΔT according to flag2, the decrypted Unix timestamp T of flag2 and the current timestamp T0 are calculated to obtain the waiting time ΔT, and ΔT=T0-T is set; In step S5, the waiting time ΔT is compared with the preset waiting time ΔT0, the validity of flag2 is judged according to the comparison result, and the result of the cookie verification is corrected according to the judgment result, wherein: When ΔT≤ΔT0, flag2 is determined to be valid, the result of cookie verification is not modified, and the target page is returned to the client; When ΔT>ΔT0, flag2 is determined to be invalid, and the cookie verification is corrected to cookie verification failure; In the step S6, when obtaining param1 of the client that fails the cookie verification, flag1 is used as the AES key, and the current date and time data timestamp is encrypted by flag1 to obtain param1; flag1 refers to a randomly generated 16-bit string containing numbers and letters; In step S7, when param1 is verified, the server decrypts param1 using a preset AES key, verifies param1 according to the decryption result, and outputs it, wherein: When the preset AES key successfully decrypts param1, it is determined that param1 verification has passed, flag2 is encrypted using the preset AES key, and the target page is returned to the client; When the preset AES key fails to decrypt param1, param1 is determined to have failed verification and a 403 error code is returned to the client.

2. The anti-crawler method based on TLS fingerprint and JS obfuscation encryption according to claim 1 is characterized in that: In step S1, the method for constructing a TLS fingerprint blacklist includes: Step S10, obtaining the data packet generated during the handshake between the client and TLS; Step S11, extracting features from the data packet through a convolutional neural network model to obtain a feature data packet; Step S12, analyzing the characteristic data packet to obtain a JA4 fingerprint; Step S13, judging the JA4 fingerprint, and constructing a TLS fingerprint blacklist according to the judgment result.

3. The anti-crawler method based on TLS fingerprint and JS obfuscation encryption according to claim 2 is characterized in that: In the step S10, a data packet in the handshake process between the client and TLS is obtained through a network packet capture tool; in the step S11, when extracting features from the data packet through the convolutional neural network model, the preset data packet is divided into a 70% training set and a 30% test set, the training set is input into the convolutional neural network model for training, and the test set is input into the trained convolutional neural network model, the parameters in the convolutional neural network model are optimized and iterated until the accuracy of the output result of the test set of the convolutional neural network model reaches 98%, the convolutional neural network model is output, and the data packet is input into the convolutional neural network model, and the convolutional neural network model is used to extract features from the data packet to obtain a feature data packet; in the step S12, the feature data packet is analyzed by the JA3 algorithm to obtain a JA4 fingerprint.

4. The anti-crawler method based on TLS fingerprint and JS obfuscation encryption according to claim 2 is characterized in that: In step S13, when judging the JA4 fingerprint, the JA4 fingerprint is compared with the preset JA4 fingerprint in the fingerprint database, the JA4 fingerprint is judged according to the comparison result, and a TLS fingerprint blacklist is constructed according to the judgment result, wherein: When there is a preset JA4 fingerprint that is consistent with the JA4 fingerprint in the fingerprint database, the JA4 fingerprint is determined to be a normal fingerprint and is not added to the TLS fingerprint blacklist; When there is no preset JA4 fingerprint consistent with the JA4 fingerprint in the fingerprint database, the JA4 fingerprint is determined to be an abnormal fingerprint and the fingerprint is added to the TLS fingerprint blacklist.

5. The anti-crawler method based on TLS fingerprint and JS obfuscation encryption according to claim 1 is characterized in that: In step S2, when performing TLS fingerprint verification on the client according to the TLS fingerprint blacklist, the client fingerprint is compared with the TLS fingerprint blacklist, and the TLS fingerprint verification result is output according to the comparison result, wherein: When the JA4 fingerprint that is consistent with the client fingerprint exists in the TLS fingerprint blacklist, the TLS fingerprint verification result is determined to be TLS fingerprint verification failure, and a 403 error code is returned to the client; When there is no JA4 fingerprint that is consistent with the client fingerprint in the TLS fingerprint blacklist, the TLS fingerprint verification result is determined to be TLS fingerprint verification passed, and the output is a client that has passed TLS fingerprint verification.

Images