Method, system, apparatus, and computer readable storage medium for preventing crawlers

Abstract

Embodiments of the present application provide a method, system, device and computer readable storage medium for preventing crawlers. The method is applied to server side and client side. The method of server side divides the text to be displayed into multiple segments, changes the order of the multiple segments in the original text, organizes the multiple segments according to the changed order to obtain a web page, at this time, the corresponding content of the web page is already disordered text, and finally sends the encrypted order mapping information and the web page to the client side. The method of client side receives the ciphertext of the order mapping information and the web page, then renders and outputs the web page to the display screen through the browser, and then corrects the disordered text in the web page displayed on the display screen according to the order mapping information, so as to obtain a web page containing correct text. Since the crawler system generally only crawls the initial web page, the purpose of preventing the crawler is achieved.

Description

Technical Field

[0001] This invention relates to the Internet field, and more specifically to a method, system, apparatus, and computer-readable storage medium for preventing web crawlers. Background Technology

[0002] Web crawlers are becoming increasingly rampant, frequently scraping exclusive resources from many websites and causing losses. Examples include price information from e-commerce websites and original articles from portal websites.

[0003] To protect these resources from being scraped in bulk by programs, the following three methods are typically used to increase the difficulty of data scraping and prevent crawlers from accessing them to the greatest extent possible:

[0004] The first method is to convert the text into an image and display the image in the browser to prevent web crawlers from directly grabbing the text;

[0005] The second method is to add a CAPTCHA when accessing specific articles, requiring manual input before the content can be viewed;

[0006] The third method is to retrieve data from the backend service API and then dynamically render the webpage using JavaScript.

[0007] However, each of the three methods for preventing web scraping has its drawbacks. The first method, converting text to images, consumes significant server resources, and image transmission also consumes considerable network bandwidth, increasing user wait time. The text style (font, size, etc.) needs to be determined on the backend during conversion, which cannot be adjusted on the frontend using CSS (Cascading Style Sheets), making it inflexible. Text in images can be identified and cracked using OCR (Optical Character Recognition) technology. The second method uses CAPTCHA verification, requiring users to enter a CAPTCHA every time they read the content, resulting in a poor user experience. While the third method dynamically renders the webpage using JavaScript, web scrapers can analyze the backend service API and directly extract data from it; if the backend API is not encrypted, scraping is even simpler. Summary of the Invention

[0008] In view of this, embodiments of the present invention provide a method, system, apparatus, and computer-readable storage medium for preventing web crawlers, in order to solve the problems existing in the prior art.

[0009] In a first aspect, embodiments of the present invention provide a method for preventing web crawlers, applied on a server side, the method comprising:

[0010] Divide the text to be displayed into multiple text segments;

[0011] The order of the multiple text segments is changed, and the order mapping information is recorded. The order mapping information is used to represent the correspondence between the order of the multiple text segments in the text to be displayed before and after the change.

[0012] Encrypt the sequence mapping information;

[0013] The multiple text segments are organized according to the changed order to obtain a web page; and

[0014] Send the encrypted web page and the sequence mapping information.

[0015] In some embodiments, the sequence mapping information is encrypted using a symmetric or asymmetric algorithm.

[0016] In some embodiments, organizing the multiple text segments to obtain a web page includes:

[0017] The multiple text segments are organized according to specific tags to obtain a web page.

[0018] In some embodiments, organizing the multiple text segments to obtain a web page includes:

[0019] In the web page, each of the multiple text segments is marked with a specific HTML tag.

[0020] In some embodiments, the method further includes: encrypting the specific tag and sending the ciphertext.

[0021] Secondly, embodiments of the present invention provide a method for preventing web crawlers, applied to a client-side application, the method comprising:

[0022] Receive encrypted sequential mapping information and a web page;

[0023] Render and output web pages in the browser;

[0024] Decrypt the ciphertext of the sequence mapping information; and

[0025] The order of multiple text segments in the web page is corrected based on the order mapping information.

[0026] In some embodiments, the method further includes: receiving and decrypting the ciphertext of the specific tag, wherein correcting the order of multiple text segments in the web page according to the order mapping information includes: locating the specific tag in the web page, and correcting the text content corresponding to the specific tag according to the order mapping information.

[0027] In some embodiments, correcting the order of multiple text segments in the web page according to the order mapping information includes: locating specific HTML tags in the web page, and correcting the text content corresponding to the specific HTML tags according to the order mapping information.

[0028] In some embodiments, when a web page is rendered and output in a browser, the multiple text segments contained therein are hidden, and when the order of the multiple text segments in the web page is corrected according to the order mapping information, the corrected multiple text segments are displayed.

[0029] In some embodiments, when a web page is rendered and output in a browser, after the multiple text segments contained therein are hidden, specific content is output at the positions of the multiple text segments, and when the order of the multiple text segments in the web page is corrected according to the order mapping information, the specific content is replaced with the corrected multiple text segments.

[0030] Thirdly, embodiments of the present invention provide a system for preventing web crawlers, applied on a server side, the system comprising:

[0031] The segmentation module is used to divide the text to be displayed into multiple segments.

[0032] The disorder module is used to change the sorting of the multiple text segments and record the order mapping information. The order mapping information is used to represent the correspondence between the sorting of the multiple text segments in the text to be displayed before and after the change.

[0033] An encryption module is used to encrypt the sequence mapping information;

[0034] Create a module that organizes the multiple text segments according to the changed sorting to obtain a web page; and

[0035] The sending module is used to send the encrypted web page and the sequence mapping information.

[0036] In some embodiments, the creation module includes:

[0037] In the web page, the multiple text segments are organized according to specific tags to obtain the web page.

[0038] In some embodiments, the creation module includes:

[0039] In the web page, each of the multiple text segments is marked with a specific HTML tag.

[0040] Fourthly, embodiments of the present invention provide a system for preventing web crawlers, applied to a client, the system comprising:

[0041] The receiving module is used to receive ciphertext of web pages and sequence mapping information;

[0042] The decryption module is used to decrypt the ciphertext of the sequence mapping information;

[0043] A correction module is used to correct the order of multiple text segments in the web page according to the order mapping information;

[0044] The rendering module is used to render and output web pages in the browser.

[0045] In some embodiments, the rendering module hides the multiple text segments contained in the web page, and the correction module displays the corrected multiple text segments when correcting the order of the multiple text segments in the web page according to the order mapping information.

[0046] In some embodiments, the rendering module hides the multiple text segments contained in the web page and outputs specific content at the positions of the multiple text segments. When the correction module corrects the sorting of the multiple text segments in the web page according to the order mapping information, it replaces the specific content with the corrected multiple text segments.

[0047] Fifthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions, which, when executed, implement the method for preventing web crawling provided in the first aspect.

[0048] Sixthly, embodiments of the present invention provide a device for preventing crawling, comprising:

[0049] Memory, used to store computer instructions;

[0050] A processor, coupled to the memory, is configured to execute computer instructions stored in the memory to implement the anti-crawler method described in the first aspect above.

[0051] In a seventh aspect, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that, when executed, implement the method for preventing web crawling provided in the second aspect above.

[0052] Eighthly, embodiments of the present invention provide a device for preventing crawling, comprising:

[0053] Memory, used to store computer instructions;

[0054] A processor, coupled to the memory, is configured to execute the anti-crawler method described in the second aspect above based on computer instructions stored in the memory.

[0055] The embodiments of the present invention have the following advantages or beneficial effects: through the cooperation of the client and the server, a web page containing disordered text is first output on the display screen, and then the front-end script corrects the order of multiple text segments in the web page according to the order mapping information, thereby ensuring that the crawler system cannot crawl the correct content. Attached Figure Description

[0056] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0057] Figure 1 The diagram shown is a structural diagram of a web system deployment.

[0058] Figures 2a-2b The flowcharts shown are respectively the client-side and server-side methods for preventing web crawlers according to embodiments of the present invention;

[0059] Figure 3 The diagram shows a schematic of the existing browser rendering process.

[0060] Figure 4 The image shows an example of a method for preventing web crawlers according to an embodiment of the present invention;

[0061] Figures 5a-5b The diagram shows a flowchart of a client-side and server-side method for preventing web crawling according to another embodiment of the present invention.

[0062] Figure 6 The diagram shows the client and server structure of a system designed to prevent web crawling.

[0063] Figure 7 This is a structural diagram of a device for preventing crawling according to an embodiment of the present invention. Detailed Implementation

[0064] The present invention is described below based on embodiments, but the invention is not limited to these embodiments. In the detailed description of the invention below, certain specific details are described in detail. Those skilled in the art will fully understand the invention even without these details. To avoid obscuring the essence of the invention, well-known methods, processes, and procedures are not described in detail. Furthermore, the accompanying drawings are not necessarily drawn to scale.

[0065] Figure 1The diagram shows the structure of a web system deployment, including multiple terminals and servers. Each terminal hosts a web system client, and server 130 hosts the web system server-side. For simplicity, Figure 1 Only two terminals, 101 and 102, are shown in the image. Figure 1 As shown, the system clients are deployed on 101 and 102 respectively. The web system client receives user requests and sends them to the server. The server then sends the web page to the client for display based on the user request. Web crawler systems typically simulate a browser to send requests and retrieve the text content from the returned web page.

[0066] To prevent web page text content from being crawled by web crawlers, this invention provides a method to prevent web crawlers from crawling, which combines server-side and client-side steps to ensure that web crawlers can only crawl out-of-order text. Figures 2a-2b The flowcharts shown are for the client-side and server-side methods for preventing web crawling according to embodiments of the present invention.

[0067] See Figure 2a The server-side process specifically includes the following steps.

[0068] In step S201, the text to be displayed is divided into multiple text segments.

[0069] In step S202, the order of multiple text segments is changed, and the order mapping information is recorded. The order mapping information is used to represent the correspondence between the order of the multiple text segments in the text to be displayed before and after the change.

[0070] In step S203, the encrypted sequence mapping information is obtained.

[0071] In step S204, multiple text segments are organized according to the changed sorting to obtain a web page.

[0072] In step S205, the ciphertext of the web page and sequence mapping information is sent.

[0073] In this embodiment, the server generates the web page.

[0074] In this embodiment, the text to be displayed is part or all of the text content that needs to be displayed on the web page. For example, to improve processing speed, only important text content is processed as described above, while other unimportant content is not processed.

[0075] According to this embodiment, the text to be displayed is first divided into multiple text segments, then the order of these multiple text segments in the original text is changed, and the multiple text segments are organized according to the changed order to obtain a web page. At this time, the corresponding content of the web page is already out of order. Finally, the encrypted sequence mapping information and the web page are sent to the client at the same time.

[0076] For detailed steps regarding the client, please refer to [link / reference]. Figure 2b See also Figure 2b The client-side steps specifically include the following steps.

[0077] In step S201', the encrypted sequence mapping information and the web page are received.

[0078] In step S202', the web page is rendered and output in the browser.

[0079] In step S203', the ciphertext of the sequence mapping information is decrypted.

[0080] In step S204', the order of multiple text segments in the web page is corrected according to the order mapping information.

[0081] In this embodiment, the client first receives the encrypted mapping information and the web page, then renders and outputs the web page to the display screen through the browser. At this time, the web page output by the browser is a web page containing disordered text. Then, the script set in the web page is called. The script decrypts the sequential mapping information and corrects the disordered text in the web page displayed on the display screen according to the sequential mapping information, thereby obtaining a web page containing the correct text.

[0082] For details regarding the browser rendering process described in step S202', please refer to [link / reference]. Figure 3 .like Figure 3 As shown, after receiving a web page, the browser parses the web page to build a DOM tree in step 301, builds a render tree in step 302, lays out the render tree in step 303, and draws the render tree in step 304, ultimately outputting the web page to the display screen. The browser rendering technology is existing technology and will not be described in detail here.

[0083] According to the above embodiments, when the crawler system simulates a browser sending a request to the server, the returned web page contains out-of-order text. Generally, the crawler system will crawl the web page containing out-of-order text, thus achieving the goal of preventing crawling. However, due to the client-side implementation described above, the web page is first rendered and output by the browser, so the user may see out-of-order text momentarily, which is then corrected. To avoid users seeing out-of-order text, the out-of-order text can be hidden using CSS when outputting the web page, or specific content can be output at the corresponding position of the out-of-order text, and then the script can correct the out-of-order text and output it to the corresponding position.

[0084] Figure 4 The image shows an example of a method for preventing web crawlers according to an embodiment of the present invention.

[0085] In this example, the server uses "This is a sample text" as the text to be displayed. On the server side, this text is split into three segments: "This is", "Sample Text", and "Example Text". The numbers 1, 2, and 3 represent their order in the original text. The order of these segments is then shuffled to obtain the scrambled text "Example Text This is a Sample Text". The correspondence between the order of the three segments before and after shuffling is recorded as 1(3), 2(1), and 3(2). This correspondence is then encrypted to obtain the ciphertext. This ciphertext, along with the scrambled text "Example Text This is a Sample Text" web page, is sent to the client. After receiving the web page, the client renders and outputs the scrambled text "Example Text This is a Sample Text" through its browser. The front-end script then decrypts the ciphertext to obtain the mapping relationship and corrects the order of the scrambled text in the web page based on this mapping relationship.

[0086] The web pages in the above embodiments can be based on any programming language. Currently, the mainstream web page is the HTML page. Below is the source code of a typical HTML page.

[0087] <web>

[0088]

[0089] <title> My first webpage< / title>

[0090]

[0091]

[0092] This is an example.

[0093]

[0094] < / web>

[0095] As you can see, this HTML page consists of tags and content. Below is the source code of an HTML page with the text "This is an example" scrambled.

[0096] <web>

[0097]

[0098] <title> My first webpage< / title>

[0099]

[0100]

[0101] This is an example.

[0102]

[0103] < / web>

[0104] Following the example above, the correct text "This is an example" cannot be obtained from the shuffled text "Example This is an example" and the sequence mapping information. To solve this problem, an alternative embodiment uses specific HTML tags to organize multiple text segments in the HTML page. For example, the source code of the above HTML page becomes:

[0105] <web>

[0106]

[0107] <title> My first webpage< / title>

[0108]

[0109]

[0110] Example

[0111] This is

[0112]

[0113] < / web>

[0114] The client and server can agree on the tags in advance. Used to represent text in a shuffled order. Therefore, when the client receives the HTML page, it locates... This results in multiple text segments with their order shuffled. Of course, the invention is not limited to this. In another optional embodiment, a specific tag is used to organize the multiple text segments. Specific tags include <>, |, (), &&, etc. Using <> yields the following HTML page source code:

[0115]

[0116] <title> My first webpage< / title>

[0117]

[0118]

[0119] <Example><This is a>

[0120]

[0121] In summary, web pages need to contain markup information indicating multiple text segments in order for the web page sent to the client to be correctly parsed.

[0122] Figures 5a-5b The diagram shows a flowchart of a client-side and server-side method for preventing web crawling according to another embodiment of the present invention.

[0123] See Figure 5a Specifically, it includes the following steps.

[0124] In step S501, the text to be displayed is divided into multiple text segments.

[0125] In step S502, the order of multiple text segments is changed, and the order mapping information is recorded. The order mapping information is used to represent the correspondence between the order of the multiple text segments in the text to be displayed before and after the change.

[0126] In step S503, the encrypted sequence mapping information is obtained.

[0127] In step S504, each segment of text in the web page is marked with a specific tag.

[0128] In step S505, the ciphertext of the web page and sequence mapping information is sent.

[0129] In step S506, a specific tag is encrypted.

[0130] In step S507, the ciphertext with a specific tag is sent.

[0131] See Figure 5b Specifically, it includes the following steps.

[0132] In step S501', the encrypted sequence mapping information and the web page are received.

[0133] In step S502', the ciphertext of the specific tag is received.

[0134] In step S503', the web page is rendered and output in the browser.

[0135] In step S504', the ciphertext of the sequence mapping information is decrypted.

[0136] In step S505', the ciphertext of the specific tag is decrypted.

[0137] In step S506', a specific marker is located, and the order of multiple text segments in the web page is corrected according to the sequence mapping information.

[0138] The difference between this embodiment and embodiments 2a-2b is that the server and the client do not agree in advance on how to mark multiple text segments. Instead, the server uses a specific tag to mark the segments, and then the server encrypts the specific tag and sends it to the client. After receiving the ciphertext of the specific tag, the client decrypts it to obtain the tag, locates the tag in the web page, and corrects the content corresponding to the tag in the web page according to the sequence mapping information.

[0139] Figure 6 The above is a schematic diagram of the client and server structure of a system for preventing web crawlers. The system 600 includes a client 602 and a server 601. The server 601 includes the following modules.

[0140] The segmentation module 6011 is used to segment the text to be displayed into multiple text segments.

[0141] The disorder module 6012 is used to change the sorting of the multiple text segments and record the order mapping information. The order mapping information is used to represent the correspondence between the sorting of the multiple text segments in the text to be displayed before and after the change.

[0142] The encryption module 6013 is used to encrypt the sequence mapping information.

[0143] Module 6014 creates a web page by organizing the multiple text segments according to the changed sorting.

[0144] The sending module 6015 is used to send ciphertext of web pages and sequence mapping information.

[0145] As an example, when creating a web page, the creation module can organize the multiple text segments according to specific tags to obtain the web page. These specific tags include, but are not limited to, the tags mentioned above. Similarly, the creation module can also use specific HTML tags to mark each segment of the multiple text segments within the web page. These specific HTML tags include, but are not limited to, `<html>`, `<body>`, `<textarea>`, `<body ... 、、 Of course, there is no limit to the number of tags that can be used; for example, two or more specific tags can be used.

[0146] Client 602 includes the following modules.

[0147] The receiving module 6021 is used to receive ciphertext of web pages and sequence mapping information.

[0148] The decryption module 6022 is used to decrypt the ciphertext of the sequence mapping information.

[0149] The correction module 6023 is used to correct the order of multiple text segments in a web page based on the order mapping information.

[0150] The rendering module 6024 is used to render and output web pages in a browser.

[0151] As an example, rendering module 6024 can hide multiple text segments contained in a web page, and then correction module 6023 can display the hidden text segments. Similarly, after rendering module 6024 hides multiple text segments contained in a web page, it can output specific content in the multiple text segments, and then correction module 6023 can replace the displayed specific content with the hidden text segments.

[0152] In summary, the anti-crawler system provided by this invention, through the cooperation of the client and server, first outputs a web page containing disordered text on the display screen, and then the front-end script corrects the order of multiple text segments in the web page according to the order mapping information. Thus, although the user may see a disordered web page for a brief moment, this time is very short and has minimal impact on the user experience, while ensuring that the crawler system cannot crawl the correct content.

[0153] In an optional embodiment, the receiving module 6021 receives the ciphertext of a specific tag, the decryption module decrypts the ciphertext of the specific tag, and the correction module 6023 includes: locating the specific tag in the web page and correcting the text content corresponding to the specific tag according to the sequence mapping information.

[0154] In another optional embodiment, the receiving module 6021 receives the ciphertext of a specific tag, the decryption module decrypts the ciphertext of the specific tag, and the correction module includes: locating the specific tag in the web page and correcting the text content corresponding to the specific tag according to the sequence mapping information.

[0155] It should be noted that the encryption methods referred to in this article can be either symmetric or asymmetric algorithms. In symmetric encryption algorithms, only one key is used; both the sender and receiver use this key to encrypt and decrypt data, requiring both parties to know the encryption key beforehand. In asymmetric encryption algorithms, two different keys are used for encryption and decryption. The basic process of exchanging confidential information using asymmetric encryption algorithms is as follows: Party A generates a key pair and discloses one of them as a public key to other parties; Party B, having obtained the public key, uses it to encrypt the confidential information before sending it to Party A; Party A then uses its own private key to decrypt the encrypted information.

[0156] In this invention, the client and server can agree on an encryption algorithm and a corresponding key acquisition method, and then use the key in the aforementioned method and system. For example, the server and client can agree to use a symmetric encryption algorithm, with the server periodically generating a key and sending it to the client as the key for encrypted transmission between the two ends.

[0157] Figure 7 This is a structural diagram of a device for preventing crawling according to the sixth embodiment of the present invention. Figure 7 The device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0158] refer to Figure 7 The device includes a processor 701, a memory 702, and an input / output device 703 connected via a bus. The memory 702 includes read-only memory (ROM) and random access memory (RAM), storing various computer instructions and data required to perform system functions. The processor 701 reads various computer instructions from the memory 702 to perform various appropriate actions and processes. The input / output device includes input sections such as a keyboard and mouse; output sections such as cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections such as hard disks; and communication sections such as network interface cards (LAN cards and modems). The memory 702 also stores computer instructions from the client or server side to complete the operations specified by the device in this embodiment of the invention. The client-side computer instructions include: dividing the text to be displayed into multiple text segments; changing the order of the multiple text segments and recording order mapping information, wherein the order mapping information is used to represent the correspondence between the order of the multiple text segments in the text to be displayed before and after the change; encrypting the order mapping information; organizing the multiple text segments according to the changed order to obtain a web page; and sending the web page and the encrypted order mapping information. The server-side computer instructions include: receiving the encrypted order mapping information and the web page; rendering and outputting the web page in a browser; decrypting the encrypted order mapping information; and correcting the order of the multiple text segments in the web page according to the order mapping information.

[0159] Accordingly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that, when executed, implement the operations specified by a client-side or server-side method.

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the possible system framework, functions, and operations of the systems, methods, and apparatuses according to embodiments of the present invention. The blocks in the flowcharts and block diagrams may represent a module, a program segment, or simply a piece of code. Modules, program segments, and code are all executable instructions used to implement a specified logical function. It should also be noted that executable instructions implementing a specified logical function can be recombine to create new modules and program segments. Therefore, the blocks and their order in the accompanying drawings are only used to better illustrate the processes and steps of the embodiments and should not be construed as limiting the invention itself.

[0161] The various modules or units of the system can be implemented through hardware, firmware, or software. Software includes, for example, coded programs written in various programming languages ​​such as JAVA, C / C++ / C#, and SQL. Although the steps and their order are given in the method and method diagrams of embodiments of the invention, the executable instructions that implement the specified logical function of each step can be recombine to create new steps. The order of steps should not be limited to the order shown in the method and method diagrams and can be adjusted at any time according to functional needs. For example, some steps can be executed in parallel or in reverse order.

[0162] The system and method according to the present invention can be deployed on one or more servers. For example, different modules can be deployed on different servers to form a dedicated server. Alternatively, the same functional units, modules, or systems can be distributed across multiple servers to reduce load pressure. Servers include, but are not limited to, multiple PCs, PC servers, blade servers, supercomputers, etc., connected on the same local area network or via the Internet.

[0163] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for preventing web crawlers, involving a server-side and a client-side, characterized in that, The server performs the following operations: Divide the text to be displayed into multiple text segments; The order of the multiple text segments is changed, and the order mapping information is recorded. The order mapping information is used to represent the correspondence between the order of the multiple text segments in the text to be displayed before and after the change. Encrypt the sequence mapping information; The multiple text segments are reorganized according to the altered sorting to obtain a web page with incorrect text sorting; and Send the web page with the incorrect text sorting and the ciphertext of the sequence mapping information; The client performs the following operations: The encrypted sequence mapping information and the web page with the incorrect text sorting are received; Render and output the web page with the incorrect text sorting in the browser; Decrypt the ciphertext of the sequence mapping information; as well as The order mapping information is used to correct the sorting of multiple text segments in the web page with incorrect text sorting, so as to display the web page with correct text sorting.

2. The method according to claim 1, characterized in that, The server uses either a symmetric or asymmetric algorithm to encrypt the sequence mapping information.

3. The method according to claim 1, characterized in that, The server-side organization of the multiple text segments to obtain a web page with incorrect text sorting includes: The multiple text segments are organized according to specific tags to obtain the web page with incorrect text sorting.

4. The method according to claim 1, characterized in that, The server-side organization of the multiple text segments to obtain a web page with incorrect text sorting includes: Each segment of the multiple text segments is marked with a specific HTML tag to obtain a web page with incorrect text sorting.

5. The method according to claim 3, characterized in that, The server side further includes: encrypting the specific tag and sending the ciphertext.

6. The method according to claim 3, characterized in that, The client further includes: receiving and decrypting the ciphertext of the specific tag; the step of correcting the sorting of multiple text segments in the web page with incorrect text sorting according to the order mapping information includes: locating the specific tag in the web page with incorrect text sorting; and correcting the text content corresponding to the specific tag according to the order mapping information.

7. The method according to claim 1, characterized in that, The client's method of correcting the sorting of multiple text segments in the web page with incorrect text sorting according to the order mapping information includes: locating specific HTML tags in the web page with incorrect text sorting, and correcting the text content corresponding to the specific HTML tags according to the order mapping information.

8. The method according to claim 1, characterized in that, When the client renders and outputs a web page in the browser, it hides the multiple text segments contained therein, and when it corrects the sorting of the multiple text segments in the web page with incorrect text sorting according to the order mapping information, it displays the corrected multiple text segments.

9. The method according to claim 1, characterized in that, When the client renders and outputs a web page in the browser, it hides the multiple text segments contained therein, then outputs specific content at the positions of the multiple text segments, and when correcting the sorting of the multiple text segments in the web page with incorrect text sorting according to the order mapping information, it replaces the specific content with the corrected multiple text segments.

10. A system for preventing web crawlers, comprising a server and a client, characterized in that, The server includes: The segmentation module is used to divide the text to be displayed into multiple segments. The disorder module is used to change the sorting of the multiple text segments and record the order mapping information. The order mapping information is used to represent the correspondence between the sorting of the multiple text segments in the text to be displayed before and after the change. An encryption module is used to encrypt the sequence mapping information; Create a module that organizes the multiple text segments according to the changed sorting to obtain a web page with incorrect text sorting; A sending module is used to send the web page with the incorrect text sorting and the ciphertext of the sequence mapping information; The client includes: A receiving module is used to receive the ciphertext of the web page with incorrect text sorting and sequence mapping information; The decryption module is used to decrypt the ciphertext of the sequence mapping information; A correction module is used to correct the sorting of multiple text segments in a web page with incorrect text sorting based on the order mapping information. The rendering module is used to render and output the web page with the incorrect text sorting in the browser.

11. The system according to claim 10, characterized in that, The creation module includes: The multiple text segments are organized according to specific tags to obtain the web page with incorrect text sorting.

12. The system according to claim 10, characterized in that, The creation module includes: Each segment of the multiple text segments is marked with a specific HTML tag to obtain a web page with incorrect text sorting.

13. The system according to claim 10, characterized in that, When the rendering module hides the multiple text segments contained in the web page, the correction module displays the corrected multiple text segments when correcting the sorting of the multiple text segments in the web page with incorrect text sorting according to the order mapping information.

14. The system according to claim 10, characterized in that, After the rendering module hides the multiple text segments contained in the web page, it outputs specific content at the positions of the multiple text segments. When the correction module corrects the sorting of the multiple text segments in the web page with incorrect text sorting according to the order mapping information, it replaces the specific content with the corrected multiple text segments.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed, implement the method as described in any one of claims 1 to 9.

16. A device for preventing crawling insects, characterized in that, include: Memory, used to store computer instructions; A processor, coupled to the memory, the processor being configured to execute computer instructions stored in the memory to implement the method as described in any one of claims 1 to 9.

Images