A page access method and device, electronic equipment and storage medium

By generating random hidden links and embedding them into pages to disguise them as legitimate links, and by blocking IP addresses a set number of times, the problem of existing anti-crawler methods failing to identify web crawlers is solved, achieving an effective anti-crawler effect.

CN115687728BActive Publication Date: 2025-12-05CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202110785162.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-12
Publication Date
2025-12-05
Estimated Expiration
2041-07-12

AI Technical Summary

Technical Problem

Web crawlers consume website resources, and existing anti-crawler methods are easily detected and rendered ineffective by web crawlers.

Method used

Random hidden links are generated and embedded into the pages to be visited, disguised as legitimate links, increasing the difficulty for web crawlers to identify them, and the IP address is blocked after a set number of times to prevent web crawlers from accessing the site.

Benefits of technology

It effectively prevents web crawlers from accessing the site, increases the difficulty for crawlers to identify hidden links, reduces crawling performance, and achieves effective anti-crawler effects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115687728B_ABST
    Figure CN115687728B_ABST
Patent Text Reader

Abstract

Embodiments of the present application are suitable for the technical field of computer security, and provide a page access method and device, electronic equipment and a storage medium, wherein the page access method comprises: in the case of receiving a page access request for a to-be-accessed page, generating at least one dark link, wherein each generated dark link points to a legal page; embedding each generated dark link into the to-be-accessed page; returning a page response of the page access request; the to-be-accessed page after embedding the dark link is included in the page response.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer security technology, and in particular to a page access method, apparatus, electronic device, and storage medium. Background Technology

[0002] Web crawlers consume website resources and harm the website. Therefore, how to implement anti-crawling measures is the technical problem that this application aims to solve. Summary of the Invention

[0003] To address the aforementioned problems, embodiments of the present invention provide a page access method, apparatus, electronic device, and storage medium, to at least resolve the issue of web crawlers causing harm to websites in related technologies.

[0004] The technical solution of this invention is implemented as follows:

[0005] In a first aspect, embodiments of the present invention provide a page access method, the method comprising:

[0006] Upon receiving a page access request for a page to be accessed, at least one hidden link is generated, wherein each of the generated hidden links points to a legitimate page;

[0007] The generated hidden links are embedded into the page to be accessed;

[0008] Return the page response to the page access request; the page response includes the page to be accessed after the hidden link is embedded.

[0009] In the above scheme, upon receiving a page access request for the page to be accessed, the method further includes:

[0010] Determine whether the access link corresponding to the page access request is a hidden link; then,

[0011] Generate at least one dark chain, including:

[0012] If the access link corresponding to the page access request is not a hidden link, generate at least one hidden link; or, if the access link corresponding to the page access request is a hidden link, and the number of times the IP address in the page access request accesses the hidden link is less than a set value, generate at least one hidden link.

[0013] The method in the above scheme further includes:

[0014] If the access link corresponding to the page access request is a hidden link, and the number of times the IP address in the page access request accesses the hidden link is greater than or equal to a set value, the page access request will be blocked.

[0015] In the above scheme, after intercepting the page access request, the method further includes:

[0016] After a set time period, the blocking of page access requests containing the IP address will be lifted.

[0017] In the above scheme, generating at least one dark chain includes:

[0018] The at least one hidden link is randomly generated based on at least one set link; each of the at least one set link points to a legitimate page.

[0019] In the above scheme, the random generation of the at least one dark link based on at least one predefined link includes:

[0020] Randomly select two adjacent fields in one of the links of at least one of the specified links;

[0021] Insert a set value between the two adjacent fields to generate one of the at least one dark chain.

[0022] In the above scheme, the random generation of the at least one dark link based on at least one predefined link includes:

[0023] Randomly select one setting value from at least two settings;

[0024] A randomly selected setting value is inserted into one of the links in the at least one set of links to generate one of the at least one dark chains.

[0025] In the above scheme, embedding the generated hidden links into the page to be accessed includes:

[0026] The generated hidden links are randomly embedded into the page to be accessed.

[0027] Secondly, embodiments of the present invention provide a page access device, the device comprising:

[0028] A generation module is used to generate at least one hidden link upon receiving a page access request for a page to be accessed, wherein each of the generated hidden links points to a legitimate page.

[0029] An embedding module is used to embed the generated hidden links into the page to be accessed;

[0030] The return module is used to return the page response to the page access request; the page response includes the page to be accessed after embedding the hidden link.

[0031] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute the steps of the page access method provided in the first aspect of the present invention.

[0032] Fourthly, embodiments of the present invention provide a computer-readable storage medium, comprising: the computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the page access method provided in the first aspect of the present invention.

[0033] This invention, upon receiving a page access request for a page to be accessed, generates at least one hidden link, each pointing to a legitimate page. These hidden links are then embedded into the page to be accessed, and a page response is returned, including the page to be accessed with the embedded hidden links. By disguising the hidden links as legitimate links, web crawlers accessing these links will be identified as legitimate pages, making it impossible for them to avoid accessing them. This allows web crawlers accessing the hidden links to be blacklisted, thus achieving anti-crawler measures. Attached Figure Description

[0034] Figure 1 This is a schematic diagram illustrating the implementation process of a page access method provided in an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram illustrating the implementation process of another page access method provided in an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram illustrating the implementation process of another page access method provided in an embodiment of the present invention;

[0037] Figure 4 This is a flowchart illustrating an anti-crawler method provided in an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of a page access device provided in an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Numerous web crawlers operate on the internet, consuming website resources and causing significant harm. Detecting and preventing web crawlers, and establishing anti-crawling mechanisms, can ensure that websites can securely provide access services and protect website content and user privacy.

[0042] One related technique involves embedding hidden links in web pages. When a request reaches this hidden link, it is identified as malicious web crawling. The response packet then sets a cookie on the client, and the generated cookie, accessed domain, and source IP are added to a blacklist, blocking HTTP request messages. However, hidden links on a page can be accidentally accessed by users, and directly blocking user access may affect normal business services. Secondly, hidden links can be gradually discovered. Web crawlers can progressively improve their link blacklists, preventing the crawling of hidden links and rendering anti-crawler methods ineffective.

[0043] Another related technique involves embedding a JavaScript mining script link in a webpage. When the script link is requested, the terminal accessing the target webpage is triggered to load and execute the JavaScript mining script. The mining script consumes a large amount of the central processing unit (CPU) resources of the crawler terminal, thus preventing the web crawler terminal from functioning properly. The drawback of this approach is that the web crawler may not necessarily execute the mining script, and the script link can also be gradually discovered by the web crawler. As the web crawler gradually improves its scanning blacklist, this anti-crawler method will become ineffective.

[0044] To address the shortcomings of the aforementioned related technologies, embodiments of the present invention provide a page access method that can at least solve the problem of web crawlers causing harm to websites in the related technologies. To illustrate the technical solution described in this invention, specific embodiments are described below.

[0045] Figure 1 This is a schematic diagram illustrating the implementation flow of a page access method provided in an embodiment of the present invention. The executing entity of the page access method can be an electronic device, such as a desktop computer, a laptop computer, or a server. (Reference) Figure 1 Page access methods include:

[0046] S101, upon receiving a page access request for the page to be accessed, at least one dark link is generated, wherein each of the generated dark links points to a legitimate page.

[0047] In this embodiment of the invention, at least one hidden link is generated regardless of whether a page access request is received from a normal user or an abnormal user. For example, an abnormal user is a web crawler.

[0048] Hidden links embedded in web pages are difficult to identify and usually require methods such as viewing the page's source code or other special detection techniques to discover. Web crawlers can scrape data from web pages, including hidden links embedded within them.

[0049] Here, a legitimate link refers to a webpage link that is accessible, provides normal business services, and points to a page with no abnormal content. For example, a legitimate link could be a link to a company's official website. The page that a legitimate link points to is a legitimate page.

[0050] In one embodiment, generating at least one dark chain includes:

[0051] The at least one hidden link is randomly generated based on at least one set link; each of the at least one set link points to a legitimate page.

[0052] In this embodiment of the invention, a hidden link is randomly generated by setting at least one link, and all of these links are legitimate. Randomly generating a hidden link ensures that each generated hidden link is different. Even if a web crawler identifies a hidden link and adds it to its blacklist, the next generated hidden link will not be the same as the previous one, rendering the web crawler's blacklist ineffective.

[0053] refer to Figure 2 In one embodiment, the random generation of the at least one dark chain based on at least one set link includes:

[0054] S201, randomly select two adjacent fields in one of the links of the at least one set of links.

[0055] S202, insert a set value between the two adjacent fields to generate one of the at least one dark chain.

[0056] Here, the settings include preset letters, numbers, and words. By inserting these settings into legitimate links, hidden links can be generated.

[0057] In this embodiment of the invention, assuming that one of the at least one links is “ / user / index”, two adjacent fields are randomly selected as r / , and the set value “s” is inserted between the two adjacent fields to generate the hidden link “ / users / index”.

[0058] refer to Figure 3 In another embodiment, the random generation of the at least one dark chain based on at least one predetermined link includes:

[0059] S301, randomly select one setting value from at least two setting values.

[0060] S302, a randomly selected setting value is inserted into one of the links of the at least one set link to generate one of the at least one dark links.

[0061] In this embodiment of the invention, a large set of set values ​​is pre-set. By randomly selecting a set value from the set value set, the set value is inserted into a link of at least one set link, thereby generating a hidden link.

[0062] For example, a legitimate link is " / user / index". By inserting a randomly selected value into the legitimate link, the generated hidden links can be " / users / index", " / use / index", " / user / indexs", " / user / indexes", " / user / index.html", etc.

[0063] In the above embodiments, multiple settings can be inserted into the link at once, and an anti-duplicate mechanism can be set to prevent the same setting from being inserted in the same position. It should be understood that the generated hidden link is different from the legitimate link.

[0064] This invention generates hidden links randomly, ensuring that each generated hidden link is different. Even if a web crawler adds the generated hidden link to its blacklist, the next generated hidden link will not be the same as the previous one, thus rendering the web crawler's hidden link blacklist ineffective.

[0065] In this embodiment of the invention, each of the at least one hidden link points to a normal page, thus disguising the hidden link as a legitimate link. When a web crawler accesses a hidden link, it will determine that it is a normal page, making it impossible for the web crawler to avoid accessing the hidden link. In practical applications, users who access the hidden link can be added to a blacklist, thereby achieving anti-crawler measures.

[0066] S102, embed the generated hidden links into the page to be accessed.

[0067] Here, if the page access request accesses a legitimate page, then the page to be accessed is the legitimate page itself. If the page access request accesses a hidden link, then the page to be accessed is a random legitimate page.

[0068] In one embodiment, embedding the generated hidden links into the page to be accessed includes:

[0069] The generated hidden links are randomly embedded into the page to be accessed.

[0070] All generated hidden links are randomly embedded into the page to be visited. For example, if 5 hidden links are generated, these 5 hidden links are randomly embedded into random positions on the page to be visited.

[0071] Related techniques typically embed hidden links in fixed locations, making it possible for web crawlers to identify patterns in the embedding locations and thus easily crawl the hidden links. However, by embedding them randomly, the difficulty for web crawlers to crawl hidden links can be increased.

[0072] S103, return the page response to the page access request; the page response includes the page to be accessed after embedding the hidden link.

[0073] The page to be accessed after embedding a hidden link is used as the page response for the page access request. Since the hidden link points to a legitimate page, the web crawler will judge the page as normal after accessing the hidden link, making it impossible for the web crawler to avoid accessing the hidden link.

[0074] This invention expands the scope of hidden links by generating at least one hidden link randomly embedded in the page to be visited. Disguising these hidden links as legitimate links increases the difficulty for web crawlers to identify them. This invention does not require web crawlers to execute scripts; by returning hidden links to the crawler, these links point to other random hidden links, misleading the crawler into crawling a large number of useless links and page information, thereby reducing the crawler's performance.

[0075] This invention, upon receiving a page access request for a page to be accessed, generates at least one hidden link, each pointing to a legitimate page. These hidden links are then embedded into the page to be accessed, and a page response is returned, including the page to be accessed with the embedded hidden links. By disguising the hidden links as legitimate links, web crawlers accessing these links will be identified as legitimate pages, making it impossible for them to avoid accessing them. This allows web crawlers accessing the hidden links to be blacklisted, thus achieving anti-crawler measures.

[0076] In one embodiment, upon receiving a page access request for a page to be accessed, the page access method further includes:

[0077] Determine whether the access link corresponding to the page access request is a hidden link; then,

[0078] Generating the at least one dark chain includes:

[0079] If the access link corresponding to the page access request is not a hidden link, generate at least one hidden link; or, if the access link corresponding to the page access request is a hidden link, and the number of times the IP address in the page access request accesses the hidden link is less than a set value, generate at least one hidden link.

[0080] Here, a hidden link database can be set up to store the generated hidden links. The access links corresponding to the page access request are matched against the hidden link database. If a match is successful, it means that the access link corresponding to the page access request is a hidden link.

[0081] If the access link corresponding to the page access request is not a hidden link, then at least one hidden link will be generated.

[0082] If the access link corresponding to the page access request is a hidden link, then determine the number of times the IP address in the page access request accesses the hidden link. If the number of times the IP address in the page access request accesses the hidden link is less than a set value, then generate at least one hidden link.

[0083] Each page access request carries the visitor's Internet Protocol (IP) address and other identity information, and the number of times each IP address accesses the hidden link is counted.

[0084] In practical applications, when a user visits the hidden link for the first time, a cookie is generated locally, storing user information such as the number of visits, IP address, and URL. The next time the user visits the hidden link, the visit count is updated in the cookie. By viewing the cookie, the number of times each IP address has accessed the hidden link can be determined.

[0085] If the number of times an IP address accesses a hidden link is less than a set value, the page access request is allowed. At this point, at least one hidden link is generated, and each of these hidden links points to a legitimate page. A page response is returned regarding the page access request, containing the page to be accessed with the embedded hidden link. Simultaneously, the number of times the IP address accesses a hidden link is updated.

[0086] In one embodiment, the method further includes:

[0087] If the access link corresponding to the page access request is a hidden link, and the number of times the IP address in the page access request accesses the hidden link is greater than or equal to a set value, the page access request will be blocked.

[0088] If an IP address accesses hidden links a certain number of times, then requests to access pages containing that IP address will be blocked, preventing web crawlers from harming the website.

[0089] Here, blocking only occurs after a set number of attempts has been made. On the one hand, this allows for accidental actions by normal users, and on the other hand, it consumes the resources of web crawlers. Web crawlers can only know the link accessed at the last time it was blocked. Web crawlers cannot detect all hidden links, making it difficult for web crawlers to build a comprehensive blacklist of hidden links, thus effectively ensuring the effectiveness of anti-crawler measures.

[0090] Additionally, if the link corresponding to the page access request is not a hidden link, at least one hidden link is generated, and each of these hidden links points to a normal page. A page response is returned regarding the page access request, containing the page to be accessed that embeds at least one hidden link. This approach allows for accidental actions by normal users while increasing the difficulty for web crawlers to identify hidden links, making it difficult for crawlers to build a comprehensive blacklist of hidden links, thus effectively ensuring the effectiveness of anti-crawler measures.

[0091] In the above embodiments, after intercepting the page access request, the method further includes:

[0092] After a set time period, the blocking of page access requests containing the IP address will be lifted.

[0093] The reason for only blocking requests for a set time period is that web crawlers may be using rented temporary IP addresses, which may not be their real IP addresses. After the web crawler abandons the IP address, it may be rented by legitimate users. If blocking is prolonged, it will affect the business services of legitimate users. Therefore, only page access requests containing the IP address are blocked within the set time period.

[0094] refer to Figure 4 , Figure 4 This is a flowchart illustrating an anti-scraping method provided in an application embodiment of the present invention. The anti-scraping method includes:

[0095] S401, receives page access request.

[0096] S402, determine whether the access link corresponding to the page access request is a hidden link.

[0097] If the access link corresponding to the page access request is a hidden link, then proceed to step S403; otherwise, proceed to step S405.

[0098] S403, determine whether the number of times the IP address in the page access request accesses the hidden link is greater than the set value.

[0099] If the number of times the IP address accesses the hidden link exceeds the set value, then proceed to step S404. Otherwise, proceed to step S405.

[0100] S404, the access request containing the IP address is intercepted.

[0101] S405, generate at least one hidden link based on the legitimate link, and embed at least one hidden link into the page to be accessed corresponding to the page access request.

[0102] Here, at least one hidden link in the hidden link points to a normal page.

[0103] Insert a set value into any one of the at least one set links to generate one of the at least one dark links.

[0104] Here, the settings include preset letters, numbers, and words. By inserting these settings into legitimate links, hidden links can be generated. For example, a legitimate link is " / user / index". By inserting settings into legitimate links, hidden links that can be generated include " / users / index", " / use / index", " / user / indexes", " / user / index.html", etc. It should be understood that the generated hidden links cannot overlap with legitimate links.

[0105] S406, Return a page response to the page access request; the page response contains the page to be accessed with an embedded hidden link.

[0106] This invention, through its application embodiments, disguises hidden links as legitimate links, increasing the difficulty for web crawlers to identify them and making it impossible for them to avoid accessing these links. This allows web crawlers accessing hidden links to be added to a blacklist, thus achieving anti-crawling protection. When the number of accesses to a hidden link exceeds a set value, access requests containing the specified IP address are blocked. This allows for accidental access by legitimate users while simultaneously increasing the difficulty for web crawlers to identify hidden links, making it difficult for them to build a robust link blacklist and effectively ensuring the effectiveness of anti-crawling measures.

[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0108] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0109] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0110] In addition, in the embodiments of the present invention, "first," "second," etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0111] refer to Figure 5 , Figure 5 This is a schematic diagram of a page access device provided in an embodiment of the present invention, such as... Figure 5 As shown, the device includes a generation module, an embedding module, and a return module.

[0112] A generation module is used to generate at least one hidden link upon receiving a page access request for a page to be accessed, wherein each of the generated hidden links points to a legitimate page.

[0113] An embedding module is used to embed the generated hidden links into the page to be accessed;

[0114] The return module is used to return the page response to the page access request; the page response includes the page to be accessed after embedding the hidden link.

[0115] In one embodiment, upon receiving a page access request for a page to be accessed, the apparatus further includes:

[0116] The judgment module is used to determine whether the access link corresponding to the page access request is a hidden link;

[0117] When generating at least one hidden link, the generation module is specifically used to: generate at least one hidden link when the access link corresponding to the page access request is not a hidden link; or, generate at least one hidden link when the access link corresponding to the page access request is a hidden link and the number of times the IP address in the page access request accesses the hidden link is less than a set value.

[0118] In one embodiment, the device further includes:

[0119] The interception module is used to intercept the page access request if the access link corresponding to the page access request is a hidden link and the number of times the IP address in the page access request accesses the hidden link is greater than or equal to a set value.

[0120] In one embodiment, after intercepting the page access request, the interception module is further configured to:

[0121] After a set time period, the blocking of page access requests containing the IP address will be lifted.

[0122] In one embodiment, the generation module, when generating at least one dark chain, is used to:

[0123] The at least one hidden link is randomly generated based on at least one set link; each of the at least one set link points to a legitimate page.

[0124] In one embodiment, when the generation module randomly generates the at least one dark link based on at least one set link, it is used to:

[0125] Randomly select two adjacent fields in one of the links of at least one of the specified links;

[0126] Insert a set value between the two adjacent fields to generate one of the at least one dark chain.

[0127] In one embodiment, when the generation module randomly generates the at least one dark link based on at least one set link, it is used to:

[0128] Randomly select one setting value from at least two settings;

[0129] A randomly selected setting value is inserted into one of the links in the at least one set of links to generate one of the at least one dark chains.

[0130] In one embodiment, the embedding module embeds the generated hidden links into the page to be accessed, for the following purposes:

[0131] The generated hidden links are randomly embedded into the page to be accessed.

[0132] In practical applications, the generation module, embedding module, and return module can be implemented by processors in electronic devices, such as central processing units (CPUs), digital signal processors (DSPs), microcontroller units (MCUs), or field-programmable gate arrays (FPGAs).

[0133] It should be noted that the page access device provided in the above embodiments is only illustrated by the division of the above modules when generating hidden links. In actual applications, the above processing can be assigned to different modules as needed, that is, the internal structure of the device can be divided into different modules to complete all or part of the processing described above. In addition, the page access device and the page access method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0134] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 6 This is a schematic diagram of the hardware structure of the electronic device according to an embodiment of this application, as shown below. Figure 6 As shown, the electronic device includes:

[0135] A communication interface enables information exchange with other devices, such as network devices.

[0136] The processor, connected to the communication interface, enables information interaction with other devices and, when running a computer program, executes the methods provided by one or more technical solutions on the electronic device side. The computer program is stored in memory.

[0137] Of course, in practical applications, the various components in an electronic device are coupled together through a bus system. It can be understood that the bus system is used to achieve communication and connection between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 6 The general will label all buses as bus systems.

[0138] The memory in this application embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0139] It is understood that memory can be volatile or non-volatile, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0140] The methods disclosed in the embodiments of this application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory. The processor reads the program from the memory and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0141] Optionally, when the processor executes the program, it implements the corresponding processes implemented by the electronic device in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0142] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a first memory storing a computer program, which can be executed by a processor of an electronic device to complete the steps described in the foregoing method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed apparatus, electronic devices, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0144] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0145] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0146] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0147] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0148] It should be noted that the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0149] In addition, in this application example, "first," "second," etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0150] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A page access method, characterized by, The method comprises: In the case of receiving a page access request for a to-be-accessed page, at least one dark link is randomly generated based on at least one set link, each link in the at least one set link points to a legal page, and each generated dark link points to a legal page; Each generated dark link is embedded into the to-be-accessed page; A page response of the page access request is returned, and the to-be-accessed page after the embedding of the dark link is included in the page response; The method further comprises: In the case of receiving a page access request for a to-be-accessed page, it is judged whether the access link corresponding to the page access request is a dark link; In the case that the access link corresponding to the page access request is not a dark link, at least one dark link is generated, or in the case that the access link corresponding to the page access request is a dark link and the number of times of accessing the dark link by the IP address in the page access request is less than a set value, at least one dark link is generated.

2. The method of claim 1, wherein, The method further comprises: In the case that the access link corresponding to the page access request is a dark link and the number of times of accessing the dark link by the IP address in the page access request is greater than or equal to a set value, the page access request is intercepted. After the page access request is intercepted, the method further comprises:

3. The method of claim 2, wherein, The interception of the page access request containing the IP address is released after a set time length. The embedding of each generated dark link into the to-be-accessed page comprises:

4. The method of claim 3, wherein, Each generated dark link is randomly embedded into the to-be-accessed page. The method comprises:

5. The method of claim 1, wherein, A generating module is configured to, in the case of receiving a page access request for a to-be-accessed page, randomly generate at least one dark link based on at least one set link; Each link in the at least one set link points to a legal page, and each generated dark link points to a legal page; 6. A page access device, characterized by An embedding module is configured to embed each generated dark link into the to-be-accessed page; A returning module is configured to return a page response of the page access request, and the to-be-accessed page after the embedding of the dark link is included in the page response; The generating module is specifically configured to: Randomly select two adjacent fields in one link of the at least one set link, insert a set value between the two adjacent fields, and generate one dark link in the at least one dark link; Or, randomly select one set value from at least two set values, and insert the randomly selected set value into one link of the at least one set link to generate one dark link in the at least one dark link. ​ ​ ​ 7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the page access method according to any one of claims 1-5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program comprises program instructions, which, when executed by a processor, cause the processor to perform the page access method according to any one of claims 1-5.

Citation Information

Patent Citations

  • Method and system for identifying web crawler, storage medium and electronic apparatus

    CN109214181A

  • Anti-web crawler honey pot designing and realizing method

    CN109474629A