Network asset anti-mapping method and system based on website transparent proxy

Abstract

The invention provides a network asset anti-mapping method and system based on website transparent proxy, and belongs to the technical field of network space mapping. According to the method, a reverse proxy website mode is used, and behavior analysis is performed on the request of the user, so that the problem of illegal access of the website is solved, and the problem of detection of illegal crawlers of the website can be solved. According to the method, a mode of combining JS code dynamic rendering with reverse proxy is used, the content of the protected website is replaced, and then the JS code is used for re-rendering, so that the problem that the website content is crawled for malicious attribution analysis is solved. According to the method, a font mapping mode is used, so that the content of the protected website is normally displayed, meaningless messy codes are presented in the webpage source codes, and the risk that the webpage content is crawled by a dynamic crawler to be used for attribution analysis is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of network space mapping, and in particular relates to a network asset anti-mapping method and system based on website transparent proxy. Background Art

[0002] In the current Internet era, the widespread application of network asset detection technology provides an effective means for website data crawling and information acquisition. This automated tool can be used for legitimate purposes, including search engine indexing, large-scale data collection, and various business purposes, facilitating the integration and analysis of information. However, with the popularization of network asset detection technology, some potential security threats have also begun to emerge, posing a challenge to the security of websites.

[0003] Although network asset detection plays an important role in legitimate applications, there is also a risk that some criminals may abuse this technology. This includes but is not limited to stealing personal information, performing malicious attacks, collecting sensitive data, or performing unauthorized large-scale crawling. These improper behaviors may lead to user privacy leakage, website service interruption, and even threaten the business secrets of enterprises and organizations.

[0004] Of particular concern is illegal probing, where network asset probing may be used to maliciously probe a website in an attempt to discover vulnerabilities or weaknesses for unauthorized access or attack. In this case, the website may be at greater risk, potentially leading to data breaches, service disruptions, or even system intrusions.

[0005] Therefore, as the application scope of network asset detection technology expands, protecting website security has become an urgent problem to be solved. In this context, researchers and network security professionals need to explore in depth how to effectively deal with the potential risks caused by network asset detection, especially the problem of illegal detection. It is crucial to formulate corresponding security strategies and technical means. At the same time, legal and regulatory agencies also need to pay attention and take measures to ensure that the use of network asset detection is within the legal scope and prevent abuse by criminals. Only by comprehensively considering technical, management and legal means can the security of the website be comprehensively improved and the legitimate rights and interests of users and enterprises be protected.

[0006] Existing related works on network asset owner identification can be divided into two categories: query-based methods and named entity recognition-based methods.

[0007] Querying public information repositories such as WHOIS of IANA and IPIP is a common method to identify the owners of large-scale network assets. By querying the Whois database to obtain the registration information of the domain name, the name of the individual or organization in the registration information is regarded as the owner of the network asset. This method is simple and fast, but because domain names and IP addresses can be sublet or sold, the registrant of the network asset cannot be equated with the owner. Many organizations in the registration information are Internet service providers or cloud service providers, not the real owners. In addition, there are also cases of missing and encrypted information in the public information database.

[0008] Therefore, in recent years, researchers have paid attention to more comprehensive and reliable network asset content information, and extracted the names of network asset owners from network asset content through named entity recognition technology. Extracting owner names from network asset content through regex rules has higher accuracy than directly using registration information, but the recall rate of rule extraction is not high. For the study of network device attribution analysis, multimodal data fusion network device attribution analysis and recognition uses website text data and image data to train the recognition model, and the recognition accuracy rate reaches 82% on the dataset constructed in the paper. The named entity recognition model that integrates multi-scale image information and multimodal information combines network asset domain name and logo information to identify the owner name from a specific sentence of the network asset, which improves the recognition accuracy and recall rate. However, since named entity recognition is a sentence-level task, the above method needs to first locate the sentence containing the owner name based on some keywords appearing in the network asset content, and then perform owner name recognition. Therefore, this method requires that the network asset content must contain specific keywords and owner names.

[0009] In the research on anti-attribution analysis of network devices, there are some studies related to the anti-IP social attribution identification of single-node micro-data micro-perturbation. The dynamic obfuscation method based on WEB reverse proxy aims to solve the problem that WEB clients are easily analyzed by malicious means, reducing the risk of network devices being attributed to analysis.

[0010] In general, the methods for identifying network asset owners are mainly based on database queries and content. The current anti-attribution analysis only uses single-node micro-data micro-disturbance to resist IP social attribution identification, but there is no relevant anti-cyberspace mapping technology to solve the problem of network asset content being used for attribution analysis. Summary of the invention

[0011] In order to solve the above technical problems, the present invention proposes a network asset anti-mapping solution based on a website transparent proxy, which is used to prevent the website from being illegally detected or the web page content from being used for malicious attribution analysis.

[0012] The present invention proposes a network asset anti-mapping method based on a website transparent proxy, which realizes network asset anti-mapping by calling a reverse proxy module, a user behavior recognition module, and a user request data processing module; the method specifically includes: Step S1, calling the reverse proxy module; the reverse proxy module includes a user input module and a user request data return module; wherein: Call the user input module, receive and cache the user's access request into the input buffer by using a reverse proxy in the website to be protected, and analyze the user's input request header and request body; Call the user request data return module, after making a request to the website the user wants to visit, modify the website's return data packet and feed it back to the user; Step S2, calling a user behavior identification module; the user behavior identification module includes a first behavior classification module that performs behavior classification based on user request header features and a second behavior classification module that performs behavior classification based on user access behavior; wherein: Calling the first behavior classification module to analyze the request header content of the user's request and identify the access of abnormal users according to keywords and preset request headers; Call the second behavior classification module to count the access time from the same customer and calculate the access frequency, and perform access control based on the frequency; Step S3, calling the user request data processing module; the user request data processing module is used to modify the content of the returned data packet response body, including a dynamic rendering module and a font mapping module; wherein: Call the dynamic rendering module to replace the keywords of the original website content in the response body of the returned data packet, and then use Javascript code dynamic rendering to reverse the replacement; The font mapping module is called to modify the returned original website content into custom text, and the font file is introduced for mapping, so that the web page is displayed normally, while the web page source code is in a garbled state.

[0013] According to the method of the first aspect of the present invention, the workflow of the reverse proxy module includes request forwarding, HTTP rewriting and redirection; wherein: the reverse proxy server receives a request from a client and forwards the request to a backend server for processing; and the client does not communicate directly with the backend server, but uses the reverse proxy server as an intermediary, and the reverse proxy rewrites the user's request to achieve URL redirection.

[0014] According to the method of the first aspect of the present invention, the request forwarding specifically includes: listening on the opened port, recording the request header of the client, constructing a new request packet with the reverse proxy information for the request body and sending it to the backend; after obtaining the return data from the backend, extracting the response body, combining it with the client's request header to construct a return data packet, and returning the return data packet to the client, thereby realizing request forwarding; and concurrently processing client requests to realize simultaneous access from multiple users.

[0015] According to the method of the first aspect of the present invention, the reverse proxy module also performs load balancing processing when executing the reverse proxy website; wherein: the requests to the back-end server are reduced by caching the static resources of the back-end server, thereby reducing the server burden and improving the response speed; at the same time, the reverse proxy acts as a load balancer, and distributes the client's requests to multiple back-end servers through polling and weight distribution, thereby improving the availability of the website.

[0016] According to the method of the first aspect of the present invention, the load balancing processing specifically includes: the reverse proxy reserves a portion of space as a buffer when each client accesses, for storing the static resources of the backend, and after obtaining the response body from the backend, stores the static resources therein in the buffer. In subsequent accesses by the client, it first searches for static resources in the cache, then sends a request to the backend, and after obtaining the response body, updates the buffer content.

[0017] According to the method of the first aspect of the present invention, the dynamic rendering module is called, and after obtaining the response data packet from the back end, the response body therein is extracted, and the original website page content is processed by js obfuscation, so that after the client receives the response data, it is unable to directly obtain useful information based on the response data. Only the client can obtain the page data normally through dynamic rendering.

[0018] According to the method of the first aspect of the present invention, a font mapping module is called to map the content in the returned response body according to certain rules, and the mapping process does not affect the display of the web page content; wherein, before performing reverse proxy, a mapping rule is configured to map the keyword text into meaningless text, and the text after keyword mapping is inserted into the response body, and css code is introduced into the HTML code in the response body to load the font file of the mapping rule, so that the HTML page loaded by the client is loaded and displayed normally, while the content crawled by the crawler is meaningless text.

[0019] According to the method of the first aspect of the present invention, a user behavior identification module is called to identify a crawler by analyzing the client behavior; specifically, the method includes: Perform rule-based matching on the request header of the client request packet, identify the crawler by checking whether the fields in the request header contain the crawler's characteristic values ​​and whether the crawler's characteristic values ​​exist in the key fields, and count the corresponding IP into the crawler IP list; Perform behavioral clustering on the proxy logs, find records similar to crawler behaviors, and save the IP addresses of the records; Calculate the access speed to limit client access. By comparing the time before and after the client access, calculate the access frequency within a period of time. If the frequency exceeds the threshold, it will prompt that the access is too frequent and set a waiting time.

[0020] The second aspect of the present invention proposes a network asset anti-mapping system based on a website transparent proxy, the system comprising a reverse proxy module, a user behavior identification module, and a user request data processing module; wherein: The reverse proxy module includes a user input module and a user request data return module; the user input module is called to receive and cache the user's access request into the input buffer by using the reverse proxy in the website to be protected, and the user's input request header and request body are analyzed; the user request data return module is called to modify the website's return data packet after making a request to the website the user wants to visit, and then feedback it to the user; The user behavior identification module includes a first behavior classification module for classifying behaviors based on user request header features and a second behavior classification module for classifying behaviors based on user access behaviors; wherein: calling the first behavior classification module to analyze the request header content of the user's request and identify the access of abnormal users according to keywords and preset request headers; calling the second behavior classification module to count the access time from the same customer and calculate the access frequency, and perform access control according to the frequency; The user request data processing module is used to modify the content of the returned data packet response body, including a dynamic rendering module and a font mapping module; wherein: the dynamic rendering module is called to replace the keywords of the original website content in the returned data packet response body, and then the Javascript code is used for dynamic rendering to perform reverse replacement; the font mapping module is called to modify the returned original website content into custom text, and the font file is introduced for mapping, so that the web page is displayed normally, while the web page source code is in a garbled state.

[0021] The technical effects brought by the present invention include: The present invention uses a reverse proxy website to perform behavioral analysis on user requests to solve the problem of illegal access faced by the website and can solve the problem of detecting illegal crawlers faced by the website.

[0022] The present invention uses JS code dynamic rendering combined with a reverse proxy approach to replace the content of a protected website and then re-render using JS code, thereby solving the problem of website content being crawled for malicious attribution analysis.

[0023] The present invention uses font mapping to make the content of the protected website display normally, while meaningless garbled characters are presented in the webpage source code, thereby reducing the risk of the webpage content being crawled by dynamic crawlers for attribution analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A schematic diagram of the composition of a network asset anti-mapping architecture according to an embodiment of the present invention; Figure 2 A diagram showing the working principle of network asset anti-mapping according to an embodiment of the present invention; Figure 3 A schematic diagram of a reverse proxy mechanism according to an embodiment of the present invention; Figure 4 A schematic diagram of a load balancing principle according to an embodiment of the present invention; Figure 5 It is a flowchart of implementation based on response body content rewriting according to an embodiment of the present invention; Figure 6 The present invention is a flowchart of implementation based on confrontation content addition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0027] The present invention proposes a network asset anti-mapping method based on a website transparent proxy, which realizes network asset anti-mapping by calling a reverse proxy module, a user behavior recognition module, and a user request data processing module; the method specifically includes: Step S1, calling the reverse proxy module; the reverse proxy module includes a user input module and a user request data return module; wherein: Call the user input module, receive and cache the user's access request into the input buffer by using a reverse proxy in the website to be protected, and analyze the user's input request header and request body; Call the user request data return module, after making a request to the website the user wants to visit, modify the website's return data packet and feed it back to the user; Step S2, calling a user behavior identification module; the user behavior identification module includes a first behavior classification module that performs behavior classification based on user request header features and a second behavior classification module that performs behavior classification based on user access behavior; wherein: Calling the first behavior classification module to analyze the request header content of the user's request and identify the access of abnormal users according to keywords and preset request headers; Call the second behavior classification module to count the access time from the same customer and calculate the access frequency, and perform access control based on the frequency; Step S3, calling the user request data processing module; the user request data processing module is used to modify the content of the returned data packet response body, including a dynamic rendering module and a font mapping module; wherein: Call the dynamic rendering module to replace the keywords of the original website content in the response body of the returned data packet, and then use Javascript code dynamic rendering to reverse the replacement; The font mapping module is called to modify the returned original website content into custom text, and the font file is introduced for mapping, so that the web page is displayed normally, while the web page source code is in a garbled state.

[0028] According to the method of the first aspect of the present invention, the workflow of the reverse proxy module includes request forwarding, HTTP rewriting and redirection; wherein: the reverse proxy server receives a request from a client and forwards the request to a backend server for processing; and the client does not communicate directly with the backend server, but uses the reverse proxy server as an intermediary, and the reverse proxy rewrites the user's request to achieve URL redirection.

[0029] According to the method of the first aspect of the present invention, the request forwarding specifically includes: listening on the opened port, recording the request header of the client, constructing a new request packet with the reverse proxy information for the request body and sending it to the backend; after obtaining the return data from the backend, extracting the response body, combining it with the client's request header to construct a return data packet, and returning the return data packet to the client, thereby realizing request forwarding; and concurrently processing client requests to realize simultaneous access from multiple users.

[0030] According to the method of the first aspect of the present invention, the reverse proxy module also performs load balancing processing when executing the reverse proxy website; wherein: the requests to the back-end server are reduced by caching the static resources of the back-end server, thereby reducing the server burden and improving the response speed; at the same time, the reverse proxy acts as a load balancer, and distributes the client's requests to multiple back-end servers through polling and weight distribution, thereby improving the availability of the website.

[0031] According to the method of the first aspect of the present invention, the load balancing processing specifically includes: the reverse proxy reserves a portion of space as a buffer when each client accesses, for storing the static resources of the backend, and after obtaining the response body from the backend, stores the static resources therein in the buffer. In subsequent accesses by the client, it first searches for static resources in the cache, then sends a request to the backend, and after obtaining the response body, updates the buffer content.

[0032] According to the method of the first aspect of the present invention, the dynamic rendering module is called, and after obtaining the response data packet from the back end, the response body therein is extracted, and the original website page content is processed by js obfuscation, so that after the client receives the response data, it is unable to directly obtain useful information based on the response data. Only the client can obtain the page data normally through dynamic rendering.

[0033] According to the method of the first aspect of the present invention, a font mapping module is called to map the content in the returned response body according to certain rules, and the mapping process does not affect the display of the web page content; wherein, before performing reverse proxy, a mapping rule is configured to map the keyword text into meaningless text, and the text after keyword mapping is inserted into the response body, and css code is introduced into the HTML code in the response body to load the font file of the mapping rule, so that the HTML page loaded by the client is loaded and displayed normally, while the content crawled by the crawler is meaningless text.

[0034] According to the method of the first aspect of the present invention, a user behavior identification module is called to identify a crawler by analyzing the client behavior; specifically, the method includes: Perform rule-based matching on the request header of the client request packet, identify the crawler by checking whether the fields in the request header contain the crawler's characteristic values ​​and whether the crawler's characteristic values ​​exist in the key fields, and count the corresponding IP into the crawler IP list; Perform behavioral clustering on the proxy logs, find records similar to crawler behaviors, and save the IP addresses of the records; Calculate the access speed to limit client access. By comparing the time before and after the client access, calculate the access frequency within a period of time. If the frequency exceeds the threshold, it will prompt that the access is too frequent and set a waiting time.

[0035] First embodiment

[0036] like Figure 1 As shown, the network asset anti-mapping method based on the transparent proxy website of the present invention includes a reverse proxy module, a user behavior identification module and a user request data processing module.

[0037] The reverse proxy module includes a user input module and a user request data return module. In the user input module, the present invention first receives the user's access request into the input buffer of the present invention by using a reverse proxy on the website to be protected, and analyzes the user's input request header and request body. In the user output module, after making a request to the website that the user wants to visit, the return data packet of the website is modified and returned to the user.

[0038] The user behavior identification module includes a first behavior classification module based on user request header features and a second behavior classification module based on user access behavior (such as frequency): in the first behavior classification module based on user request header features, the request header content of the user's request is analyzed to identify abnormal user access based on keywords and preset request headers; in the second behavior classification module based on user access behavior (such as frequency), the present invention will count the access time from the same customer and calculate the access frequency, and implement access control based on the frequency.

[0039] The user request processing module mainly modifies the content of the response body of the returned data packet, and is divided into a dynamic rendering module and a font mapping module: in the dynamic rendering module, by replacing the keywords of the original website content in the returned response body, and then using Javascript code to dynamically render and modify it again, it is possible to modify the key content to confuse the text content of the network asset attribution analysis to reduce the mapping effect; in the font mapping module, by modifying the returned original website content into custom text and introducing font files for mapping, the content displayed on the web page is completely normal, while the source code of the web page is in a garbled state, reducing the risk of the web page content being used for attribution analysis.

[0040] Second embodiment (such as Figure 2 shown) 1. The reverse proxy of the website (such as Figure 3 shown) The present invention is based on the website reverse proxy, so the key technical points of the website reverse proxy are considered: request forwarding mechanism and HTTP rewriting and redirection: the reverse proxy server receives requests from the client, and then forwards these requests to the back-end server for processing. In this process, the client does not communicate directly with the back-end server, but uses the reverse proxy server as an intermediary, and the reverse proxy must rewrite the user's request and can realize URL redirection.

[0041] The request forwarding mechanism is implemented by listening on the open port, recording the client's request header, and using the reverse proxy information to form a new request packet for the request body and sending it to the backend; after obtaining the backend return data, the response body is extracted, combined with the client's request header to construct a return data packet, and returned to the client, thereby realizing the request forwarding mechanism. At the same time, high concurrency processing of client requests is required to ensure simultaneous access from multiple users.

[0042] 2. Consider load balancing when reverse proxying a website (such as Figure 4 shown) The purpose of the present invention is to enhance the security of the website without bringing too much burden to the back-end server, but it cannot interfere with normal user access. Therefore, when performing reverse proxy, it is necessary to consider reducing the requests to the back-end server by caching the static resources (such as pictures, CSS, JavaScript files, etc.) of the back-end server, thereby reducing the server burden and improving the response speed. At the same time, the reverse proxy can act as a load balancer to distribute the client's requests to multiple back-end servers, thereby improving the availability and performance of the website. This can be achieved through polling, weight distribution or other algorithms.

[0043] The design of the present invention improves the client access speed and reduces the access pressure of the backend by caching the static resources of the backend on the reverse proxy end. The specific scheme is as follows: the reverse proxy end reserves a part of space (buffer) for storing the static resources of the backend when each client accesses. After obtaining the response body from the backend, the static resources therein are stored in the buffer. In the subsequent access of the client, the static resources are searched in the cache first, and then requested to the backend. After obtaining the response body, the content of the buffer is updated.

[0044] 3. Security issues in the reverse proxy website process In the traditional reverse proxy mode, the reverse proxy can be used as an SSL terminal to process encrypted HTTPS connections. In this way, the communication between the client and the reverse proxy can be encrypted, and the communication between the reverse proxy and the back-end server can be encrypted or unencrypted as needed. At the same time, it brings hidden dangers to data security. The present invention requires that the client and the reverse proxy, the reverse proxy and the back-end all use encrypted communication to ensure communication security.

[0045] 4. Anti-mapping method based on response body content rewriting (such as Figure 5 shown) In order to counter the currently popular surveying and mapping methods, the present invention designs a method of hiding the original content of the website by rewriting the user request into code that needs to be executed and rendered by the browser: Specifically, before returning the client request to the client, the reverse proxy will process the response body content in the return package in some way, such as encryption, obfuscation, and rewriting it into dynamically generated content, and then reconstructing the return package and returning it to the client.

[0046] After obtaining the response data packet from the back end, the present invention extracts the response body therein and processes the original website page content through js obfuscation, so that after the client receives the response data, it is unable to directly obtain useful information based on the response data. Only the client can obtain the page data normally through dynamic rendering.

[0047] 5. Mapping interference methods based on adding adversarial content (such as Figure 6 shown) In order to deal with crawlers based on dynamic rendering, the present invention designs a font mapping method to map the content in the response body of the return package to another meaningless content according to certain rules, but it does not affect the display of the web page content. Specifically: when returning the response data packet to the client, the content in the response body is mapped according to the mapping algorithm, and the client then loads a font file. The font file can correctly present the mapped fonts, and these texts will produce erroneous mapping results after being thrown into the recognition algorithm by the mapping engine.

[0048] The present invention takes into account some non-static crawlers, and designs a font mapping method to map the content in the response body of the return package to another meaningless content according to certain rules, but does not affect the display of the web page content. The specific scheme is as follows: before performing reverse proxy, a series of mapping rules can be designed to map the keyword text into meaningless text, and these keywords are replaced with the mapped text and then inserted into the response body content, and then the CSS code is introduced into the HTML code in the response body, and the font file of the mapping rule is loaded, so that the HTML page loaded by the client can be loaded and displayed normally, while the content crawled by the crawler is meaningless text.

[0049] 6. Crawler identification based on client access behavior The present invention designs a behavior recognition module based on client access data. A large number of experiments show that crawler data packets carry a large number of features, such as: characteristic words in the request header, too frequent access, batch enumeration of form submission and other behaviors. In the present invention, malicious access can be identified based on the analysis of the client request packet, thereby discovering and blocking the access of crawlers or malicious users.

[0050] The present invention designs a method for identifying crawlers by analyzing client behaviors, and the specific scheme is as follows: first, a request header of a client request packet is matched based on rules, and the crawler is identified by finding whether the fields in the request header contain characteristic values ​​of the crawler and whether key fields such as cookie values ​​exist, and its IP is recorded and included in the crawler IP list; second, behavior clustering is performed based on the proxy log, and the records similar to the crawler behavior are found by performing behavior clustering on the proxy log, and the IP is recorded; third, client access is restricted by calculating the access speed (frequency), and the access frequency within a period of time is calculated by comparing the time before and after the client access. If the frequency is too high, a prompt of "too frequent access" is given and a period of time is set for the user to wait.

[0051] The second aspect of the present invention proposes a network asset anti-mapping system based on a website transparent proxy, the system comprising a reverse proxy module, a user behavior identification module, and a user request data processing module; wherein: The reverse proxy module includes a user input module and a user request data return module; the user input module is called to receive and cache the user's access request into the input buffer by using the reverse proxy in the website to be protected, and the user's input request header and request body are analyzed; the user request data return module is called to modify the website's return data packet after making a request to the website the user wants to visit, and then feedback it to the user; The user behavior identification module includes a first behavior classification module for classifying behaviors based on user request header features and a second behavior classification module for classifying behaviors based on user access behaviors; wherein: calling the first behavior classification module to analyze the request header content of the user's request and identify the access of abnormal users according to keywords and preset request headers; calling the second behavior classification module to count the access time from the same customer and calculate the access frequency, and perform access control according to the frequency; The user request data processing module is used to modify the content of the returned data packet response body, including a dynamic rendering module and a font mapping module; wherein: the dynamic rendering module is called to replace the keywords of the original website content in the returned data packet response body, and then the Javascript code is used for dynamic rendering to perform reverse replacement; the font mapping module is called to modify the returned original website content into custom text, and the font file is introduced for mapping, so that the web page is displayed normally, while the web page source code is in a garbled state.

[0052] The technical effects brought by the present invention include: The present invention uses a reverse proxy website to perform behavioral analysis on user requests to solve the problem of illegal access faced by the website and can solve the problem of detecting illegal crawlers faced by the website.

[0053] The present invention uses JS code dynamic rendering combined with a reverse proxy approach to replace the content of a protected website and then re-render using JS code, thereby solving the problem of website content being crawled for malicious attribution analysis.

[0054] The present invention uses font mapping to make the content of the protected website display normally, while meaningless garbled characters are presented in the webpage source code, thereby reducing the risk of the webpage content being crawled by dynamic crawlers for attribution analysis.

[0055] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above-mentioned embodiments only express several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present invention, several variations and improvements can be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the attached claims.

Claims

1. A network asset anti-mapping method based on website transparent proxy, characterized in that: The method implements network asset anti-mapping by calling a reverse proxy module, a user behavior identification module, and a user request data processing module; the method specifically includes: Step S1, calling the reverse proxy module; the reverse proxy module includes a user input module and a user request data return module; wherein: Call the user input module, receive and cache the user's access request into the input buffer by using a reverse proxy in the website to be protected, and analyze the user's input request header and request body; Call the user request data return module, after making a request to the website the user wants to visit, modify the website's return data packet and feed it back to the user; Step S2, calling a user behavior identification module; the user behavior identification module includes a first behavior classification module that performs behavior classification based on user request header features and a second behavior classification module that performs behavior classification based on user access behavior; wherein: Calling the first behavior classification module to analyze the request header content of the user's request and identify the access of abnormal users according to keywords and preset request headers; Call the second behavior classification module to count the access time from the same customer and calculate the access frequency, and perform access control based on the frequency; Step S3, calling the user request data processing module; the user request data processing module is used to modify the content of the returned data packet response body, including a dynamic rendering module and a font mapping module; wherein: Call the dynamic rendering module to replace the keywords of the original website content in the response body of the returned data packet, and then use Javascript code dynamic rendering to reverse the replacement; The font mapping module is called to modify the returned original website content into custom text, and the font file is introduced for mapping, so that the web page is displayed normally, while the web page source code is in a garbled state.

2. According to claim 1, a network asset anti-mapping method based on website transparent proxy is characterized in that: In the method, the workflow of the reverse proxy module includes request forwarding, HTTP rewriting and redirection; wherein: the reverse proxy server receives a request from a client and forwards the request to a backend server for processing; and the client does not communicate directly with the backend server, but uses the reverse proxy server as an intermediary, and the reverse proxy rewrites the user's request to achieve URL redirection.

3. According to claim 2, a network asset anti-mapping method based on website transparent proxy is characterized in that: The request forwarding specifically includes: listening on the opened port, recording the client's request header, and using the reverse proxy information to construct a new request packet for the request body and send it to the backend; after obtaining the backend return data, extracting the response body, combining it with the client's request header to construct a return data packet, and returning the return data packet to the client, thereby realizing request forwarding; and concurrently processing client requests to realize simultaneous access from multiple users.

4. According to the method of claim 1, the method is characterized in that: In the method, the reverse proxy module also performs load balancing processing when executing the reverse proxy website; wherein: the requests to the back-end server are reduced by caching the static resources of the back-end server, thereby reducing the server burden and improving the response speed; at the same time, the reverse proxy acts as a load balancer, and distributes the client's requests to multiple back-end servers through polling and weight distribution, thereby improving the availability of the website.

5. The method for resisting network asset mapping based on website transparent proxy according to claim 4 is characterized in that: The load balancing process specifically includes: the reverse proxy reserves a portion of space as a buffer when each client accesses the server, for storing the static resources of the backend; after obtaining the response body from the backend, the static resources therein are stored in the buffer; in subsequent accesses by the client, the static resources are searched for in the cache first, and then a request is sent to the backend; after obtaining the response body, the buffer content is updated.

6. The network asset anti-mapping method based on website transparent proxy according to claim 1 is characterized in that: In the method, the dynamic rendering module is called, and after obtaining the response data packet from the back end, the response body is extracted, and the original website page content is processed by js obfuscation, so that after the client receives the response data, it is unable to directly obtain useful information based on the response data. Only through dynamic rendering can the client obtain the page data normally.

7. A network asset anti-mapping method based on website transparent proxy according to claim 6, characterized in that: In the method, a font mapping module is called to map the content in the returned response body according to certain rules, and the mapping process does not affect the display of the web page content; wherein, before performing reverse proxy, a mapping rule is configured to map the keyword text into meaningless text, and the text after keyword mapping is inserted into the response body, and CSS code is introduced into the HTML code in the response body to load the font file of the mapping rule, so that the HTML page loaded by the client is loaded and displayed normally, while the content crawled by the crawler is meaningless text.

8. The network asset anti-mapping method based on website transparent proxy according to claim 7 is characterized in that: In the method, a user behavior identification module is called to identify crawlers by analyzing client behavior; specifically, the method includes: Perform rule-based matching on the request header of the client request packet, identify the crawler by checking whether the fields in the request header contain the crawler's characteristic values ​​and whether the crawler's characteristic values ​​exist in the key fields, and count the corresponding IP into the crawler IP list; Perform behavioral clustering on the proxy logs, find records similar to crawler behaviors, and save the IP addresses of the records; Calculate the access speed to limit client access. By comparing the time before and after the client access, calculate the access frequency within a period of time. If the frequency exceeds the threshold, it will prompt that the access is too frequent and set a waiting time.

9. A network asset anti-mapping system based on website transparent proxy, characterized in that: The system includes a reverse proxy module, a user behavior identification module, and a user request data processing module; wherein: The reverse proxy module includes a user input module and a user request data return module; the user input module is called to receive and cache the user's access request into the input buffer by using the reverse proxy in the website to be protected, and the user's input request header and request body are analyzed; the user request data return module is called to modify the website's return data packet after making a request to the website the user wants to visit, and then feedback it to the user; The user behavior identification module includes a first behavior classification module for classifying behaviors based on user request header features and a second behavior classification module for classifying behaviors based on user access behaviors; wherein: calling the first behavior classification module to analyze the request header content of the user's request and identify the access of abnormal users according to keywords and preset request headers; calling the second behavior classification module to count the access time from the same customer and calculate the access frequency, and perform access control according to the frequency; The user request data processing module is used to modify the content of the returned data packet response body, including a dynamic rendering module and a font mapping module; wherein: the dynamic rendering module is called to replace the keywords of the original website content in the returned data packet response body, and then the Javascript code is used for dynamic rendering to perform reverse replacement; the font mapping module is called to modify the returned original website content into custom text, and the font file is introduced for mapping, so that the web page is displayed normally, while the web page source code is in a garbled state.

Images