A proactive defense system and method against information loss

Abstract

The application provides an active defense system and method for preventing information loss, and belongs to the technical field of network data security; the active defense system is composed of a crawler behavior sensing module, a strategy scheduling module, a data processing module, an emergency data source module and a business recovery module; malicious crawlers are accurately identified through multi-dimensional features, and protection strategies are scheduled according to threat levels; in a low-risk scenario, sensitive information is temporarily tampered with in batches in the cache layer, and the original snapshot is backed up; in a medium-high risk scenario, the front-end interface routing is quickly switched to an independent emergency camouflage database to ensure that the crawler only obtains invalid data; after the attack is terminated, the real data in the buffer layer and the interface and business database mapping relationship are automatically restored. The application can actively defend against crawler attacks, implement hierarchical data protection, realize non-intrusive tampering and automatic recovery, block information theft from the source, ensure the security of real data, reduce false positives, and does not affect normal business operation, and is suitable for high-strength security protection of key information.

Description

Technical Field

[0001] This invention relates to the field of network data security, specifically to an active defense system and method for preventing information loss. Background Technology

[0002] With the widespread adoption of digital operations and online business, customer information such as names, contact details, identity information, address data, and transaction records has become a core business secret and important data asset for enterprises. Currently, enterprise data interaction often adopts a "front-end page request → back-end database retrieval → data return and display" model. Under this model, interfaces and databases are exposed to the outside world, making them extremely vulnerable to attacks by malicious web crawlers.

[0003] Traditional data protection and anti-crawler technologies mostly employ a passive interception model, with the specific process being: IP rate limiting → CAPTCHA verification → IP blocking → alarm logging. This model heavily relies on access blocking and cannot prevent data theft at its source, exhibiting numerous technical flaws and security vulnerabilities. Based on practical application and existing technology research, traditional anti-crawler and data protection models have the following prominent problems:

[0004] The protection is highly delayed and the leakage is irreversible: it can only block subsequent access and cannot intervene in the database requests that have already been initiated. The crawler has already completed the batch data crawling before the rate limiting and blocking take effect. Once customer information is leaked, it cannot be recovered.

[0005] High false positive rate, affecting normal business operations: Relying solely on IP access frequency to determine web crawlers can easily misjudge legitimate users who perform high-frequency operations (such as customer service bulk queries and administrator backend maintenance) as malicious web crawlers, leading to the interruption of normal business operations and a reduction in user experience;

[0006] The protection methods are limited and cannot be bypassed: relying solely on rate limiting, blocking, and CAPTCHA interception is insufficient to deal with distributed crawlers and simulated browser crawlers. The protection barriers are easily breached and have extremely poor applicability.

[0007] Lack of proactive data protection and insufficient security: It is impossible to process real customer information in the database. Once the interception is broken, the real data can be obtained completely. There is no emergency backup plan, and the risk of data loss is extremely high.

[0008] While some anti-scraping systems exist on the market, they generally suffer from low identification accuracy, passive interception, inability to actively intervene in data, and lack of tiered protection strategies, making it difficult to meet the high-intensity protection needs of enterprises' core customer information. Therefore, the industry urgently needs a customer information loss prevention technology that accurately identifies, actively protects, adapts to different levels, and can automatically recover lost information, addressing the pain points of traditional anti-scraping methods. Summary of the Invention

[0009] To address the problems of existing technologies, the present invention aims to provide an active defense system and method for preventing information loss. It employs a multi-dimensional feature joint identification and hierarchical active defense mechanism to actively defend against web crawler attacks and implement hierarchical data protection. It adopts dual protection measures of temporary tampering at the cache layer and independent emergency disguised database to achieve non-intrusive tampering and automatic recovery, thereby blocking information theft at the source and ensuring the security of real data.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] On the one hand, this invention provides an active defense system to prevent information loss, including a crawler behavior perception module, a strategy scheduling module, a data processing module, an emergency data source module, and a business recovery module;

[0012] The crawler behavior perception module is used to collect full-dimensional features of front-end interface requests in real time, identify malicious crawler behavior, and output alarm signals.

[0013] The strategy scheduling module is connected to the crawler behavior perception module. It is used to receive alarm signals and determine the crawler threat level, schedule the corresponding active protection strategy and issue execution instructions, and record the entire process operation log.

[0014] The data processing module, connected to the strategy scheduling module and the business master database, is used to execute low-risk protection strategies. It performs batch temporary tampering on customer-sensitive fields in the business master database stored in the cache layer according to data tampering rules, and automatically backs up the original data snapshot before tampering.

[0015] The emergency data source module is connected to the strategy scheduling module and the front-end interface. It is used to execute medium- and high-risk protection strategies and has a built-in independent emergency camouflage database. After receiving an instruction, the emergency data source module switches the front-end interface to route to the emergency camouflage database.

[0016] The business recovery module is connected to the data processing module and the emergency data source module. It is used to monitor the termination status of crawler behavior. The business recovery module can automatically restore the original data of the business master database stored in the cache layer and restore the mapping relationship between the front-end interface and the business master database.

[0017] Preferably, the crawler behavior perception module has a built-in multi-dimensional feature recognition model, which can combine IP access frequency, request parameter consistency, device identifier and UA information to jointly determine malicious crawlers. The crawler behavior perception module is configured with an IP blacklist / whitelist filtering mechanism.

[0018] Preferably, the batch tampering operation of the data processing module is only performed in the cache layer. The data tampering rules include: replacing the middle 4 digits of the mobile phone number with a random number, retaining the first 6 and last 4 digits of the ID card, and replacing the name with a virtual name. The data format and length remain unchanged after tampering, and only the core sensitive content is invalidated.

[0019] Preferably, the batch tampering operation of the data processing module is only performed in the cache layer, without modifying the physical storage of the business main database, and the data format and length remain unchanged after tampering, with only core sensitive content being invalidated.

[0020] According to another aspect of the present invention, an active defense method for preventing information loss is also provided, implemented based on the active defense system, comprising the following steps:

[0021] Step S1, System Initialization: Deploy the protection middleware, connect the front-end business interface and the main business database, configure the crawler identification threshold, customer sensitive field list, data tampering rules, emergency disguise database data and cooldown time, and enter the real-time protection state after importing the IP blacklist and whitelist.

[0022] Step S2, Crawler Identification: Real-time collection of IP addresses and multi-dimensional behavioral characteristics of front-end interface requests; determination of whether a crawler is malicious using a multi-dimensional identification model; if so, generation of an alarm signal.

[0023] Step S3, Strategy Scheduling: Receive alarm signals and determine the threat level of the crawler, schedule low-risk or medium-to-high-risk proactive protection strategies according to their level, and issue execution instructions;

[0024] Step S4, Tiered Protection: Low-risk crawlers execute a temporary batch tampering strategy for key information in the main database, while medium- and high-risk crawlers execute an emergency disguised database switching strategy to ensure that crawlers only obtain invalid data;

[0025] Step S5, Automatic Recovery: After detecting the termination of the crawler attack, the system automatically restores the cached data from the snapshot to the original real data or restores the mapping relationship between the front-end interface and the main database, clears the alarm markers and optimizes the identification model threshold, and the system returns to the real-time protection state.

[0026] Preferably, in step S2, the multidimensional behavioral features include: frequency, UA, and parameters.

[0027] More preferably, in step S2, the criteria for determining a malicious crawler are: the same IP continuously exceeds the preset access threshold, and there are typical crawler behavior characteristics such as fixed parameters, no valid login status, and high-frequency repeated requests, and IPs on blacklists and whitelists are excluded.

[0028] Preferably, in step S3, the threat level determination rule is as follows: if the first threshold ≤ the number of requests per minute per IP ≤ the second threshold, it is determined to be a low-risk crawler, triggering a temporary batch tampering strategy for key information in the main database; if the number of requests per minute per IP > the second threshold, it is determined to be a medium-to-high-risk crawler, triggering an emergency disguised database switching strategy.

[0029] More preferably, the first threshold is 30 times / minute and the second threshold is 60 times / minute.

[0030] Preferably, in step S5, the condition for determining the termination of the crawler attack is: the abnormal IP continuously reaches the preset cooldown time without any abnormal requests, and low-risk and medium-to-high-risk crawlers are adapted to different cooldown times.

[0031] The beneficial effects of this invention are as follows:

[0032] This invention completely changes the shortcomings of traditional anti-crawler passive interception by using multi-dimensional feature joint identification and hierarchical active defense mechanism. It can block malicious crawlers from stealing real business data from the source. Even if the crawler breaks through the access control layer, it can only obtain invalid information that is compliant in format but has no value. It effectively prevents the leakage of sensitive customer information and significantly improves the security protection capability of core data assets.

[0033] This invention employs a dual protection approach: temporary tampering at the cache layer and an independent emergency spoofed database. Data tampering does not affect the physical storage of the main business database. After the attack is terminated, the original data and interface routes can be automatically restored. The entire process is non-intrusive, damage-free, and without business interruption. At the same time, it accurately distinguishes between legitimate high-frequency users and malicious crawlers, significantly reducing the false positive rate and ensuring the stable operation of normal business. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the system logic architecture of the present invention.

[0035] Figure 2 This is a flowchart illustrating the active defense method of the present invention. Detailed Implementation

[0036] In the description of this embodiment, it should be noted that the terms "upper", "lower", "center", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of the invention is usually placed in when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0037] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0038] This embodiment provides a detailed description of the proactive defense system and method for preventing information loss. The system is deployed as a non-intrusive protection middleware between the front-end business interface and the main business database. The specific technology stack is constructed using OpenResty gateway service, Lua script execution unit, Redis cache storage unit, Spring Boot scheduling framework, emergency disguise database, and Canal data synchronization component.

[0039] The proactive defense system for preventing information loss consists of a crawler behavior perception module, a strategy scheduling module, a data processing module, an emergency data source module, and a business recovery module. These modules work together to complete the entire defense process, including malicious crawler identification, threat level determination, tiered proactive protection, and automatic business recovery.

[0040] The crawler behavior awareness module relies on the OpenResty gateway service and Lua script execution unit to realize real-time access to front-end requests and full-dimensional feature collection. It uses the Redis cache storage unit to complete the statistics and recording of IP access frequency, UA information, device identification, request parameter consistency and login status. The module has a built-in multi-dimensional feature recognition model and IP blacklist and whitelist filtering mechanism, which can accurately identify malicious crawlers by combining multiple behavioral features, effectively exclude legitimate high-frequency access users, and output alarm signals in a timely manner after confirming crawler behavior.

[0041] The strategy scheduling module is built around the Spring Boot scheduling framework to establish a scheduling center. It receives alarm signals uploaded by the crawler behavior perception module and determines the crawler threat level according to the crawler threat level determination rules. The module can automatically schedule matching protection strategies based on different threat levels and issue execution instructions to the data processing module or emergency data source module. At the same time, it records complete logs of the entire process to meet the requirements of security auditing and traceability.

[0042] Data processing module: It directly interfaces with the main business database and uses Redis cache storage unit and database Binlog log component to automatically back up the original snapshot of sensitive data. When dealing with low-risk crawler scenarios, the module only performs batch temporary tampering on sensitive information such as customer name, mobile phone number, ID card number, and address at the cache layer without changing the physical storage of the main business database. The tampered data retains its original format and length, and only the core sensitive content is invalidated, so that the crawler can only obtain invalid data that is compliant in format but has no actual value.

[0043] Emergency Data Source Module: This module deploys an independent emergency masquerading database. The table structure, field format, and data length of this database are completely consistent with the main business database. The database uses the Canal data synchronization component to verify and synchronize structural consistency in real time. Simulated customer data with no real connection is pre-installed in the database. After receiving the policy scheduling instruction, the module can switch the interface route (in milliseconds) through the OpenResty gateway service and Lua script execution unit, directing all requests from abnormal IPs to the emergency masquerading database, thus achieving strong isolation and protection of real business data.

[0044] Business recovery module: This module connects to both the data processing module and the emergency data source module. It continuously monitors the request status of abnormal IPs through the expiration monitoring mechanism of the Redis cache storage unit. When there are no requests from abnormal IPs within the preset cooldown period, the crawler behavior is determined to be terminated and the recovery operation is automatically executed. In low-risk scenarios, the original data in the main database is restored from the data snapshot. In medium- and high-risk scenarios, the normal mapping relationship between the front-end interface and the business main database is restored. After the recovery is completed, the alarm flag is cleared and the system returns to the real-time protection state.

[0045] The proactive defense method to prevent information loss is implemented based on the above system and is executed in the following steps:

[0046] Step S1 System Initialization

[0047] This step involves deploying a protection middleware, using the OpenResty gateway service in conjunction with a Lua script execution unit to access the front-end business interface and complete the connection with the main business database. Configurations are made according to the specific business scenario:

[0048] Crawler level identification rules: Access with the number of requests per minute from a single IP in the range of the first threshold (30 times / minute) to the second threshold (60 times / minute) is judged as a low-risk crawler, and access exceeding the second threshold (60 times / minute) is judged as a medium-to-high-risk crawler.

[0049] List of sensitive customer fields: customer name, mobile phone number, ID card number, address;

[0050] Data tampering rules: The middle 4 digits of the mobile phone number are replaced with random numbers, the first 6 and last 4 digits of the ID card are retained, and the name is replaced with a virtual name. The data format and length remain unchanged after tampering, and only the core sensitive content is invalidated.

[0051] Emergency camouflage database: The table structure is completely consistent with the main database, with pre-set simulated data, and the structure is synchronized through the data synchronization tool Canal;

[0052] Cooling duration based on risk level: 10 minutes for low-risk areas, 15 minutes for medium-to-high-risk areas;

[0053] Simultaneously, the IP blacklist and whitelist are imported, and the multi-dimensional feature recognition model is initialized. After the system completes its self-check, it officially enters the real-time protection state.

[0054] Step S2: Crawler Identification

[0055] After entering real-time protection, the crawler behavior perception module continuously collects multi-dimensional behavioral characteristics of front-end interface requests, such as IP address, access frequency, UA information, request parameters, and login status. It makes a comprehensive judgment through a multi-dimensional feature recognition model. When the same IP continuously exceeds the preset access threshold and has typical crawler characteristics such as fixed parameters, no valid login status, and high-frequency repeated requests, and after excluding IPs within the blacklist and whitelist range, it can be determined as a malicious crawler and an alarm signal can be generated.

[0056] Step S3: Strategy Scheduling

[0057] After receiving the alarm signal, the policy scheduling module determines the threat level according to the crawler level identification rules. Accesses with 30 to 60 requests per minute from a single IP are classified as low-risk crawlers, while accesses exceeding 60 requests per minute are classified as medium- to high-risk crawlers. Based on the determination results, the module schedules the corresponding proactive protection policy and issues execution instructions to the corresponding execution module to ensure that the protection measures are accurately matched with the threat level.

[0058] Step S4 Graded Protection

[0059] For low-risk crawlers, the data processing module first automatically backs up the original data snapshot of the main business database using the Redis cache storage unit and the database Binlog log component. Then, it batch-modifies sensitive information in the cache layer according to preset data tampering rules and a list of sensitive customer fields, so that the crawler obtains invalid data that is legally formatted but has no actual value. For medium- and high-risk crawlers, the emergency data source module quickly switches interface routes through the OpenResty gateway service and the Lua script execution unit, directing all abnormal IP requests to the emergency disguised database. The crawler cannot access real business data throughout the process, thus preventing information leakage at the source.

[0060] Step S5 Automatic Recovery

[0061] The business recovery module continuously monitors the request status of abnormal IPs based on Redis expiration keys. When there are no abnormal requests from abnormal IPs within a preset tiered cooling period, the crawler attack is determined to have terminated. In low-risk scenarios, the module restores the cached data from the snapshot to the original real data. In medium- and high-risk scenarios, it restores the normal mapping relationship between the front-end interface and the main business database. After the restoration is completed, the alarm flags are cleared and the identification model thresholds are optimized. The system returns to real-time protection status and continues to provide security protection capabilities.

[0062] Although embodiments of the present invention have been described in the specification, these embodiments are merely illustrative and should not be construed as limiting the scope of protection of the present invention. Various omissions, substitutions, and modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A proactive defense system for preventing information loss, characterized in that, include: The module includes a crawler behavior awareness module, a strategy scheduling module, a data processing module, an emergency data source module, and a business recovery module. The crawler behavior perception module is used to collect full-dimensional features of front-end interface requests in real time, identify malicious crawler behavior, and output alarm signals. The strategy scheduling module is connected to the crawler behavior perception module. It is used to receive alarm signals and determine the crawler threat level, schedule the corresponding active protection strategy and issue execution instructions, and record the entire process operation log. The data processing module, connected to the strategy scheduling module and the business master database, is used to execute low-risk protection strategies. It performs batch temporary tampering on customer-sensitive fields in the business master database stored in the cache layer according to data tampering rules, and automatically backs up the original data snapshot before tampering. The emergency data source module is connected to the strategy scheduling module and the front-end interface. It is used to execute medium- and high-risk protection strategies and has a built-in independent emergency camouflage database. After receiving an instruction, the emergency data source module switches the front-end interface to route to the emergency camouflage database. The business recovery module is connected to the data processing module and the emergency data source module. It is used to monitor the termination status of crawler behavior. The business recovery module can automatically restore the original data of the business master database stored in the cache layer and restore the mapping relationship between the front-end interface and the business master database.

2. The active defense system for preventing information loss as described in claim 1, characterized in that, The crawler behavior perception module has a built-in multi-dimensional feature recognition model, which can combine IP access frequency, request parameter consistency, device identifier and UA information to jointly determine malicious crawlers. The crawler behavior perception module is configured with an IP blacklist and whitelist filtering mechanism.

3. The active defense system for preventing information loss as described in claim 1, characterized in that, The emergency spoofing database in the emergency data source module has the same table structure, field format, and data length as the main business database. It stores simulated customer data that has no real connection to the main database, and the emergency spoofing database verifies the structural consistency with the main database in real time through a data synchronization tool.

4. The active defense system for preventing information loss as described in claim 1, characterized in that, The batch tampering operation of the data processing module is only performed in the cache layer. The data tampering rules include: replacing the middle 4 digits of the mobile phone number with a random number, retaining the first 6 and last 4 digits of the ID card, and replacing the name with a virtual name. The data format and length remain unchanged after tampering, and only the core sensitive content is invalidated.

5. A proactive defense method for preventing information loss, implemented based on the system described in any one of claims 1 to 4, characterized in that, Includes the following steps: Step S1, System Initialization: Deploy the protection middleware, connect the front-end business interface and the main business database, configure the crawler identification threshold, customer sensitive field list, data tampering rules, emergency disguise database data and cooldown time, and enter the real-time protection state after importing the IP blacklist and whitelist. Step S2, Crawler Identification: Real-time collection of IP addresses and multi-dimensional behavioral characteristics of front-end interface requests; determination of whether a crawler is malicious using a multi-dimensional identification model; if so, generation of an alarm signal. Step S3, Strategy Scheduling: Receive alarm signals and determine the threat level of the crawler, schedule low-risk or medium-to-high-risk proactive protection strategies according to their level, and issue execution instructions; Step S4, Tiered Protection: Low-risk crawlers execute a temporary batch tampering strategy for key information in the main database, while medium- and high-risk crawlers execute an emergency disguised database switching strategy to ensure that crawlers only obtain invalid data; Step S5, Automatic Recovery: After detecting the termination of the crawler attack, the system automatically restores the cached data from the snapshot to the original real data or restores the mapping relationship between the front-end interface and the main database, clears the alarm markers and optimizes the identification model threshold, and the system returns to the real-time protection state.

6. The active defense method for preventing information loss as described in claim 5, characterized in that, In step S2, the multidimensional behavioral features include: frequency, UA, and parameters.

7. The active defense method for preventing information loss as described in claim 6, characterized in that, In step S2, the criteria for determining a malicious crawler are: the same IP continuously exceeds the preset access threshold, and there are typical crawler behavior characteristics such as fixed parameters, no valid login status, and high-frequency repeated requests, and IPs on blacklists and whitelists are excluded.

8. The active defense method for preventing information loss as described in claim 5, characterized in that, In step S3, the crawler threat level determination rule is as follows: if the first threshold ≤ number of requests per minute per IP ≤ second threshold, it is determined to be a low-risk crawler; if the number of requests per minute per IP > second threshold, it is determined to be a medium-to-high-risk crawler.

9. The active defense method for preventing information loss as described in claim 8, characterized in that, The first threshold is 30 times / minute, and the second threshold is 60 times / minute.

10. The active defense method for preventing information loss as described in claim 5, characterized in that, In step S5, the condition for determining the termination of the crawler attack is: the abnormal IP continuously reaches the preset cooldown time without any abnormal requests, and low-risk and medium-to-high-risk crawlers are adapted to different cooldown times.

Images