Data leakage prevention method and device, computer equipment and storage medium

Abstract

The invention discloses a data leakage prevention method and device, computer equipment and a storage medium, and belongs to the technical field of big data. According to the application, a pre-constructed log cloud platform is deployed at a cloud end, and the log cloud platform is accessed to a service system; deploying a request interceptor in the service system, and connecting the request interceptor with each interface; monitoring by using a docking interface, and when a user request is monitored, obtaining the request; obtaining log data matched with the request, and intercepting a log field corresponding to the identity information and the access address information from the log data; synchronizing the log field to a log cloud platform, and performing log anomaly detection on the log field by using a log anomaly detection model to obtain a detection result; and determining the data query permission of the user according to the detection result. The invention further relates to the technical field of block chains, and the log data are stored on the block chain nodes. According to the invention, the data leakage prevention capability of the system is improved, and the memory consumption of the system is reduced.

Description

Technical Field

[0001] The present application belongs to the field of big data technology, and specifically relates to a data leakage prevention method, device, computer equipment and storage medium. Background Art

[0002] For business systems, preventing user data from leaking has always been a matter of great concern to security and supervision. Business systems have a large number of interfaces, and the risk of user data leaks is also high. Attackers may register accounts with certain permissions, locate interfaces that can access certain user data, and then use parameter traversal to swipe system data. Therefore, when interfaces are not necessarily completely secure, business systems must have certain anti-swipe measures to prevent data leakage risks.

[0003] At present, there are data anti-brushing mechanisms in the industry, such as verification codes, IP restrictions, encryption and decryption mechanisms, but these mechanisms can only take such measures for some interfaces of the business system, and cannot achieve comprehensive interface anti-brushing. In addition, the interface access frequency limit is generally to use redis or token buckets to count the number of interface visits per unit time by a single user for a specific interface, and set the threshold for triggering alarms or interception, but it is basically impossible to prevent all URLs in the system from being swiped, because the cost is very high, because there are two dimensions of users and URLs, if redis or token buckets are used, it will bring a lot of memory consumption, and even lead to insufficient performance problems. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a data leakage prevention method, device, computer equipment and storage medium to solve the problem that the existing data anti-brushing mechanism is not able to realize comprehensive interface anti-brushing and is prone to memory consumption, resulting in insufficient performance.

[0005] In order to solve the above technical problems, the present application provides a data leakage prevention method, which adopts the following technical solutions:

[0006] A data leakage prevention method, comprising:

[0007] Deploy a pre-built log cloud platform on the cloud and connect it to the business system. The log cloud platform has a log anomaly detection model pre-deployed on it.

[0008] Deploy the request interceptor in the business system and connect the request interceptor to various interfaces of the business system;

[0009] Monitor each interface of the business system, and when a data transmission request initiated by a user is detected, obtain the data transmission request;

[0010] Obtaining log data matching the data transmission request, and intercepting a target log field from the log data, wherein the target log field is a log field corresponding to user identity information and access data address information;

[0011] Synchronize the target log field to the log cloud platform, use the log anomaly detection model to perform anomaly detection on the target log field, and obtain the log anomaly detection result;

[0012] Determine the user's data query permissions in the business system based on the log anomaly detection results.

[0013] Furthermore, before the step of deploying the pre-built log cloud platform on the cloud and connecting the log cloud platform to the business system, the following steps are also included:

[0014] Obtaining historical log data, wherein the historical log data is log data corresponding to historical data transmission requests initiated by the user;

[0015] Extracting a historical log field from the historical log data, wherein the historical log field is a log field in the historical log data corresponding to the user identity information and the access data address information;

[0016] Use historical log fields as model training data and use historical log fields to train the log anomaly detection model;

[0017] Deploy the log anomaly detection model to the log cloud platform.

[0018] Furthermore, the log anomaly detection model is trained based on a support vector machine algorithm, and the historical log fields are used as model training data. The steps of using the historical log fields to train the log anomaly detection model specifically include:

[0019] Label the historical log fields and generate log exception labels for the historical log fields;

[0020] Construct an initial detection model based on the support vector machine algorithm;

[0021] Import the historical log field into the initial detection model, learn the characteristics of the historical log field through the initial detection model, and output the log anomaly prediction results corresponding to the historical log field;

[0022] Based on the log anomaly prediction results and log anomaly labels, the hyperparameters of the initial detection model are tuned in combination with the cross-validation method to obtain the optimal parameter combination of the initial detection model;

[0023] Based on the optimal parameter combination, a trained log anomaly detection model is generated.

[0024] Furthermore, the steps of importing the historical log field into the initial detection model, learning the features of the historical log field through the initial detection model, and outputting the log anomaly prediction result corresponding to the historical log field specifically include:

[0025] Perform feature processing on the historical log field to obtain the historical log field features;

[0026] Use the kernel function built into the initial detection model to map historical log field features into a high-dimensional space;

[0027] The distribution of historical log field features is obtained in the high-dimensional space, and the log anomaly prediction results are generated according to the distribution of historical log field features in the high-dimensional space.

[0028] Furthermore, before the step of using the log anomaly detection model to perform anomaly detection on the target log field and obtaining the log anomaly detection result, the method further includes:

[0029] Get the user access whitelist and user access blacklist pre-configured in the log cloud platform;

[0030] Parse the target log field to obtain user identity information;

[0031] According to the user identity information, determine whether the data transmission request falls into the user access whitelist or the user access blacklist;

[0032] When a data transmission request falls into the user access whitelist, the user is granted corresponding data query permissions based on the records in the user access whitelist;

[0033] When a data transmission request falls into the user's access blacklist, the user's data transmission request is rejected.

[0034] Furthermore, the data transmission request is an HTTP request, and each interface of the business system is monitored. When a data transmission request initiated by a user is detected, the step of obtaining the data transmission request specifically includes:

[0035] Generate corresponding data monitoring rules according to the basic composition form of HTTP request, wherein the basic composition form of HTTP request includes request line, request header and request body;

[0036] Use data monitoring rules to monitor the data flow of each interface of the business system;

[0037] When there is a data stream whose format conforms to the basic component form of the HTTP request, it is determined that there is a user initiating a data transmission request, and the data transmission request is obtained.

[0038] Furthermore, the step of obtaining log data matching the data transmission request and intercepting the target log field from the log data specifically includes:

[0039] determining a log storage location for the log data based on the data transmission request;

[0040] Read log data according to the log storage location;

[0041] The log data is parsed, and the log fields corresponding to the user identity information and the access data address information are intercepted from the log parsing results to obtain the target log fields.

[0042] In order to solve the above technical problems, the embodiment of the present application further provides a data leakage prevention device, which adopts the following technical solution:

[0043] A data leakage prevention device, comprising:

[0044] The log cloud deployment module is used to deploy a pre-built log cloud platform on the cloud and connect the log cloud platform to the business system. The log cloud platform has a log anomaly detection model pre-deployed.

[0045] The interceptor deployment module is used to deploy the request interceptor in the business system and connect the request interceptor with various interfaces of the business system;

[0046] The request monitoring module is used to monitor various interfaces of the business system and obtain the data transmission request when a data transmission request initiated by a user is detected;

[0047] A log interception module, used to obtain log data matching the data transmission request, and intercept a target log field from the log data, wherein the target log field is a log field corresponding to the user identity information and the access data address information;

[0048] The anomaly detection module is used to synchronize the target log field to the log cloud platform, use the log anomaly detection model to perform anomaly detection on the target log field, and obtain the log anomaly detection result;

[0049] The data permission module is used to determine the user's data query permissions in the business system based on the log anomaly detection results.

[0050] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0051] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any of the above-mentioned data leakage prevention methods when executing the computer-readable instructions.

[0052] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0053] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of any of the above-mentioned data leakage prevention methods.

[0054] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0055] The present application discloses a data leakage prevention method, device, computer equipment and storage medium, which belongs to the field of big data technology. The present application deploys a pre-built log cloud platform in the cloud and connects it to the business system, thereby realizing comprehensive monitoring and protection of each interface. At the same time, by deploying a request interceptor in the business system, the data transmission request initiated by the user is monitored in real time, relevant log data is obtained, and the log anomaly detection model preset in the log cloud platform is used to perform log data anomaly analysis to effectively identify potential data leakage risks. The present application performs anomaly detection processing based on the log cloud platform, avoids performing heavy computing tasks locally in the business system, reduces memory consumption, and thus improves system performance. It not only realizes comprehensive protection of user data transmission, but also optimizes the efficiency of system resource utilization, solves the shortcomings of the existing data anti-brush mechanism in interface coverage and performance, and enhances the data leakage prevention capability and security of the business system. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 An exemplary system architecture diagram is shown in which the present application can be applied;

[0058] Figure 2 A flow chart of an embodiment of a data leakage prevention method according to the present application is shown;

[0059] Figure 3 A schematic structural diagram of an embodiment of a data leakage prevention device according to the present application is shown;

[0060] Figure 4 A schematic structural diagram of an embodiment of a computer device according to the present application is shown. DETAILED DESCRIPTION

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0062] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0063] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0064] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0065] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0066] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0067] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0068] It should be noted that the data leakage prevention method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the data leakage prevention device is generally set in the server / terminal device.

[0069] It should be understood that Figure 1 The numbers of terminal devices, networks and servers in the embodiment are merely illustrative. The above system may have any number of terminal devices, networks and servers according to implementation requirements.

[0070] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for preventing data leakage according to the present application. The method for preventing data leakage comprises the following steps:

[0071] S201, deploying a pre-built log cloud platform on the cloud, and connecting the log cloud platform to the business system, wherein a log anomaly detection model is pre-deployed in the log cloud platform.

[0072] Specifically, we first need to build a log cloud platform in a cloud environment. The platform is scalable and highly available, and can handle large amounts of log data from multiple business systems. The core function of the platform is anomaly detection. Therefore, during the deployment process, we need to build and integrate an intelligent log anomaly detection model in advance. The model can identify abnormal behaviors in logs based on machine learning algorithms or rule engines, usually including detection of user authentication failures, excessive access frequency, abnormal data transmission paths, etc. The log cloud platform needs to be closely integrated with the business system to ensure that it can receive the log data stream from the business system and perform real-time data analysis without affecting the normal operation of the business system. In addition, the log cloud platform must also have data backup, log management, and security protection mechanisms to ensure the security and privacy protection of all log data.

[0073] In a specific embodiment of the present application, it can be an expansion scheme based on the ELK (Elasticsearch, Logstash, Kibana) log cloud access system. Among them, the log cloud platform uses Elasticsearch as a data storage engine for efficiently storing and retrieving large amounts of log data; Logstash is responsible for receiving log data from the business system, filtering and converting data, and transferring the cleaned data to Elasticsearch; Kibana is used as a visualization tool to display log data, monitor and analyze. In this expansion scheme, the pre-deployed log anomaly detection model can be integrated into Elasticsearch, and real-time abnormal log detection is performed using custom rules or machine learning algorithms. This ELK architecture not only has high-performance log processing capabilities, but also can be seamlessly connected with the business system to achieve comprehensive management and efficient analysis of log data.

[0074] S202, deploying a request interceptor in the business system, and connecting the request interceptor to various interfaces of the business system.

[0075] Specifically, it is necessary to deploy request interceptors in the business system and connect the request interceptors to various interfaces of the business system. The function of the request interceptor is to intercept and preprocess the request when the user initiates a data request through the interface. The interceptor can be set on interfaces at different levels, including user login interface, data query interface, file download interface, etc., to ensure that all types of data transmission requests are effectively monitored. During the deployment process, the efficiency of the interceptor must be guaranteed, and it must not cause delays to normal data transmission or affect the overall performance of the system. The request interceptor can also perform some predefined operations, such as verifying user identity, checking the legitimacy of data requests, and generating relevant log records. By seamlessly connecting these request interceptors to the interfaces of the business system, it can be ensured that all transmission operations are under control and the security and integrity of the system are guaranteed.

[0076] S203, monitor each interface of the business system, and when a data transmission request initiated by a user is detected, obtain the data transmission request.

[0077] Specifically, the system needs to implement real-time monitoring of each interface. In order to ensure that no potential security risks are missed, the system will continuously monitor and track the interaction between users and interfaces. When a user initiates a data transmission request through an interface of the business system, the monitoring module immediately senses the request and intercepts all information related to the request, including the time of the request, user identity, and the target data address to be accessed. In this monitoring mode, no user request can bypass the system's detection, ensuring that all request behaviors are transparent. This monitoring mechanism not only helps detect potential security threats, but also records user operations.

[0078] S204, obtaining log data matching the data transmission request, and intercepting a target log field from the log data, wherein the target log field is a log field corresponding to the user identity information and the access data address information.

[0079] Specifically, when a user initiates a data transfer request, the system will extract log data that matches the request from the corresponding log record based on the request information. Usually, these log data include multiple fields such as the identity information of the initiator of the request, the specific data address to be accessed, the timestamp of the request, and the type of operation. In order to detect anomalies more accurately, the system will only intercept target log fields, that is, key fields that are directly related to user identity and access behavior. The selection of these fields is based on the analysis of potential security risks. For example, the user's authentication information may show abnormal login behavior, and the access data address may reveal abnormal access to sensitive data. In actual operation, the system needs to dynamically select the target fields that need to be monitored based on the business scenario to ensure that the analysis data of each request can reflect key security issues without affecting the performance of the system.

[0080] S205, synchronizing the target log field to the log cloud platform, using the log anomaly detection model to perform anomaly detection on the target log field, and obtaining the log anomaly detection result.

[0081] Specifically, the system will synchronize the target log fields obtained from the business system to the log cloud platform previously deployed in the cloud through a secure communication protocol. During the synchronization process, it is necessary to ensure the security and integrity of data transmission to prevent sensitive information from being stolen or tampered with during transmission. After the synchronization is completed, the anomaly detection model in the log cloud platform will perform anomaly detection on these log fields. The detection model identifies various abnormal behaviors in the log according to the set rules or pre-trained algorithm model. For example, the model can detect multiple failed login requests in a short period of time, remote logins of the same user in different time zones, abnormal data download behaviors, etc. Through in-depth analysis of log data, the system can accurately identify potential security threats and generate detailed detection results.

[0082] S206, determining the user's data query authority in the business system according to the log anomaly detection result.

[0083] Specifically, the system determines whether the user's current data query permissions need to be adjusted based on the anomaly detection results returned by the log cloud platform. If the detection results show that the user has abnormal behavior, such as frequent access to sensitive data or identity authentication anomalies, the system can take measures to limit the user's access rights. This permission adjustment can be temporary, such as requiring users to perform additional identity authentication, or permanent, such as directly restricting users' access to specific data. When adjusting permissions, the system needs to handle it flexibly according to the company's security policy to ensure that it can effectively prevent data leakage without affecting the normal user experience. At the same time, the system can also record these abnormal behaviors to ensure that the system maintains a high degree of transparency and traceability in data leakage prevention.

[0084] In the above embodiments, this application proposes an expansion scheme based on the ELK log cloud access system, which aims to increase log records, use machine learning for anomaly detection, and ultimately realize the functions of user access control and alarm. Based on the log cloud platform, anomaly detection processing is carried out to avoid performing heavy computing tasks locally in the business system, reduce memory consumption, and thus improve system performance. It not only realizes comprehensive protection of user data transmission, but also optimizes the efficiency of system resource utilization, solves the shortcomings of the existing data anti-brush mechanism in interface coverage and performance, and enhances the data leakage prevention capability and security of the business system.

[0085] Furthermore, before the step of deploying the pre-built log cloud platform on the cloud and connecting the log cloud platform to the business system, the following steps are also included:

[0086] Obtaining historical log data, wherein the historical log data is log data corresponding to historical data transmission requests initiated by the user;

[0087] Extracting a historical log field from the historical log data, wherein the historical log field is a log field in the historical log data corresponding to the user identity information and the access data address information;

[0088] Use historical log fields as model training data and use historical log fields to train the log anomaly detection model;

[0089] Deploy the log anomaly detection model to the log cloud platform.

[0090] In this embodiment, before deploying the log cloud platform, by acquiring historical log data, the system can collect historical data transmission requests initiated by users in the business system, as well as related log records generated by historical data transmission requests. Specifically, the historical log data contains important fields such as user identity information, data access address, and operation time. The system extracts historical log fields related to user identity and data access as key input data for model training. These historical log fields can reflect the user's daily behavior patterns and access habits. Then, the system uses these historical log fields to train the log anomaly detection model to ensure that the model can identify the difference between normal and abnormal behaviors. The trained model has the ability to identify potential abnormal behaviors by capturing patterns in user operations. After testing, the model is deployed to the cloud log platform for real-time monitoring of future log data.

[0091] Through the above steps, the system can build a more accurate log anomaly detection model based on real historical data, greatly improving the accuracy and sensitivity of anomaly detection.

[0092] Furthermore, the log anomaly detection model is trained based on a support vector machine algorithm, and the historical log fields are used as model training data. The steps of using the historical log fields to train the log anomaly detection model specifically include:

[0093] Label the historical log fields and generate log exception labels for the historical log fields;

[0094] Construct an initial detection model based on the support vector machine algorithm;

[0095] Import the historical log field into the initial detection model, learn the characteristics of the historical log field through the initial detection model, and output the log anomaly prediction results corresponding to the historical log field;

[0096] Based on the log anomaly prediction results and log anomaly labels, the hyperparameters of the initial detection model are tuned in combination with the cross-validation method to obtain the optimal parameter combination of the initial detection model.

[0097] Based on the optimal parameter combination, a trained log anomaly detection model is generated.

[0098] Support Vector Machine (SVM) is a machine learning algorithm that is widely used in classification and regression problems. SVM has the ability of multi-category classification and can classify log behaviors into different anomaly categories, thereby providing more sophisticated anomaly detection. The core idea of ​​SVM is to maximize the boundaries between different categories of data, which helps the model maintain good predictive ability on unknown data. In log anomaly detection, the SVM model can accurately identify anomalies in new and unseen log data. And by introducing the kernel function, SVM can map the original data into a higher-dimensional space, thereby finding a linear segmentation hyperplane in the new space, which is crucial for dealing with nonlinearly separable log anomaly detection problems.

[0099] In this embodiment, the log anomaly detection model is trained using a support vector machine (SVM) algorithm. During model training, the historical log fields are first labeled, that is, each historical log field is assigned a corresponding abnormal or normal label, thereby forming labeled training data. Based on these labeled log data, the system uses the support vector machine algorithm to build an initial detection model. The task of this model is to determine which log data may be abnormal by learning the characteristics of the historical log fields. Next, the system imports these historical log fields into the initial model. The model outputs the log anomaly prediction results corresponding to each log field by learning the data. These prediction results are compared with the previously labeled log anomaly labels to evaluate the accuracy of the model. In this process, the cross-validation method is used to tune the hyperparameters of the model. The cross-validation method divides the data into a training set and a validation set, and iterates the model parameters multiple times to ensure that the optimal hyperparameter combination is obtained, so that the performance of the model is more stable and accurate. Finally, based on these optimal parameter combinations, the system generates a trained and more accurate log anomaly detection model.

[0100] Through the above steps, the system can not only improve the prediction accuracy of the support vector machine model, but also ensure the robustness of the log anomaly detection model in actual scenarios, thereby effectively identifying abnormal log behavior and improving data leakage prevention capabilities.

[0101] Furthermore, the steps of importing the historical log field into the initial detection model, learning the features of the historical log field through the initial detection model, and outputting the log anomaly prediction result corresponding to the historical log field specifically include:

[0102] Perform feature processing on the historical log field to obtain the historical log field features;

[0103] Use the kernel function built into the initial detection model to map historical log field features into a high-dimensional space;

[0104] The distribution of historical log field features is obtained in the high-dimensional space, and the log anomaly prediction results are generated according to the distribution of historical log field features in the high-dimensional space.

[0105] In this embodiment, feature processing is performed on the historical log field, aiming to convert the original historical log data into features suitable for machine learning model processing. The content of feature processing includes standardizing and normalizing the original log data and removing noise data, ensuring that the features of different dimensions are calculated on the same scale, and the processed log field features can better reflect the user behavior pattern and the law of data access. On this basis, these historical log field features are mapped from the original space to a high-dimensional space through the kernel function built into the initial detection model. In the high-dimensional space, the support vector machine algorithm can more easily identify the distribution of data of different categories, especially the difference between abnormal log fields and normal log fields. In this high-dimensional space, the model finds the best decision boundary by analyzing the distribution of log field features, thereby generating corresponding log anomaly prediction results. This classification method after high-dimensional mapping can effectively capture complex nonlinear relationships and improve the recognition accuracy of abnormal behavior.

[0106] The kernel function of SVM is the key to realizing nonlinear classification. Through the kernel function, the original data can be mapped to a high-dimensional space, making the data linearly separable in the high-dimensional space. Commonly used kernel functions include linear kernel, polynomial kernel, Gaussian kernel, etc.

[0107] Through the above steps, the system uses kernel functions to map features to high-dimensional space, successfully enhancing the model's ability to recognize complex data features, thereby improving the accuracy and detection effect of log anomaly detection.

[0108] Furthermore, before the step of using the log anomaly detection model to perform anomaly detection on the target log field and obtaining the log anomaly detection result, the method further includes:

[0109] Get the user access whitelist and user access blacklist pre-configured in the log cloud platform;

[0110] Parse the target log field to obtain user identity information;

[0111] According to the user identity information, determine whether the data transmission request falls into the user access whitelist or the user access blacklist;

[0112] When a data transmission request falls into the user access whitelist, the user is granted corresponding data query permissions based on the records in the user access whitelist;

[0113] When a data transmission request falls into the user's access blacklist, the user's data transmission request is rejected.

[0114] In this embodiment, the system first obtains the pre-configured user access whitelist and blacklist from the log cloud platform. The users in the whitelist are usually trusted users authenticated by the system and have corresponding data access rights, while the blacklist contains users marked by the system as potential risks or confirmed to have bad behavior. Next, the system parses the acquired target log field and extracts the user's identity information, such as user name, IP address, device identification, etc. Based on these identity information, the system determines whether the specific data transmission request initiated by the user complies with the rules of the whitelist or blacklist. If the user's identity information matches the record in the whitelist, the system will grant it the corresponding data query permission and allow normal data transmission requests. If the user's identity falls into the blacklist, the user has illegal or malicious behavior, and the system will immediately reject the data transmission request, thereby effectively preventing potential security threats. In addition, the system can continuously update the whitelist and blacklist in dynamically changing business scenarios to ensure the accuracy and real-time nature of permission management.

[0115] Through the above steps, the system can effectively distinguish and manage the access rights of different users, prevent unauthorized users from accessing sensitive data, and thus enhance the data security and stability of the business system.

[0116] Furthermore, the data transmission request is an HTTP request, and each interface of the business system is monitored. When a data transmission request initiated by a user is detected, the step of obtaining the data transmission request specifically includes:

[0117] Generate corresponding data monitoring rules according to the basic composition form of HTTP request, wherein the basic composition form of HTTP request includes request line, request header and request body;

[0118] Use data monitoring rules to monitor the data flow of each interface of the business system;

[0119] When there is a data stream whose format conforms to the basic component form of the HTTP request, it is determined that there is a user initiating a data transmission request, and the data transmission request is obtained.

[0120] In this embodiment, the system generates corresponding data monitoring rules by analyzing the basic components of HTTP requests. The basic components of HTTP requests include request lines, request headers, and request bodies. The request lines usually contain request methods (such as GET, POST), target URLs, and HTTP protocol versions; the request headers contain metadata, such as user agent, cookies, Host, and other information; the request body contains the actual data payload, especially in POST requests. The system builds data monitoring rules based on these components to ensure that all standard HTTP requests can be identified and parsed. Next, the system applies these monitoring rules to various interfaces of the business system and continuously monitors data flows. When the data flow conforms to the format of the HTTP request, the system determines that the user has initiated a new data transmission request and extracts relevant information from it, such as user identity, target address accessed, and specific content of the request. This monitoring mechanism based on the HTTP protocol can effectively cover most data transmission behaviors in modern network communications, thereby ensuring comprehensive monitoring of various interfaces and data flows.

[0121] Through the above steps, the system can accurately capture user data transmission requests based on the HTTP protocol, realize efficient monitoring of data flows of various interfaces of the business system, and effectively improve the visibility and security of data transmission behavior.

[0122] Furthermore, the step of obtaining log data matching the data transmission request and intercepting the target log field from the log data specifically includes:

[0123] determining a log storage location for the log data based on the data transmission request;

[0124] Read log data according to the log storage location;

[0125] The log data is parsed, and the log fields corresponding to the user identity information and the access data address information are intercepted from the log parsing results to obtain the target log fields.

[0126] In this embodiment, the system first determines the storage location of the relevant log data based on the data transmission request initiated by the user. The storage location of the log data is usually divided according to the system configuration and can be stored in different servers, distributed databases or cloud storage platforms. By parsing the metadata information in the data transmission request, such as timestamp, IP address or request path, the system can locate the specific entry in the relevant log file or log database. Next, the system reads the corresponding log data according to the determined log storage location to ensure that the log data can be retrieved accurately and efficiently, avoiding interference from irrelevant data. The read log data contains a large amount of original information, so the system needs to parse it. The parsing step includes structuring the log data, identifying and extracting key log fields, which are usually closely related to user identity information and access data address information.

[0127] Through the above steps, the system can quickly and accurately obtain and parse log data, ensuring that the extracted log fields are targeted and efficient, thereby providing reliable data support for log anomaly detection and data security analysis.

[0128] In this embodiment, the present application discloses a data leakage prevention method, which belongs to the field of big data technology. The present application deploys a pre-built log cloud platform in the cloud and connects it to the business system, thereby realizing comprehensive monitoring and protection of each interface. At the same time, by deploying a request interceptor in the business system, the data transmission request initiated by the user is monitored in real time, relevant log data is obtained, and the log anomaly detection model preset in the log cloud platform is used to perform log data anomaly analysis to effectively identify potential data leakage risks. The present application performs anomaly detection processing based on the log cloud platform, avoids performing heavy computing tasks locally in the business system, reduces memory consumption, and thus improves system performance. It not only realizes comprehensive protection of user data transmission, but also optimizes the efficiency of system resource utilization, solves the shortcomings of the existing data anti-brush mechanism in interface coverage and performance, and enhances the data leakage prevention capability and security of the business system.

[0129] In this embodiment, the electronic device (eg, Figure 1 The server shown in the figure) can receive instructions or obtain data through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0130] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned log data information, the above-mentioned log data information can also be stored in a node of a blockchain.

[0131] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0132] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0133] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0134] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0135] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0136] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a data leakage prevention device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0137] like Figure 3 As shown, the data leakage prevention device 300 described in this embodiment includes:

[0138] The log cloud deployment module 301 is used to deploy a pre-built log cloud platform on the cloud and connect the log cloud platform to the business system, wherein a log anomaly detection model is pre-deployed in the log cloud platform;

[0139] The interceptor deployment module 302 is used to deploy the request interceptor in the business system and connect the request interceptor to various interfaces of the business system;

[0140] The request monitoring module 303 is used to monitor various interfaces of the business system and obtain the data transmission request when a data transmission request initiated by a user is detected;

[0141] The log interception module 304 is used to obtain the log data matching the data transmission request and intercept the target log field from the log data, wherein the target log field is the log field corresponding to the user identity information and the access data address information;

[0142] The anomaly detection module 305 is used to synchronize the target log field to the log cloud platform, perform anomaly detection on the target log field using the log anomaly detection model, and obtain the log anomaly detection result;

[0143] The data authority module 306 is used to determine the data query authority of the user in the business system according to the log anomaly detection result.

[0144] Furthermore, the data leakage prevention device 300 also includes a model training module, which is specifically used for:

[0145] Obtain historical log data, wherein the historical log data is log data corresponding to historical data transmission requests initiated by users; extract historical log fields from the historical log data, wherein the historical log fields are log fields in the historical log data corresponding to user identity information and access data address information; use the historical log fields as model training data, and use the historical log fields to train the log anomaly detection model; deploy the log anomaly detection model to the log cloud platform.

[0146] Furthermore, the log anomaly detection model is trained based on the support vector machine algorithm, and the model training module is further used to:

[0147] Label the historical log fields and generate log anomaly labels for the historical log fields; build an initial detection model based on the support vector machine algorithm; import the historical log fields into the initial detection model, learn the characteristics of the historical log fields through the initial detection model, and output the log anomaly prediction results corresponding to the historical log fields; according to the log anomaly prediction results and the log anomaly labels, and in combination with the cross-validation method, tune the hyperparameters of the initial detection model to obtain the optimal parameter combination of the initial detection model; based on the optimal parameter combination, generate a trained log anomaly detection model.

[0148] Furthermore, the model training module is further used to:

[0149] Perform feature processing on historical log fields to obtain historical log field features; use the built-in kernel function in the initial detection model to map the historical log field features to a high-dimensional space; obtain the distribution of historical log field features in the high-dimensional space, and generate log anomaly prediction results based on the distribution of historical log field features in the high-dimensional space.

[0150] Furthermore, the data leakage prevention device 300 further includes a pre-judgment module, which is specifically used for:

[0151] Obtain the user access whitelist and user access blacklist pre-configured in the log cloud platform; parse the target log field to obtain the user identity information; determine whether the data transmission request falls into the user access whitelist or the user access blacklist based on the user identity information; when the data transmission request falls into the user access whitelist, grant the user the corresponding data query permission based on the records in the user access whitelist; when the data transmission request falls into the user access blacklist, reject the user's data transmission request.

[0152] Furthermore, the data transmission request is an HTTP request, and the request monitoring module 303 is specifically used for:

[0153] Generate corresponding data monitoring rules according to the basic composition form of HTTP request, where the basic composition form of HTTP request includes request line, request header and request body; use data monitoring rules to monitor the data flow of each interface of the business system; when there is a data flow whose format conforms to the basic composition form of HTTP request, determine that a user has initiated a data transmission request, and obtain the data transmission request.

[0154] Furthermore, the log interception module 304 is specifically used for:

[0155] determining a log storage location for the log data based on the data transmission request;

[0156] Read log data according to the log storage location;

[0157] The log data is parsed, and the log fields corresponding to the user identity information and the access data address information are intercepted from the log parsing results to obtain the target log fields.

[0158] In this embodiment, the present application discloses a data leakage prevention device, which belongs to the field of big data technology. The present application deploys a pre-built log cloud platform in the cloud and connects it to the business system, thereby realizing comprehensive monitoring and protection of each interface. At the same time, by deploying a request interceptor in the business system, the data transmission request initiated by the user is monitored in real time, relevant log data is obtained, and the log anomaly detection model preset in the log cloud platform is used to perform log data anomaly analysis to effectively identify potential data leakage risks. The present application performs anomaly detection processing based on the log cloud platform, avoids performing heavy computing tasks locally in the business system, reduces memory consumption, and thus improves system performance. It not only realizes comprehensive protection of user data transmission, but also optimizes the efficiency of system resource utilization, solves the shortcomings of the existing data anti-brush mechanism in interface coverage and performance, and enhances the data leakage prevention capability and security of the business system.

[0159] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0160] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0161] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0162] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the data leakage prevention method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0163] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the data leakage prevention method.

[0164] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0165] In this embodiment, the present application discloses a computer device, which belongs to the field of big data technology. The present application deploys a pre-built log cloud platform in the cloud and connects it to the business system, thereby realizing comprehensive monitoring and protection of each interface. At the same time, by deploying a request interceptor in the business system, the data transmission request initiated by the user is monitored in real time, relevant log data is obtained, and the log anomaly detection model preset in the log cloud platform is used to perform log data anomaly analysis to effectively identify potential data leakage risks. The present application performs anomaly detection processing based on the log cloud platform, avoids performing heavy computing tasks locally in the business system, reduces memory consumption, and thus improves system performance. It not only realizes comprehensive protection of user data transmission, but also optimizes the efficiency of system resource utilization, solves the shortcomings of the existing data anti-brush mechanism in interface coverage and performance, and enhances the data leakage prevention capability and security of the business system.

[0166] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the data leakage prevention method as described above.

[0167] In this embodiment, the present application discloses a computer-readable storage medium, which belongs to the field of big data technology. The present application deploys a pre-built log cloud platform in the cloud and connects it to the business system, thereby realizing comprehensive monitoring and protection of each interface. At the same time, by deploying a request interceptor in the business system, the data transmission request initiated by the user is monitored in real time, relevant log data is obtained, and the log anomaly detection model preset in the log cloud platform is used to perform log data anomaly analysis to effectively identify potential data leakage risks. The present application performs anomaly detection processing based on the log cloud platform, avoids performing heavy computing tasks locally in the business system, reduces memory consumption, and thus improves system performance. It not only realizes comprehensive protection of user data transmission, but also optimizes the efficiency of system resource utilization, solves the shortcomings of the existing data anti-brush mechanism in interface coverage and performance, and enhances the data leakage prevention capability and security of the business system.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0169] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0170] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A data leakage prevention method, characterized in that: include: Deploy a pre-built log cloud platform in the cloud, and connect the log cloud platform to the business system, wherein a log anomaly detection model is pre-deployed in the log cloud platform; Deploy a request interceptor in the business system and connect the request interceptor to various interfaces of the business system; Monitor each interface of the business system, and when a data transmission request initiated by a user is detected, obtain the data transmission request; Acquire log data matching the data transmission request, and intercept a target log field from the log data, wherein the target log field is a log field corresponding to user identity information and access data address information; Synchronize the target log field to the log cloud platform, use the log anomaly detection model to perform anomaly detection on the target log field, and obtain the log anomaly detection result; The data query authority of the user in the business system is determined according to the log anomaly detection result.

2. The data leakage prevention method according to claim 1, characterized in that: Before the step of deploying the pre-built log cloud platform on the cloud and connecting the log cloud platform to the business system, the step further includes: Acquire historical log data, wherein the historical log data is log data corresponding to historical data transmission requests initiated by the user; Extracting a historical log field from the historical log data, wherein the historical log field is a log field in the historical log data corresponding to the user identity information and the access data address information; Using the historical log fields as model training data, and using the historical log fields to train the log anomaly detection model; The log anomaly detection model is deployed to the log cloud platform.

3. The data leakage prevention method according to claim 2, characterized in that: The log anomaly detection model is trained based on a support vector machine algorithm, and the step of using the historical log field as model training data and using the historical log field to train the log anomaly detection model specifically includes: Annotating the historical log field to generate a log exception label for the historical log field; Building an initial detection model based on the support vector machine algorithm; Importing the historical log field into the initial detection model, learning the features of the historical log field through the initial detection model, and outputting the log anomaly prediction result corresponding to the historical log field; According to the log anomaly prediction result and the log anomaly label, and in combination with a cross-validation method, the hyperparameters of the initial detection model are tuned to obtain an optimal parameter combination of the initial detection model; Based on the optimal parameter combination, a trained log anomaly detection model is generated.

4. The data leakage prevention method according to claim 3, characterized in that: The step of importing the historical log field into the initial detection model, learning the features of the historical log field through the initial detection model, and outputting the log anomaly prediction result corresponding to the historical log field specifically includes: Performing feature processing on the historical log field to obtain historical log field features; Mapping the historical log field features to a high-dimensional space using a kernel function built into the initial detection model; The distribution of the historical log field features is obtained in the high-dimensional space, and the log anomaly prediction result is generated according to the distribution of the historical log field features in the high-dimensional space.

5. The data leakage prevention method according to claim 1, characterized in that: Before the step of using the log anomaly detection model to perform anomaly detection on the target log field and obtaining the log anomaly detection result, the step further includes: Obtaining a user access whitelist and a user access blacklist pre-configured in the log cloud platform; Parsing the target log field to obtain the user identity information; Determining, according to the user identity information, whether the data transmission request falls into the user access whitelist or the user access blacklist; When the data transmission request falls into the user access whitelist, granting the user corresponding data query authority according to the record of the user access whitelist; When the data transmission request falls into the user access blacklist, the data transmission request of the user is rejected.

6. The data leakage prevention method according to claim 1, characterized in that: The data transmission request is an HTTP request, and each interface of the business system is monitored. When a data transmission request initiated by a user is detected, the step of obtaining the data transmission request specifically includes: Generate corresponding data monitoring rules according to the basic composition form of HTTP request, wherein the basic composition form of HTTP request includes request line, request header and request body; Use the data monitoring rules to monitor the data flow of each interface of the business system; When there is a data stream whose format conforms to the basic component form of the HTTP request, it is determined that a user has initiated a data transmission request, and the data transmission request is obtained.

7. The data leakage prevention method according to claim 1, characterized in that: The step of acquiring log data matching the data transmission request and intercepting a target log field from the log data specifically includes: determining a log storage location of the log data based on the data transmission request; Read the log data according to the log storage location; The log data is parsed, and the log field corresponding to the user identity information and the access data address information is intercepted from the log parsing result to obtain the target log field.

8. A data leakage prevention device, characterized in that: include: A log cloud deployment module, used to deploy a pre-built log cloud platform on the cloud and connect the log cloud platform to the business system, wherein a log anomaly detection model is pre-deployed in the log cloud platform; An interceptor deployment module, used to deploy a request interceptor in the business system and connect the request interceptor to various interfaces of the business system; A request monitoring module is used to monitor various interfaces of the business system, and when a data transmission request initiated by a user is monitored, obtain the data transmission request; A log interception module, used to obtain log data matching the data transmission request, and intercept a target log field from the log data, wherein the target log field is a log field corresponding to user identity information and access data address information; An anomaly detection module, used to synchronize the target log field to the log cloud platform, perform anomaly detection on the target log field using the log anomaly detection model, and obtain a log anomaly detection result; The data permission module is used to determine the data query permission of the user in the business system according to the log anomaly detection result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data leakage prevention method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data leakage prevention method according to any one of claims 1 to 7 are implemented.

Images