Topic model-based database privacy disclosure automatic detection method and system

Abstract

According to the automatic database privacy disclosure detection method and system based on the topic model, by dynamically updating topic and word distribution, combining gradient optimization and learning rate adjustment factors and capturing time dynamic characteristics of data and behaviors, the model can more accurately recognize an attack mode and adapt to constantly changing attack modes. According to the method, gradient optimization and adaptive learning rate adjustment are introduced, so that the model can adaptively adjust the step length in the training process, the database activity is monitored in real time, an alarm is given, and the real-time performance and the adaptability of detection are ensured. According to the method, a complex attack mode is recognized from multi-dimensional'topic-word 'and'topic-document' distribution, # imgabs0 # is introduced as a gradient item, the topic distribution parameter updating process is optimized, the method depends on the expected probability of words, the optimization direction of a target function can be better captured, the recognition capacity for complex attacks is enhanced, and the method is suitable for being applied to the field of complex attack recognition. The specific instruction and theme distribution in the malicious code can be accurately identified.

Description

Technical Field

[0001] The invention relates to a method and system for automatically detecting database privacy leakage based on a topic model, and belongs to the field of database security. Background Art

[0002] As database-related platforms become more functional and data volumes grow, they will indirectly lead to security risks in the database, and companies and organizations will pay great attention to data platform security and privacy protection. Databases often contain personal information, business secrets, and other sensitive data. The value and sensitivity of these data make them targets for hackers, internal threats, and data leaks. In addition, during data sharing and large-scale data analysis, there are more touchpoints and potential vulnerabilities when data is transmitted, processed, and analyzed, making it more susceptible to privacy leaks. Once a privacy leak occurs, it will pose a serious threat to personal privacy and corporate secrets. Therefore, in order to effectively protect sensitive information in the database, automatic detection of database privacy leaks becomes particularly critical.

[0003] At present, research related to platform privacy leakage detection has made great progress, such as statistical script attack vulnerabilities using static analysis and dynamic analysis detection tools, analyzing static data flows and dynamic data flows to find security vulnerabilities in platform applications, but this method is sometimes difficult to deal with complex internal threats and external attacks. Use data desensitization technology to blur sensitive information to protect privacy, but this method will cause the data to lose some details, affecting the integrity of the data and the analysis results. In the process of data analysis and sharing, differential privacy technology is applied to protect the privacy of individual data by adding noise, but this method requires higher computing resources.

[0004] In big data platforms, malicious code attacks often lead to sensitive information leakage. This type of code uses reverse engineering technology to convert portable files into assembly language code, thereby parsing sensitive data. In addition, malicious code attack methods are becoming increasingly diverse, and attackers use zero-day vulnerabilities, social engineering, and spear phishing to break through traditional security defenses. The targets of attacks have also become more complex, no longer limited to a single system, but targeting the entire network environment, using multiple vulnerabilities to conduct joint attacks. In order to deal with these threats, it is crucial to develop and implement effective malicious code detection technologies that can detect and prevent malicious code intrusions in a timely manner, thereby effectively reducing the risk of privacy leakage in databases and big data platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 It is the system flow chart. Summary of the invention

[0006] In view of the problems existing in the above existing technologies on the database platform, this application proposes an automatic detection method for database privacy leakage based on the topic model, which optimizes the distribution of topics and words by improving variational inference and ELBO objective function, identifies specific instructions and topic distribution in malicious code, learns malicious behavior patterns, and realizes dynamic privacy leakage detection and alarm. It includes the following steps:

[0007] (1) Constructing data time series;

[0008] (2) Model definition: Construct a topic model, treat malicious code samples as document collections, and assembly instructions as vocabulary. Analyze the “topic-word” and “topic-document” distributions to extract high-frequency malicious instructions.

[0009] (3) Topic model optimization and inference: Complete parameter optimization and inference, approximate the true posterior distribution, and extract malicious code features from it.

[0010] (4) Malicious code feature detection: Real-time analysis and topic model update, continuous monitoring of database access and system activities, and triggering an alarm mechanism to notify relevant security personnel when potential privacy leakage behavior is detected.

[0011] Furthermore, step (2) comprises the following steps:

[0012] (1) From the topic distribution of document d at time t, it is θ d,t We can get a topic z by sampling d,n,t ;

[0013] (2) According to the theme d,n,t , from the word distribution of the topic β k,t Sampling word w d,n,t .

[0014] Further, step (3) comprises the following steps:

[0015] 3.1 Calculation in, The objective function EIBO is d,t The gradient of t is the sum of squares of historical gradients, ∑ n q(z d,n ) is the expected probability that a word belongs to the topic;

[0016] 3.2 Calculation where q(z d,n =k) ​​is the expected probability that the word belongs to topic k, ferq(w (d,n) ) is the word w (d,n) The frequency in document d, t is the current time, t′ is the previous time point;

[0017] 3.3 Calculation of variational objective function Where; D is the total number of documents, T is the total number of time, K is the total number of topics, N d is the total number of words in document d, q(z dn =k) ​​is the probability that word n ​​belongs to topic k in document d, Submit word w for topic k dn The generation probability, θ d,t,k is the distribution of topic k of document d at time point t, α t is the prior parameter of the document-topic distribution, η t is the prior parameter of the topic-word distribution, Dir(θ d,t |α t ) is the topic distribution θ of document d at time t d,t Dirichlet distribution, Dir(β k,t |η t ) is the word distribution β of document d at time t k,t Dirichlet distribution, q(θ d,t ) and q(β k,t ) are all variational distributions.

[0018] Based on the above method, this application proposes a database privacy leakage automatic detection system based on topic model, which optimizes the distribution of topics and words by improving variational inference and ELBO objective function, identifies specific instructions and topic distribution in malicious code, learns malicious behavior patterns, and realizes dynamic privacy leakage detection and alarm. Specifically, the system includes:

[0019] (1) Build a data time series module;

[0020] (2) Model definition module: This module builds a topic model, treats malicious code samples as document collections, and assembly instructions as vocabulary. It analyzes the “topic-word” and “topic-document” distributions and extracts high-frequency malicious instructions.

[0021] (3) Topic model optimization and inference module: This module completes parameter optimization and inference, approximates the true posterior distribution, and extracts malicious code features from it.

[0022] (4) Malicious code feature detection module: This module performs real-time analysis and updates the topic model, continuously monitors database access and system activities, and triggers an alarm mechanism to notify relevant security personnel when potential privacy leakage behavior is detected.

[0023] Furthermore, the model definition module takes the following steps:

[0024] (1) From the topic distribution of document d at time t, it is θ d,t We can get a topic z by samplingd,n,t ;

[0025] (2) According to the theme d,n,t , from the word distribution of the topic β k,t Sampling word w d,n,t .

[0026] Furthermore, the topic model optimization and inference module adopts the following steps:

[0027] 3.1 Calculation in, The objective function EIBO is d,t The gradient of t is the sum of squares of historical gradients, ∑ n q(z d,n ) is the expected probability that a word belongs to the topic;

[0028] 3.2 Calculation where q(z d,n =k) ​​is the expected probability that the word belongs to topic k, ferq(w (d,n) ) is the word w (d,n) The frequency in document d, t is the current time, t′ is the previous time point;

[0029] 3.3 Calculation of variational objective function Where; D is the total number of documents, T is the total number of time, K is the total number of topics, N d is the total number of words in document d, q(z dn =k) ​​is the probability that word n ​​belongs to topic k in document d, Submit word w for topic k dn The generation probability, θ d,t,k is the distribution of topic k of document d at time point t, α t is the prior parameter of the document-topic distribution, η t is the prior parameter of the topic-word distribution, Dir(θ d,t |α t ) is the topic distribution θ of document d at time t d,t Dirichlet distribution, Dir(β k,t |η t ) is the word distribution β of document d at time t k,t Dirichlet distribution, q(θ d,t ) and q(β k,t ) are all variational distributions.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] 1. Traditional methods usually rely on static analysis and predefined rules, which are difficult to deal with complex internal threats and dynamically changing external attacks. This application captures the temporal dynamic characteristics of data and behavior by dynamically updating the topic and word distribution, combining gradient optimization and learning rate adjustment factors, so that the model can more accurately identify attack patterns and adapt to changing attack patterns.

[0032] 2. In order to solve the problems of delayed updates and difficulty in rapid adjustment in existing technologies when responding to new attacks, this application introduces gradient optimization and adaptive learning rate adjustment to enable the model to adaptively adjust the step size during training, monitor database activities in real time and issue alarms, ensuring the real-time and adaptability of detection.

[0033] 3. Compared with traditional methods that rely on single-dimensional analysis, this application identifies complex attack patterns from multiple dimensions of "topic-word" and "topic-document" distribution, and introduces As a gradient term, optimizing the topic distribution parameter update process not only relies on the expected probability of words, but also better captures the optimization direction of the objective function, enhances the recognition ability of complex attacks, and can accurately identify specific instructions and topic distributions in malicious codes. DETAILED DESCRIPTION

[0034] Example 1

[0035] In order to avoid the leakage of personal privacy and corporate secrets caused by malicious code attacks, this application proposes an automatic detection method for database privacy leakage based on topic model. This method realizes database privacy protection through the following process. The system flow chart is as follows: Figure 1 shown.

[0036] 1. Construct data time series:

[0037] Time window: Split the data into time windows (such as days, weeks, or months).

[0038] Document construction: The data in each time window is constructed into a document collection. Documents are constructed according to different database activities (such as query logs and operation records). Each document represents the data in a time window.

[0039] 2. Model definition:

[0040] Construct a topic model for malicious code feature extraction, count all assembly instructions in the malicious sample training set, and build a dictionary containing all words. Next, use unsupervised learning. Through this model, malicious code samples can be regarded as a document collection, and each document (i.e., malicious code sample) consists of a series of words (i.e., assembly instructions). In the model, each topic k represents a specific type of malicious behavior pattern, and each assembly instruction is the characteristic word of the topic.

[0041] The topic distribution of document d at time t is θ d,t , the word distribution of each topic k at time t is β k,t , the nth word w of each document d at time t in the generation process d,n,t The generation is achieved through the following steps:

[0042] (1) Topic distribution sampling: The topic distribution of document d at time t is θ d,t We can get a topic z by sampling d,n,t At the same time, the topic distribution of the document θ d,t It describes the weight distribution of each topic of document d at time t, and the probability of each topic k reflects the importance of topic k in the document.

[0043] (2) Based on the sampled topic z d,n,t , from the word distribution of the topic β k,t Sampling word w d,n,t This step reflects the distribution of words for each topic k in time t, capturing the lexical characteristics of the topic.

[0044] Through the above process, we can construct the “topic-word” distribution β k,t , extract different malicious code patterns. By analyzing these distributions, we can identify which instructions have a higher probability of appearing under certain topics, thereby revealing the behavioral patterns in malicious code. Constructing the “topic-document” θ d,t Distribution, which represents the weight of each topic in the document, can identify which topics dominate in different malicious code samples, thus helping to classify and detect malicious code.

[0045] 3. Topic model optimization and inference stage

[0046] During the model training process, the posterior distribution p(θ,β|w) is approximated by maximizing the variational objective function to optimize the model parameters. The following are detailed steps and formulas:

[0047] Initialize the topic distribution and word distribution for each time point, use the optimization of the variational objective function to estimate the topic distribution and word distribution and infer the model, and introduce the variational distribution q(θ d,t ,β k,t ) to approximate the true posterior distribution p(θ,β|w), and the variational objective function is

[0048]

[0049] Where D is the total number of documents, T is the total number of time, K is the total number of topics, and N d is the total number of words in document d, q(zdn =k) ​​is the variational distribution q, which represents the probability that word n ​​belongs to topic k in document d. Indicates word w under topic k dn The generation probability, θ d,t,k represents the distribution of topic k of document d at time point t, α t is the prior parameter of the document-topic distribution, η t is the prior parameter of the topic-word distribution, Dir(θ d,t |α t ) represents the topic distribution θ of document d at time t d,t Dirichlet distribution, Dir(β k,t |η t ) represents the word distribution β of document d at time t k,t The Dirichlet distribution.

[0050] In the process of calculating the variational objective function (ELBO), the current parameter settings are used to calculate the expected value of the topic distribution and word distribution, determine the probability of each word belonging to each topic, and the relationship between the document and the topic. On the other hand, the expected value calculated in the previous step is used to update the model parameters, and in the process of maximizing ELBO, the model's ability to explain the data is improved.

[0051] Specific process: Use the topic-word association probability to update the topic distribution parameters of each document at each time point. The formula is as follows: in, The objective function EIBO is d,t The gradient of t is the sum of squares of historical gradients, ∑ n q(z d,n ) is the expected probability that a word belongs to a topic. Using the calculated topic-word association probability, the word distribution parameters of each topic at each time point are updated. In order to better capture the temporal dynamic changes of the topic and reduce the impact of noise, the time factor is introduced to control the contribution of different time points. The formula is as follows: where q(z d,n =k) ​​is the expected probability that a word belongs to topic k, ferq(w d,n ) is the word w d,n The frequency in document d.

[0052] By continuously iterating until the change of ELBO is less than a preset threshold, it indicates that the parameters of the variational distribution have converged. This iterative optimization process ensures that the model has good generalization ability on new data and can stably extract the topic structure in the document.

[0053] 4. Malicious code feature detection stage

[0054] Through variational inference, the optimized parameters θ d,t and β k,t Can be used to extract malicious code features.

[0055] Topic distribution θ d,t : Represents the topic distribution of each document at different time points. By analyzing these distributions, topics related to malicious code can be identified.

[0056] Word distribution: represents the word distribution of each topic at different time points. By analyzing these distributions, specific words or patterns related to malicious code can be extracted.

[0057] Through real-time analysis and topic model updates, database access and system activities are continuously monitored. When potential privacy leaks are detected, an alarm mechanism is triggered to notify relevant security personnel.

[0058] Example 2

[0059] Assume that there is an online database management platform through which users can perform various SQL operations. Malicious users may attempt to attack through SQL injection, privilege escalation, etc.

[0060] Assume that the query log data is as follows:

[0061] 2024-08-01U001 SELECT*FROM users WHERE username='admin'AND password='1234'

[0062] 2024-08-01U002 INSERT INTO transactions(user_id,amount)VALUES(1,1000)

[0063] 2024-08-02U003 DELETE FROM users WHERE id=1; DROP TABLE users

[0064] 2024-08-02U001 UPDATE users SET password='newpass'WHERE username='admin'

[0065] 2024-08-03U002 SELECT*FROM orders WHERE amount>500; --bypassingauthentication

[0066] Documentation build:

[0067] Each day's operation log constitutes a document:

[0068] Document on August 1: SELECT*FROM users WHERE username='admin'AND password='1234',INSERT INTO transactions(user_id,amount)VALUES(1,1000)

[0069] Document on August 2: DELETE FROM users WHERE id=1; DROP TABLE users, UPDATEusers SET password='newpass' WHERE username='admin'

[0070] August 3rd document: SELECT * FROM orders WHERE amount>500; --bypassingauthentication

[0071] A topic model is constructed to identify the topics of malicious behavior, where each topic represents a possible malicious behavior pattern.

[0072] Topic 1 (SQL injection): related to keywords such as SELECT, WHERE, --, etc.

[0073] Topic 2 (Data Destruction): Related to keywords such as DELETE, DROP, and UPDATE.

[0074] Initialize topic distribution and word distribution:

[0075] Topic distribution θ d,t Initially: [0.5, 0.5] (assuming two topics), word distribution β k,t Initially, for topic 1: [0.3, 0.4, 0.3] (for keywords SELECT, WHERE, --); for topic 2: [0.4, 0.4, 0.2] (for keywords DELETE, DROP, UPDATE).

[0076] Through the optimized topic distribution and word distribution, malicious code features are identified:

[0077] Topic 1 (SQL injection): The probability is high in the documents on August 1 and August 3, indicating that there may be SQL injection attacks.

[0078] Topic 2 (Data Destruction): The probability is higher in the August 2 document, indicating that data destruction may have occurred.

[0079] In summary, the present invention has significant advantages over traditional technologies in terms of dynamics, real-time, multi-dimensional analysis capabilities, and distribution optimization accuracy. In particular, when dealing with complex and changeable attack modes and malicious code feature extraction, the present invention provides a more comprehensive and efficient solution.

[0080] The units, devices or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described separately by functions divided into various modules. Of course, when implementing this application, the functions of each module can be implemented in the same or more software and / or hardware, or the modules that implement the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0081] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0082] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0083] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0084] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0085] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for automatic detection of database privacy leakage based on topic model, which optimizes the distribution of topics and words by improving variational inference and ELBO objective function, identifies specific instructions and topic distribution in malicious code, learns malicious behavior patterns, and realizes dynamic privacy leakage detection and alarm. The method is characterized by: The method comprises the following steps: (1) Constructing data time series; (2) Model definition: Construct a topic model, treat malicious code samples as document collections, and assembly instructions as vocabulary. Analyze the "topic-word" and "topic-document" distributions to extract high-frequency malicious instructions. (3) Topic model optimization and inference: Complete parameter optimization and inference, approximate the true posterior distribution, and extract malicious code features from it. (4) Malicious code feature detection: Real-time analysis and topic model update, continuous monitoring of database access and system activities, and triggering an alarm mechanism to notify relevant security personnel when potential privacy leakage behavior is detected.

2. The method for automatic detection of database privacy leakage based on a topic model as claimed in claim 1, characterized in that: Step (2) comprises the following steps: (1) From the topic distribution of document d at time t, it is θ d,t We can get a topic z by sampling d,n,t ; (2) According to the theme d,n,t , from the word distribution of the topic β k,t Sampling word w d,n,t .

3. The method for automatic detection of database privacy leakage based on a topic model as claimed in claim 1, characterized in that: Step (3) comprises the following steps: 3.1 Calculation in, The objective function EIBO is d,t The gradient of t is the sum of squares of historical gradients, ∑ n q(z d,n ) is the expected probability that a word belongs to the topic; 3.2 Calculation where q(z d,n =k) ​​is the expected probability that the word belongs to topic k, ferq(w (d,n) ) is the word w (d,n) The frequency in document d, t is the current time, t′ is the previous time point; 3.3 Calculation of variational objective function Where; D is the total number of documents, T is the total number of time, K is the total number of topics, N d is the total number of words in document d, q(z dn =k) ​​is the probability that word n ​​belongs to topic k in document d, Submit word w for topic k dn The generation probability, θ d,t,k is the distribution of topic k of document d at time point t, α t is the prior parameter of the document-topic distribution, η t is the prior parameter of the topic-word distribution, Dir(θ d,t |α t ) is the topic distribution θ of document d at time t d,t Dirichlet distribution, Dir(β k,t |η t ) is the word distribution β of document d at time t k,t Dirichlet distribution, q(θ d,t ) and q(β k,t ) are all variational distributions.

4. A database privacy leakage automatic detection system based on topic model, which optimizes the distribution of topics and words by improving variational inference and ELBO objective function, identifies specific instructions and topic distribution in malicious code, learns malicious behavior patterns, and realizes dynamic privacy leakage detection and alarm. It is characterized by: The system includes: (1) Build a data time series module; (2) Model definition module: This module builds a topic model, treats malicious code samples as document collections, and assembly instructions as vocabulary. It analyzes the "topic-word" and "topic-document" distributions and extracts high-frequency malicious instructions. (3) Topic model optimization and inference module: This module completes parameter optimization and inference, approximates the true posterior distribution, and extracts malicious code features from it. (4) Malicious code feature detection module: This module performs real-time analysis and updates the topic model, continuously monitors database access and system activities, and triggers an alarm mechanism to notify relevant security personnel when potential privacy leakage behavior is detected.

5. The automatic detection system for database privacy leakage based on topic model as claimed in claim 4, characterized in that: The model definition module takes the following steps: (1) From the topic distribution of document d at time t, it is θ d,t We can get a topic z by sampling d,n,t ; (2) According to the theme d,n,t , from the word distribution of the topic β k,t Sampling word w d,n,t .

6. The automatic detection system for database privacy leakage based on topic model as claimed in claim 4, characterized in that: The topic model optimization and inference module uses the following steps: 3.1 Calculation in, The objective function EIBO is d,t The gradient of t is the sum of squares of historical gradients, ∑ n q(z d,n ) is the expected probability that a word belongs to the topic; 3.2 Calculation where q(z d,n =k) ​​is the expected probability that the word belongs to topic k, ferq(w (d,n) ) is the word w (d,n) The frequency in document d, t is the current time, t′ is the previous time point; 3.3 Calculation of variational objective function Where; D is the total number of documents, T is the total number of time, K is the total number of topics, N d is the total number of words in document d, q(z dn =k) ​​is the probability that word n ​​belongs to topic k in document d, Submit word w for topic k dn The generation probability, θ d,t,k is the distribution of topic k of document d at time point t, α t is the prior parameter of the document-topic distribution, η t is the prior parameter of the topic-word distribution, Dir(θ d,t |α t ) is the topic distribution θ of document d at time t d,t Dirichlet distribution, Dir(β k,t |η t ) is the word distribution β of document d at time t k,t Dirichlet distribution, q(θ d,t ) and q(β k,t ) are all variational distributions.

Images