Log audit security policy generation method based on large model

Abstract

The invention discloses a log auditing security policy generation method based on a large model, which comprises the following steps of: 1, acquiring a security log, and performing data preprocessing and feature extraction to obtain a security feature vector; step 2, vector database construction: constructing a vector database, and storing the extracted security feature vectors in the vector database; step 3, security feature retrieval enhancement: retrieving in a vector database for new log data to obtain similar historical vectors; step 4, generating a security policy by the large model: inputting the retrieved correlation vector and metadata thereof as context information into the pre-trained large model to generate a security policy adapted to the current situation; and step 5, based on the security policy, performing log security audit identification and analysis to obtain security threat behaviors. According to the method, the accuracy of generating the log auditing security policy by the large model is improved, so that the log auditing efficiency is improved, and the data security is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of data security and information security, and in particular to a log audit security policy generation method based on a large model. Background Art

[0002] With the increasing complexity of modern information system business and the growth of data volume, traditional log audit methods are increasingly unable to meet the requirements of security audit of massive log data. Existing methods mainly rely on manually setting fixed audit policy rules and manually writing SQL analysis logic code, which is not only inefficient but also difficult to cope with the ever-changing new dangerous attack threats. The current log audit platform system mainly faces the following problems. 1. The current security audit based on large models often directly uses security log data to generate security policies. The generated security policy model is extremely inaccurate and cannot be adapted to the security environment of on-site production data. It is difficult to achieve the ability to build security policies through large models. 2. The problem of missing user data leads to the inability to pre-train user data models: it involves data privacy issues, production data security issues, personal privacy issues and other security restrictions, resulting in the log audit platform being unable to directly obtain key user behavior data for training user data models, which brings huge challenges to subsequent data analysis. 3. Security policy model construction problem: due to the lack of data models, it is impossible to predict the risk factors of user behavior in the real environment in advance, and it is impossible to build a security policy model for real scenarios. Summary of the invention

[0003] The purpose of the present invention is to provide a log audit security policy generation method based on a large model. By utilizing advanced natural language large model processing technology, abnormal behavior can be automatically identified from massive log data. By building a security feature vector database, security feature retrieval enhancement and other technologies, the accuracy of log audit security policy generation based on the large model can be improved, thereby improving the efficiency of log auditing while ensuring data security.

[0004] The technical solution adopted by the present invention is: A log audit security policy generation method based on a large model comprises the following steps: Step 1: Obtain security logs, perform data preprocessing and feature extraction to obtain security feature vectors; Furthermore, in step 1, data preprocessing is performed by cleaning the original log data, removing irrelevant information, and extracting important fields such as timestamp and operation type.

[0005] Furthermore, in step 1, feature extraction uses word embedding technology to convert text information into feature vectors, and extracts features in combination with user behavior patterns.

[0006] Step 2, vector database construction: construct a vector database and store the extracted security feature vectors in the vector database; Step 3: Enhanced security feature retrieval: retrieve similar historical vectors from the vector database for new log data; Furthermore, in step 3, the new log data is first converted into a security feature vector, and then similar historical vectors are retrieved from the vector database.

[0007] Furthermore, in step 3, a support vector distance algorithm is used to achieve similar vector comparison, thereby obtaining historically similar safety risk behavior log data. Specifically, the distance algorithm includes a cost distance algorithm and a Euclidean distance algorithm.

[0008] Step 4: The big model generates security policies: The retrieved relevant vectors and their metadata are input as context information into the pre-trained big model to generate security policies that adapt to the current situation.

[0009] Specifically, the retrieved relevant vectors and their metadata are submitted to the big model as contextual information to enhance its understanding and generation capabilities.

[0010] Step 5: Perform log security audit based on security policies to identify and analyze security threat behaviors.

[0011] Furthermore, in step 5, the log audit system combines the security audit strategy generated by the large model to perform real-time stream calculation on the current real-time log to analyze user behavior piece by piece and identify and analyze security threat behaviors.

[0012] The present invention adopts the above technical solution and has the following technical advantages: (1) Security feature retrieval enhancement: The security feature retrieval enhancement technology is used to retrieve relevant security feature vectors from the vector database, which enhances the understanding ability of the large model and enables it to generate more accurate and adaptable security policies. (2) Security feature vectorization based on vector database: By vectorizing security-related data into high-dimensional vectors and storing them in the vector database, efficient similarity retrieval is achieved and the accuracy of anomaly detection is improved. (3) Adaptive security policy generation mechanism: The security policy is automatically adjusted according to the real-time detection results, so that the system can quickly respond to new threats, enhancing the adaptability and flexibility of the system.

[0013] The present invention utilizes advanced natural language large model processing technology to automatically identify abnormal behavior from massive log data, and improves the accuracy of log audit security policy generated by large models by building a security feature vector database, security feature retrieval enhancement and other technologies, thereby improving the efficiency of log auditing while ensuring data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments; Figure 1 The present invention is a flowchart of a method for generating a log audit security policy based on a large model. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0016] Large Language Model (LLM): An AI large model refers to a machine learning model with ultra-large-scale parameters (usually more than one billion) and complex computing structures. It is usually able to process massive amounts of data and complete various complex tasks, such as natural language processing and image recognition. Such as Transformer, etc. Log Audit: It refers to the security audit of log activities by monitoring, detecting and analyzing the logs of various production systems to ensure the compliance of user behavior and discover potential security threats. Security Policy: It refers to a set of rule-policy combinations that define how to protect the security of system operation. Security Feature Vector: High-dimensional vectorization of security-related data for vectorized retrieval and calculation. Security Feature Retrieval Enhancement: Use feature vector data as the object of retrieval enhancement to use the retrieval results as decision-making reference data for generating security policies for large models.

[0017] like Figure 1 As shown, the present invention discloses a log audit security policy generation method based on a large model, which includes the following steps: Step 1: Obtain security logs, perform data preprocessing and feature extraction to obtain security feature vectors; Specifically, security logs are mainly divided into historical retention logs and current real-time logs, and both types of logs need to be preprocessed and extracted.

[0018] Data preprocessing cleans the raw log data, removes irrelevant information, and extracts important fields such as timestamps and operation types. Log cleaning uses regular expressions and other techniques to clean the raw log data and remove irrelevant information. For example, timestamps, IP addresses, operation types, user IDs, etc. The cleaned log data is parsed into a structured format for subsequent processing. The Pandas library can be used to store data as a DataFrame.

[0019] Feature extraction uses word embedding technology to convert text information into feature vectors, and extracts features in combination with user behavior patterns. Word embedding technology uses BERT or other word embedding technologies to convert text information into vector representations. BERT is a pre-trained Transformer model that can generate high-quality text embeddings. Features are extracted in combination with user behavior patterns, such as user login frequency, access path, operation sequence, etc. These features can be extracted using statistical methods or machine learning models (such as LSTM).

[0020] Step 2, vector database construction: construct a vector database and store the extracted security feature vectors in the vector database; Specifically, data vectorization processing: the security threat behavior feature vectors extracted from the historical security data logs are processed and stored in the vector database. Each vector carries corresponding metadata, such as timestamp, source, and other related data. Database vector retrieval: the database improves the efficient vector similarity search and retrieval capabilities, thereby achieving the retrieval capability of large-scale high-dimensional vectors.

[0021] Step 3: Enhanced security feature retrieval: retrieve similar historical vectors from the vector database for new log data; Furthermore, in step 3, the new log data is first converted into a security feature vector, and then similar historical vectors are retrieved from the vector database.

[0022] Specifically, step 3 includes the following aspects: Retrieve relevant vectors: When new real-time security log data comes in, it is first converted into a security feature vector, and then similar historical vectors are retrieved in the vector database.

[0023] Vector similarity algorithm: The present invention supports vector distance algorithms including cost distance algorithm and Euclidean distance algorithm to achieve similar vector comparison, thereby obtaining historically similar safety risk behavior log data.

[0024] Context enhancement: The retrieved relevant vectors and their metadata are submitted to the big model as context information, which can enhance the big model's understanding ability and enable it to better generate security policies.

[0025] Example: Assume that the system detects a new malware variant. Through security feature retrieval enhancement technology, the system can retrieve behavior patterns similar to malware from historical data and generate targeted security policies. Through vector similarity analysis retrieval, the system can identify the propagation path of the malware and generate network isolation rules to prevent it from spreading further.

[0026] Step 4: The big model generates security policies: The retrieved relevant vectors and their metadata are input as context information into the pre-trained big model to generate security policies that adapt to the current situation.

[0027] Specifically, the retrieved relevant vectors and their metadata are submitted to the big model as contextual information to enhance its understanding and generation capabilities.

[0028] Log audit security policy generation uses a large model to retrieve and analyze the result data set based on the input vector to generate corresponding security policy rules, such as access control rules, anomaly detection rules, etc.

[0029] Example: Using pre-trained large models to automatically generate security policies without manually writing rules not only improves response speed but also reduces human errors. When abnormal login behavior is detected, the system can automatically generate access control rules to limit the access rights of the IP address and immediately notify the administrator, so as to quickly respond to potential threats.

[0030] Step 5: Perform log security audit based on security policies to identify and analyze security threat behaviors.

[0031] Furthermore, in step 5, the log audit system combines the security audit strategy generated by the large model to perform real-time stream calculation on the current real-time log to analyze user behavior piece by piece and identify and analyze security threat behaviors.

[0032] The present invention adopts the above technical solution and has the following technical advantages: (1) Security feature retrieval enhancement: The security feature retrieval enhancement technology is used to retrieve relevant security feature vectors from the vector database, which enhances the understanding ability of the large model and enables it to generate more accurate and adaptable security policies. (2) Security feature vectorization based on vector database: By vectorizing security-related data into high-dimensional vectors and storing them in the vector database, efficient similarity retrieval is achieved and the accuracy of anomaly detection is improved. (3) Adaptive security policy generation mechanism: The security policy is automatically adjusted according to the real-time detection results, so that the system can quickly respond to new threats, enhancing the adaptability and flexibility of the system.

[0033] The present invention utilizes advanced natural language large model processing technology to automatically identify abnormal behavior from massive log data, and improves the accuracy of log audit security policy generated by large models by building a security feature vector database, security feature retrieval enhancement and other technologies, thereby improving the efficiency of log auditing while ensuring data security.

[0034] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians of the art without making creative work are within the scope of protection of the present application.

Claims

1. A log audit security policy generation method based on a large model, characterized by: It includes the following steps: Step 1: Obtain security logs, perform data preprocessing and feature extraction to obtain security feature vectors; Step 2, vector database construction: construct a vector database and store the extracted security feature vectors in the vector database; Step 3: Enhanced security feature retrieval: retrieve similar historical vectors from the vector database for new log data; Step 4: The big model generates security policies: The retrieved relevant vectors and their metadata are used as context information and input into the pre-trained big model to generate security policies that adapt to the current situation. Step 5: Perform log security audit based on security policies to identify and analyze security threat behaviors.

2. According to the method for generating a log audit security policy based on a large model as described in claim 1, it is characterized by: In step 1, data preprocessing cleans the original log data, removes irrelevant information, and extracts important fields such as timestamp and operation type.

3. According to the method for generating a log audit security policy based on a large model as described in claim 1, it is characterized in that: In step 1, feature extraction uses word embedding technology to convert text information into feature vectors, and extracts features based on user behavior patterns.

4. According to the method for generating a log audit security policy based on a large model as described in claim 1, it is characterized in that: In step 3, the new log data is first converted into a security feature vector, and then similar historical vectors are retrieved from the vector database.

5. According to the method for generating a log audit security policy based on a large model as described in claim 1, it is characterized by: In step 3, the support vector distance algorithm is used to realize similarity vector comparison, so as to obtain historically similar safety risk behavior log data.

6. The method for generating a log audit security policy based on a large model according to claim 1, characterized in that: In step 5, the log audit system combines the security audit strategy generated by the large model to perform real-time stream calculation on the current real-time log to analyze user behavior piece by piece and identify and analyze security threat behaviors.

Images