Audit data generation method and device, equipment, medium and program product

Abstract

The invention discloses an audit data generation method and device, equipment, a medium and a program product. The method comprises the steps of obtaining target log data meeting a query condition in response to an audit query instruction; constructing an attack graph based on the target log data; determining a risk parameter corresponding to the target log data through an anomaly detection model; calling the blockchain evidence of the smart contract based on log data and the blockchain evidence of the current version of the anomaly detection model, and verifying the validity of the risk parameters; and if the risk parameter is valid, generating a log audit report. According to the method, the log data and the exception detection model version are both chained for evidence storage, and the smart contract performs dual tampering verification on the target log data and the model based on the blockchain evidence storage of the log data and the exception detection model version, so that the validity of the risk parameter is verified. Through deep cooperation of the block chain technology and the artificial intelligence technology, the accuracy and effectiveness of the log risk parameters are ensured, and the accuracy and credibility of the log anomaly detection result are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data, and in particular to an audit data generation method, device, equipment, medium and program product. BACKGROUND

[0002] A trusted data space is a data flow utilization infrastructure based on consensus rules, connecting multiple parties, realizing data resource sharing, and is an important carrier for supporting the construction of an integrated data market. The trusted data space is an important support and guarantee for promoting the high-quality development of the data ecosystem, and is also the key to driving the development and evolution of the paradigm from enterprise internal circulation to inter-enterprise and social circulation.

[0003] Log auditing is one of the core mechanisms for building a trusted data space. Log auditing is the process of collecting, storing, analyzing and reporting logs generated in the system. Logs record various events during system operation. In log auditing, potential security threats and violations can be tracked and identified by analyzing these logs to ensure the security and compliance of data circulation.

[0004] With the rapid development of blockchain technology and artificial intelligence technology, a log auditing method combining blockchain and artificial intelligence has gradually emerged. In this method, blockchain is used to store evidence of logs, and an artificial intelligence model is used to detect anomalies in logs.

[0005] In the process of implementing the present application, the applicant found at least the following problems in the prior art:

[0006] In the prior art, the blockchain technology and artificial intelligence technology lack deep collaboration, which limits the accuracy and credibility of log anomaly detection results. SUMMARY

[0007] The present application provides an audit data generation method, device, electronic equipment and storage medium, which can deeply collaborate blockchain technology and artificial intelligence technology to improve the accuracy and credibility of log anomaly detection results.

[0008] In a first aspect, the embodiments of the present application provide an audit data generation method, which comprises:

[0009] In response to an audit query instruction carrying a query condition for log data, target log data satisfying the query condition is obtained;

[0010] An attack graph is constructed based on the target log data;

[0011] A risk parameter corresponding to the target log data is determined by an anomaly detection model;

[0012] The smart contract deployed on the blockchain is invoked to verify the validity of the risk parameters based on the blockchain notarization of the pre-stored log data and the blockchain notarization of the current version of the anomaly detection model; the encrypted log data is also pre-stored on the blockchain.

[0013] If the risk parameters are verified to be valid, a log audit report is generated based on the risk parameters and the attack graph.

[0014] In an optional embodiment, the method further includes:

[0015] Based on the attack graph and / or the risk parameters, detect whether the model update triggering conditions are met;

[0016] If the model update triggering condition is met, the smart contract is invoked to incrementally train the anomaly detection model using federated learning and privacy protection technologies, so as to update the version of the anomaly detection model;

[0017] After the anomaly detection model completes its version update, the smart contract is invoked to store the updated version of the anomaly detection model.

[0018] In an optional embodiment, the step of calling the smart contract to store the updated version of the anomaly detection model includes:

[0019] The hash value of the updated version of the anomaly detection model is determined based on the hash value of the training data used in this version update, the model parameters obtained after this version update, and the performance metrics of the anomaly detection model after this version update.

[0020] The smart contract is invoked to write the hash value of the updated version of the anomaly detection model into the blockchain, thereby obtaining a blockchain-based notarized record of the updated version of the anomaly detection model.

[0021] In an optional embodiment, the invocation of the smart contract deployed on the blockchain verifies the validity of the risk parameters based on the pre-stored blockchain evidence of the log data and the current version of the anomaly detection model, including:

[0022] Determine the hash value of the target log data, and the hash value of the current version of the anomaly detection model used when determining the risk parameters;

[0023] The smart contract deployed on the blockchain is invoked to compare the hash value of the target log data with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and to compare the hash value of the current version of the anomaly detection model used when determining the risk parameter with the blockchain evidence of the current version of the anomaly detection model.

[0024] The validity of the risk parameters is determined based on the comparison results.

[0025] In an optional embodiment, determining the validity of the risk parameter based on the comparison results includes:

[0026] If the hash value of the target log data is consistent with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and the hash value of the current version of the anomaly detection model used when determining the risk parameter is consistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, then the risk parameter is determined to be valid.

[0027] If the hash value of the target log data is inconsistent with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and / or the hash value of the current version of the anomaly detection model used when determining the risk parameter is inconsistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, the risk parameter is determined to be invalid.

[0028] In an optional embodiment, constructing the attack graph based on the target log data includes:

[0029] The target log data is transformed into structured log data using a large language model;

[0030] The structured log data is then used to construct a time series diagram.

[0031] The attack graph is obtained by constructing the time sequence graph using a graph structure analysis model.

[0032] Secondly, embodiments of this application also provide an audit data generation apparatus, the apparatus comprising:

[0033] The data acquisition module is used to respond to an audit query instruction carrying query conditions for log data and acquire target log data that meets the query conditions.

[0034] The graph construction module is used to construct an attack graph based on the target log data.

[0035] The risk parameter determination module is used to determine the risk parameters corresponding to the target log data through an anomaly detection model.

[0036] The validity verification module is used to call the smart contract deployed on the blockchain to verify the validity of the risk parameters based on the blockchain notarization of the pre-stored log data and the blockchain notarization of the current version of the anomaly detection model; the blockchain also pre-stores the encrypted log data.

[0037] The report generation module is used to generate a log audit report based on the risk parameters and the attack graph, provided that the risk parameters have been verified as valid.

[0038] In an optional embodiment, the device is further configured to:

[0039] Based on the attack graph and / or the risk parameters, detect whether the model update triggering conditions are met;

[0040] If the model update triggering condition is met, the smart contract is invoked to incrementally train the anomaly detection model using federated learning and privacy protection technologies, so as to update the version of the anomaly detection model;

[0041] After the anomaly detection model completes its version update, the smart contract is invoked to store the updated version of the anomaly detection model.

[0042] In an optional embodiment, the step of calling the smart contract to store the updated version of the anomaly detection model includes:

[0043] The hash value of the updated version of the anomaly detection model is determined based on the hash value of the training data used in this version update, the model parameters obtained after this version update, and the performance metrics of the anomaly detection model after this version update.

[0044] The smart contract is invoked to write the hash value of the updated version of the anomaly detection model into the blockchain, thereby obtaining a blockchain-based notarized record of the updated version of the anomaly detection model.

[0045] In an optional embodiment, the invocation of the smart contract deployed on the blockchain verifies the validity of the risk parameters based on the pre-stored blockchain evidence of the log data and the current version of the anomaly detection model, including:

[0046] Determine the hash value of the target log data, and the hash value of the current version of the anomaly detection model used when determining the risk parameters;

[0047] The smart contract deployed on the blockchain is invoked to compare the hash value of the target log data with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and to compare the hash value of the current version of the anomaly detection model used when determining the risk parameter with the blockchain evidence of the current version of the anomaly detection model.

[0048] The validity of the risk parameters is determined based on the comparison results.

[0049] In an optional embodiment, determining the validity of the risk parameter based on the comparison results includes:

[0050] If the hash value of the target log data is consistent with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and the hash value of the current version of the anomaly detection model used when determining the risk parameter is consistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, then the risk parameter is determined to be valid.

[0051] If the hash value of the target log data is inconsistent with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and / or the hash value of the current version of the anomaly detection model used when determining the risk parameter is inconsistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, the risk parameter is determined to be invalid.

[0052] In an optional embodiment, constructing the attack graph based on the target log data includes:

[0053] The target log data is transformed into structured log data using a large language model;

[0054] The structured log data is then used to construct a time series diagram.

[0055] The attack graph is obtained by constructing the time sequence graph using a graph structure analysis model.

[0056] Thirdly, embodiments of this application provide an electronic device, including:

[0057] One or more processors;

[0058] Memory, used to store one or more programs.

[0059] When the one or more programs are executed by the one or more processors, the one or more processors implement the audit data generation method described in any embodiment of this application.

[0060] Fourthly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the audit data generation method described in any embodiment of this application.

[0061] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the audit data generation method described in any embodiment of this application.

[0062] This application proposes an audit data generation method, apparatus, device, medium, and program product. Specifically, in the technical solution of this application, firstly, in response to an audit query instruction carrying query conditions for log data, target log data that meets the query conditions can be obtained; then, an attack graph can be constructed based on the target log data; subsequently, risk parameters corresponding to the target log data are determined through an anomaly detection model; both the log data and the current version of the anomaly detection model are stored on the blockchain for evidence preservation, and a smart contract deployed on the blockchain can perform dual tamper verification of the target log data and the model used to determine the risk parameters based on the blockchain evidence preservation of both, thereby verifying the validity of the risk parameters corresponding to the target log data. Thus, the deep collaboration between blockchain technology and artificial intelligence technology ensures the accuracy and validity of log risk parameters, improving the accuracy and credibility of log anomaly detection results. When the risk parameters are valid, an accurate and reliable log audit report can be generated based on the risk parameters and the attack graph. Attached Figure Description

[0063] Figure 1 A flowchart illustrating an audit data generation method provided in an embodiment of this application;

[0064] Figure 2 A flowchart illustrating an audit data generation method provided in another embodiment of this application;

[0065] Figure 3 A flowchart illustrating an audit data generation method provided in another embodiment of this application;

[0066] Figure 4 This is a schematic diagram of the structure of an audit data generation apparatus provided in an embodiment of this application;

[0067] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0068] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0069] Before detailing the embodiments of this application, the prior art will be further described first.

[0070] In traditional log auditing methods, logs are centrally stored in a centralized database, and abnormal patterns are matched by manually configured rules (such as regular expressions), relying on manual analysis by auditors.

[0071] With the rapid development of blockchain and artificial intelligence technologies, log auditing methods that combine blockchain and artificial intelligence have gradually emerged. In this approach, blockchain is used to store logs as evidence, artificial intelligence models are used to detect anomalies in the logs, and model training relies on a centralized log library.

[0072] In the process of developing this application, the applicant discovered that traditional log auditing methods rely on manual rules, resulting in low efficiency in anomaly identification and an inability to identify new types of attacks; the combination of massive logs with manual analysis leads to high latency in anomaly analysis and a high false alarm rate; centralized storage of logs is easily tampered with, and the audit results lack credibility, resulting in low reliability of the audit results.

[0073] The log auditing method combining blockchain and artificial intelligence can improve the efficiency of anomaly detection through artificial intelligence models and prevent log tampering through blockchain evidence storage compared to traditional log auditing methods. However, this method still has its shortcomings:

[0074] 1. The lack of deep collaboration between blockchain technology and artificial intelligence technology limits the accuracy and reliability of log anomaly detection results;

[0075] 2. After a period of use, the artificial intelligence model cannot adapt to new attack patterns, and the model parameters become outdated, which further reduces the accuracy of log anomaly detection results.

[0076] 3. Centralized training of artificial intelligence using raw logs can easily lead to data leaks and make it difficult to protect the privacy requirements of training data.

[0077] This application provides an audit data generation method to address or at least partially address the aforementioned deficiencies of the prior art.

[0078] Figure 1This is a flowchart illustrating an audit data generation method according to an embodiment of this application. The method can be executed by an audit data generation device or electronic device, which can be implemented in software and / or hardware, and can be integrated into any smart device with network communication capabilities. Figure 1 As shown, the audit data generation method may include the following steps:

[0079] S101. In response to an audit query instruction carrying query conditions for log data, obtain the target log data that meets the query conditions.

[0080] In this step, an audit query command carrying query conditions for log data can be received from the user. Optionally, a visual graphical user interface can be provided, and input boxes, selection boxes, and other controls can be provided in the graphical user interface to receive the audit query command input by the user.

[0081] Query conditions for log data can include at least one of the following: the time period to be queried (e.g., from May 1, 2025 to May 5, 2025), the event to be queried (e.g., abnormal login event), the device to be queried (e.g., device A), and the user to be queried (e.g., user 642), but are not limited to these. For example, query conditions for log data could be: Query abnormal login events on device A from May 1, 2025 to May 5, 2025.

[0082] In response to an audit query instruction carrying query conditions for log data, target log data that meets the query conditions can be retrieved from stored log data (such as standardized log data described below). For example, if the query condition is "query login failure events on device A from 2025-05-01 to 2025-05-05", then log data with timestamps from 2025-05-01 to 2025-05-05 that resulted in login failures on device A can be retrieved from the stored log data; this is the target log data.

[0083] In an optional embodiment, raw log data can first be collected. This raw log data can be multi-source data, meaning data from multiple sources. In one optional embodiment, raw log data can be collected from multiple data sources in real time or periodically using software programs or hardware devices for automatic log data collection. After collecting the raw log data, it can be preprocessed, for example, parsing it into a unified format (e.g., JSON format) to obtain standardized log data. Standardized log data can be stored uniformly for easy retrieval. This application does not limit the storage method of standardized logs; for example, it can be dynamically fragmented or fragmented by time.

[0084] S102. An attack graph is constructed based on the target log data.

[0085] In this step, after obtaining the target log data that meets the query conditions, the correlations between logs in the target log data can be analyzed to identify attack paths, thereby constructing an attack graph. An attack graph is a directed graph that can display attack paths, abnormal events, etc. between devices and users. It shows the development and impact of attack events through nodes (such as devices and users) and edges (such as events).

[0086] S103. Determine the risk parameters corresponding to the target log data through the anomaly detection model.

[0087] In this step, an artificial intelligence model can be used to detect anomalies in the target log data and determine its corresponding risk parameters. Optionally, the risk parameter can be represented by a specific numerical value, such as a risk value or risk score; alternatively, the risk parameter can also be represented by a level, such as a risk grade. This application does not specifically limit this.

[0088] In an optional embodiment, the risk parameter corresponding to the target log data can be determined for each log entry; that is, a corresponding risk parameter can be determined for each log entry in the target log data. In an optional embodiment, a corresponding risk parameter can also be determined for the target log data as a whole; this application does not specifically limit this.

[0089] In optional embodiments, the model architecture of the anomaly detection model may adopt a deep learning model based on a self-attention mechanism, but this application does not limit the specific anomaly detection model.

[0090] S104. Call the smart contract deployed on the blockchain to verify the validity of the risk parameters based on the blockchain notarization of pre-stored log data and the current version of the anomaly detection model; encrypted log data is also pre-stored on the blockchain.

[0091] After determining the risk parameters corresponding to the target log data using an artificial intelligence model, this step involves calling a smart contract deployed on the blockchain to verify the validity of the risk parameters. This can be done by pre-depositing the log data and the current version of the anomaly detection model on the blockchain. The smart contract verifies whether the target log data (i.e., the audited object) has been tampered with by verifying the blockchain evidence of the log data, and whether the current version of the anomaly detection model (i.e., the tool used to determine the risk parameters corresponding to the audited object) has been tampered with by verifying the blockchain evidence of the current version. If the audited target log data has been tampered with, the determined risk parameters will be inaccurate; similarly, if the current version of the anomaly detection model used to determine the risk parameters corresponding to the target log data has been tampered with, the determined risk parameters will also be inaccurate. If the audited target log data has not been tampered with, and the current version of the anomaly detection model used to determine the risk parameters corresponding to the target log data has also not been tampered with, then the risk parameters corresponding to the target log data determined in step S103 were not caused by tampering, indicating that the risk parameters are valid.

[0092] In this embodiment, both the current version of the log data and the anomaly detection model are stored on the blockchain for evidence preservation. The smart contract deployed on the blockchain can perform dual tamper verification on the target log data and the model used to determine risk parameters based on the blockchain evidence preservation of both, thereby verifying the validity of the risk parameters corresponding to the target log data. Thus, the deep collaboration between blockchain and artificial intelligence technologies ensures the accuracy and validity of log risk parameters, improving the accuracy and reliability of log anomaly detection results. Furthermore, lightweight tamper verification based on blockchain evidence preservation reduces the overhead of storing all logs on the blockchain.

[0093] Furthermore, log data can be encrypted and stored using blockchain for easy traceability. In an optional embodiment, the original log data undergoes standardized preprocessing to obtain standardized log data. This standardized log data can then be encrypted and stored at a designated address on the blockchain, such as a node on the InterPlanetary File System (IPS) within the blockchain. This application does not limit the storage method of encrypted logs on the blockchain; for example, it can use dynamic sharding or time-based sharding. In optional embodiments, the log data encryption method can employ symmetric or asymmetric encryption; this application does not specifically limit the encryption method used for logs.

[0094] S105. If the risk parameters are verified to be valid, generate a log audit report based on the risk parameters and attack graph.

[0095] In this step, if the risk parameter is verified to be valid, a log audit report can be generated based on the risk parameter and the attack graph. For example, the log audit report can show the log data with high risk parameters (e.g., exceeding the preset risk parameter threshold) in the target log data, as well as the constructed attack graph.

[0096] The audit data generation method proposed in this application first responds to an audit query instruction carrying query conditions for log data, thereby obtaining target log data that meets the query conditions. Then, an attack graph is constructed based on the target log data. Next, an anomaly detection model determines the risk parameters corresponding to the target log data. Both the log data and the current version of the anomaly detection model are stored on the blockchain for verification. A smart contract deployed on the blockchain can perform dual tamper verification of the target log data and the model used to determine the risk parameters based on the blockchain-stored evidence, thus verifying the validity of the risk parameters corresponding to the target log data. In this way, the deep collaboration between blockchain technology and artificial intelligence technology ensures the accuracy and validity of the log risk parameters, improving the accuracy and credibility of the log anomaly detection results. When the risk parameters are valid, an accurate and reliable log audit report can be generated based on the risk parameters and the attack graph.

[0097] Figure 2 This is a flowchart illustrating an audit data generation method according to another embodiment of this application. Further optimizations and extensions based on the above technical solution are possible, and it can be combined with the various optional implementation methods described above. For example... Figure 2 As shown, the audit data generation method may include the following steps:

[0098] S201. In response to an audit query instruction carrying query conditions for log data, obtain the target log data that meets the query conditions.

[0099] S202. An attack graph is constructed based on the target log data.

[0100] S203. Determine the risk parameters corresponding to the target log data through the anomaly detection model.

[0101] S204. Call the smart contract deployed on the blockchain to verify the validity of the risk parameters based on the blockchain notarization of pre-stored log data and the current version of the anomaly detection model; encrypted log data is also pre-stored on the blockchain.

[0102] S205. If the risk parameters are verified to be valid, generate a log audit report based on the risk parameters and attack graph.

[0103] S206. Detect whether the model update triggering conditions are met based on the attack graph and / or risk parameters.

[0104] In this step, the model update trigger condition can be detected based on the attack graph and / or risk parameters. If the model update trigger condition is met, the anomaly detection model can be updated; otherwise, the anomaly detection model will not be updated.

[0105] In an optional embodiment, the model update trigger condition can be determined when a new attack pattern is identified from the attack graph.

[0106] In an optional embodiment, the model update trigger condition can be determined to be met when the risk parameter corresponding to the target log data is higher than the risk parameter threshold.

[0107] In an optional embodiment, when a new attack pattern is identified from the attack graph and the risk parameter corresponding to the target log data is higher than the risk parameter threshold, it can be determined that the model update trigger condition is met.

[0108] S207. If the model update triggering condition is met, the smart contract is invoked to incrementally train the anomaly detection model using federated learning and privacy protection technologies in order to update the anomaly detection model.

[0109] In this step, if the model update trigger condition is met, the smart contract can be invoked to incrementally train the anomaly detection model, thereby updating the model. The training data used for incremental training can be newly collected log data.

[0110] In this embodiment of the application, when the model update triggering condition is met, the anomaly detection model can be updated through a smart contract, so that the capabilities of the anomaly detection model can dynamically adapt to newly emerging attack patterns, thereby enabling the anomaly detection model to maintain a high level for a long time and ensuring the accuracy of log anomaly detection results for a long time.

[0111] In this embodiment, an anomaly detection model can be trained using federated learning and privacy-preserving technologies. Federated learning allows multiple participants (e.g., multiple companies, multiple departments) to jointly train the model without sharing the original data, exchanging only encrypted model parameters. Federated learning supports data remaining within its domain, reducing training time compared to centralized AI training methods. Privacy-preserving technologies (e.g., differential privacy, homomorphic encryption) can add noise during gradient aggregation during model training to prevent reverse engineering of the original data. The formula is:

[0112] g′ i =g i +Laplace(0,b)

[0113] Where g iIt is the local gradient of the i-th participant, g′ i is the gradient after adding noise, and b is the noise scale.

[0114] In this application, the anomaly detection model is trained using federated learning and privacy-preserving technologies. Compared to centralized artificial intelligence training using raw logs, this avoids the leakage of training data and meets the privacy requirements of training data.

[0115] S208. After the anomaly detection model completes its version update, call the smart contract to store the updated version of the anomaly detection model.

[0116] This application records the new version on the blockchain after each model iteration. Its core value lies in providing transparent and authoritative endorsement for the entire lifecycle of the model through the immutability, traceability, and distributed trust mechanism of the blockchain.

[0117] In an optional embodiment, the anomaly detection model can be stored on the blockchain in the following ways:

[0118] The hash value of the current version of the anomaly detection model is determined based on the hash value of the training data used by the current version of the anomaly detection model, the model parameters (such as hyperparameters) of the current version of the anomaly detection model, and the performance metrics of the current version of the anomaly detection model.

[0119] The smart contract is invoked to write the hash value of the current version of the anomaly detection model to the blockchain, thus obtaining the blockchain evidence of the current version of the anomaly detection model.

[0120] For example, the hash value H of the model training data can be used. data The current hyperparameters θ of the model hyper Current performance metrics of the model M performance Package it into structured data, and then calculate its hash value H. model Then, the smart contract is called to transfer the hash value H. model Write it to the blockchain to obtain a blockchain certificate of the current version of the model, ensuring traceability. In case of disputes, the model reasoning process can be reproduced based on the on-chain data.

[0121] Accordingly, the step of calling the smart contract to store the updated version of the anomaly detection model in this process can be implemented in the following way:

[0122] The hash value of the updated version of the anomaly detection model is determined based on the hash value of the training data used in this version update, the model parameters obtained after this version update, and the performance metrics of the anomaly detection model after this version update.

[0123] The smart contract is invoked to write the hash value of the updated version of the anomaly detection model into the blockchain, thus obtaining a blockchain-based notarized record of the updated version of the anomaly detection model.

[0124] For example, after each model update, the hash value H of the incremental training data can be... data The current hyperparameters θ of the model hyper Current performance metrics of the model M performance Package it into structured data, and then calculate its hash value H. model Then, the smart contract is called to transfer the hash value H. model The data is written to the blockchain to obtain a blockchain record of the updated version of the model, ensuring traceability. In case of disputes, the model's reasoning process can be reproduced based on the on-chain data.

[0125] The audit data generation method proposed in this application can call a smart contract to incrementally train the anomaly detection model when the model update trigger condition is met, thereby updating the anomaly detection model and enabling its capabilities to dynamically adapt to newly emerging attack patterns. This allows the anomaly detection model to maintain a high level of performance over a long period, and the accuracy of log anomaly detection results can be guaranteed in the long term. By using federated learning and privacy protection technologies to iterate the anomaly detection model, compared to centralized AI training using raw logs in existing technologies, training data leakage can be avoided, meeting the privacy requirements of training data. After each version update of the anomaly detection model, a smart contract can be called to store the updated version, ensuring the traceability of the anomaly detection model version. Furthermore, this application combines federated learning with privacy protection technologies, using data localization and parameter encryption collaboration to train a better model with more diverse data while ensuring data privacy, compliance, and non-conflicting collaboration.

[0126] Figure 3 This is a flowchart illustrating an audit data generation method provided in another embodiment of this application. Further optimizations and extensions based on the above technical solution are possible, and it can be combined with the various optional implementation methods described above. For example... Figure 3 As shown, the audit data generation method may include the following steps:

[0127] S301. In response to an audit query instruction carrying query conditions for log data, obtain the target log data that meets the query conditions.

[0128] S302. Transform the target log data into structured log data using a large language model.

[0129] In this step, the natural language text in the unstructured logs can be parsed using a large language model to extract key information, thereby converting the unstructured target log data into structured log data.

[0130] For example, if the unstructured log data "User admin failed to log in on 2023-10-01 14:00" is input into the large language model, the large language model can output the structured log data "User=admin, Time=2023-10-01 14:00, Event Type=Login Failed".

[0131] In the embodiments of this application, the accuracy of parsing unstructured logs using a large language model is significantly improved compared to traditional regular expression solutions.

[0132] S303. Construct a time series diagram from the structured log data.

[0133] In this step, structured log data can be constructed into a time series diagram according to time, device (e.g., host), and user dimensions. The time series diagram can reflect the interaction between objects (e.g., devices, users).

[0134] S304. An attack graph is constructed based on a time sequence graph using a graph structure analysis model.

[0135] Graph structure analysis models can process graph structure data based on machine learning. In this step, the graph structure analysis model can be used to analyze and process time-series graphs to construct attack graphs. For example, the graph structure analysis model can generate cross-host attack path graphs (e.g., lateral movement paths).

[0136] In the embodiments of this application, constructing an attack graph using a graph structure analysis model can reduce the false negative rate for identifying multi-hop attack chains.

[0137] S305. Determine the risk parameters corresponding to the target log data through the anomaly detection model.

[0138] S306. Calculate the hash value of the target log data, and the hash value of the current version of the anomaly detection model used when determining risk parameters.

[0139] In this embodiment of the application, the blockchain evidence can specifically be a hash value.

[0140] In this step, the calculation method for the target log data hash value is the same as the calculation method for the blockchain notarization of the log data, allowing the two to be compared to verify the validity of the risk parameters; similarly, the calculation method for the model version hash value is the same as the calculation method for the blockchain notarization of the model version, allowing the two to be compared to verify the validity of the risk parameters.

[0141] In this step, the hash value of the target log data can be calculated using a 256-bit cryptographic hash algorithm.

[0142] Optionally, multiple hash values ​​can be calculated for the target log data in segments, or a single hash value can be calculated for the entire target log data. This application is not intended to limit this.

[0143] Optionally, the target log data can be encrypted first, and then the hash value can be calculated on the encrypted target log data; alternatively, the hash value can be calculated directly on the target log data.

[0144] In this step, the hash value of the current version of the anomaly detection model used in determining the risk parameters in step S305 can be calculated using a 256-bit cryptographic hash algorithm. For example, the hash value of the training data of the current version of the anomaly detection model used in determining the risk parameters, the hyperparameters of the current version of the anomaly detection model used in determining the risk parameters, and the performance metrics of the current version of the anomaly detection model used in determining the risk parameters can be packaged into structured data before calculating its hash value.

[0145] S307. Call the smart contract deployed on the blockchain to compare the hash value of the target log data with the blockchain evidence of the target log data contained in the pre-stored blockchain evidence of the log data, and compare the hash value of the current version of the anomaly detection model used to determine the risk parameters with the blockchain evidence of the current version of the pre-stored anomaly detection model.

[0146] In this step, a smart contract can be invoked to compare the hash value of the currently calculated target log data with the blockchain evidence of the target log data contained in the pre-stored blockchain evidence of the log data to determine if they are consistent; and the hash value of the current version of the currently calculated model can be compared with the pre-stored blockchain evidence of the current version of the model to determine if they are consistent. The core value of this verification method is to replace the verification of the integrity of the original data with the uniqueness of the hash value. While ensuring accuracy, it significantly improves efficiency, reduces costs, and strengthens credibility and traceability through technical means, making it particularly suitable for fields with high requirements for security, real-time performance, and consistency.

[0147] S308. Determine the validity of the risk parameters based on the comparison results.

[0148] If the hash value of the target log data is consistent with the blockchain evidence of the target log data contained in the pre-stored blockchain evidence of the log data, and the hash value of the current version of the anomaly detection model used to determine the risk parameters is consistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, then the risk parameters are determined to be valid.

[0149] If the hash value of the target log data is inconsistent with the blockchain evidence of the target log data contained in the pre-stored blockchain evidence of the log data, and / or the hash value of the current version of the anomaly detection model used to determine the risk parameters is inconsistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, the determination of the risk parameters is invalid.

[0150] In other words, if the comparison results for the target log data and the current version of the model are consistent, it indicates that neither the target log data nor the current version of the model has been tampered with, further demonstrating that the risk parameters determined using the current version of the model for the target log data are valid. If the comparison results for the target log data are inconsistent, it indicates that the target log data has been tampered with, further demonstrating that the risk parameters determined using the current version of the model for the target log data are invalid. If the comparison results for the current version of the model are inconsistent, it indicates that the current version of the model has been tampered with, further demonstrating that the risk parameters determined using the current version of the model for the target log data are invalid. This not only improves the accuracy of business decisions and the stability of system operation, but also reduces risks in compliance, security, and collaboration scenarios, providing a solid guarantee for the enterprise's risk control and long-term operation.

[0151] S309. If the risk parameters are verified to be valid, generate a log audit report based on the risk parameters and attack graph.

[0152] S310, Visualize and display log audit reports.

[0153] In this step, the contents of the log audit report can be visualized through a graphical user interface, allowing users to view the specific content of the log audit report, such as attack graphs and high-risk log content.

[0154] The audit data generation method proposed in this application can transform target log data into structured log data through a large language model. The accuracy of parsing unstructured logs using the large language model is significantly improved compared to traditional regular expression solutions. Furthermore, this method can construct a time-series graph from structured log data and then use a graph structure parsing model to build an attack graph based on the time-series graph. Constructing the attack graph using the graph structure parsing model can reduce the false negative rate for identifying multi-hop attack chains. By incorporating a large language model and a graph structure parsing model into log analysis and processing, the synergy of artificial intelligence technologies can be further deepened, effectively improving the efficiency and accuracy of log auditing, i.e., enhancing the log anomaly detection capability. This application introduces a large language model and graph neural networks into log data processing, specifically addressing the limitations of traditional log processing methods (such as rule engines and simple statistical analysis) in areas such as unstructured parsing, complex relationship mining, and dynamic pattern recognition.

[0155] Figure 4This is a schematic diagram of the structure of an audit data generation device provided in one embodiment of this application. Figure 4 As shown, the audit data generation device 400 includes:

[0156] The data acquisition module 401 is used to respond to an audit query instruction carrying query conditions for log data and acquire target log data that meets the query conditions.

[0157] The graph construction module 402 is used to construct an attack graph based on the target log data;

[0158] The risk parameter determination module 403 is used to determine the risk parameters corresponding to the target log data through an anomaly detection model.

[0159] The validity verification module 404 is used to call the smart contract deployed on the blockchain to verify the validity of the risk parameters based on the blockchain notarization of the pre-stored log data and the blockchain notarization of the current version of the anomaly detection model; the blockchain also pre-stores the encrypted log data.

[0160] The report generation module 405 is used to generate a log audit report based on the risk parameters and the attack graph, provided that the risk parameters have been verified as valid.

[0161] In an optional embodiment, the device is further configured to:

[0162] Based on the attack graph and / or the risk parameters, detect whether the model update triggering conditions are met;

[0163] If the model update triggering condition is met, the smart contract is invoked to incrementally train the anomaly detection model using federated learning and privacy protection technologies, so as to update the version of the anomaly detection model;

[0164] After the anomaly detection model completes its version update, the smart contract is invoked to store the updated version of the anomaly detection model.

[0165] In an optional embodiment, the step of calling the smart contract to store the updated version of the anomaly detection model includes:

[0166] The hash value of the updated version of the anomaly detection model is determined based on the hash value of the training data used in this version update, the model parameters obtained after this version update, and the performance metrics of the anomaly detection model after this version update.

[0167] The smart contract is invoked to write the hash value of the updated version of the anomaly detection model into the blockchain, thereby obtaining a blockchain-based notarized record of the updated version of the anomaly detection model.

[0168] In an optional embodiment, the invocation of the smart contract deployed on the blockchain verifies the validity of the risk parameters based on the pre-stored blockchain evidence of the log data and the current version of the anomaly detection model, including:

[0169] Determine the hash value of the target log data, and the hash value of the current version of the anomaly detection model used when determining the risk parameters;

[0170] The smart contract deployed on the blockchain is invoked to compare the hash value of the target log data with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and to compare the hash value of the current version of the anomaly detection model used when determining the risk parameter with the blockchain evidence of the current version of the anomaly detection model.

[0171] The validity of the risk parameters is determined based on the comparison results.

[0172] In an optional embodiment, determining the validity of the risk parameter based on the comparison results includes:

[0173] If the hash value of the target log data is consistent with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and the hash value of the current version of the anomaly detection model used when determining the risk parameter is consistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, then the risk parameter is determined to be valid.

[0174] If the hash value of the target log data is inconsistent with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and / or the hash value of the current version of the anomaly detection model used when determining the risk parameter is inconsistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, the risk parameter is determined to be invalid.

[0175] In an optional embodiment, constructing the attack graph based on the target log data includes:

[0176] The target log data is transformed into structured log data using a large language model;

[0177] The structured log data is then used to construct a time series diagram.

[0178] The attack graph is obtained by constructing the time sequence graph using a graph structure analysis model.

[0179] The audit data generation apparatus described above can execute the methods provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the methods. Technical details not described in detail in this embodiment can be found in the audit data generation methods provided in any embodiment of this application.

[0180] It should be noted that the information collected in this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, necessary confidentiality measures have been taken, and it does not violate public order and good morals. Corresponding operation portals are provided for users to choose to authorize or refuse.

[0181] This invention also provides a computer program product.

[0182] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer program products, which may include one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be an application-specific or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, with reference to... Figure 5 , Figure 5 The electronic device 12 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this application. Figure 5 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0184] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0185] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0186] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0187] A program / utility 40 having a set (at least one) of program modules 46 may be stored, for example, in memory 28. Such program modules 46 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 46 typically perform the functions and / or methods described in the embodiments of this application.

[0188] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although... Figure 5 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0189] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28. For example, it implements an audit data generation method provided in this embodiment of the invention: in response to an audit query instruction carrying query conditions for log data, it obtains target log data that meets the query conditions; it constructs an attack graph based on the target log data; it determines the risk parameters corresponding to the target log data through an anomaly detection model; it calls a smart contract deployed on the blockchain to verify the validity of the risk parameters based on the blockchain notarization of the pre-stored log data and the current version of the anomaly detection model; encrypted log data is also pre-stored on the blockchain; and if the risk parameters are verified to be valid, it generates a log audit report based on the risk parameters and the attack graph.

[0190] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements an audit data generation method as provided in all embodiments of this invention: responding to an audit query instruction carrying query conditions for log data, obtaining target log data that meets the query conditions; constructing an attack graph based on the target log data; determining risk parameters corresponding to the target log data through an anomaly detection model; invoking a smart contract deployed on the blockchain to verify the validity of the risk parameters based on pre-stored blockchain evidence of the log data and the current version of the anomaly detection model's blockchain evidence; encrypted log data is also pre-stored on the blockchain; and, if the risk parameters are verified to be valid, generating a log audit report based on the risk parameters and the attack graph. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic device, apparatus, or device that is electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an electronic device, apparatus, or device that executes instructions.

[0191] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in conjunction with an electronic device, apparatus, or device that executes instructions.

[0192] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0193] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0194] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for generating audit data, characterized in that, The method includes: In response to an audit query instruction carrying query conditions for log data, the target log data that meets the query conditions is obtained; An attack graph is constructed based on the target log data; The risk parameters corresponding to the target log data are determined using an anomaly detection model. The smart contract deployed on the blockchain is invoked to verify the validity of the risk parameters based on the blockchain notarization of the pre-stored log data and the blockchain notarization of the current version of the anomaly detection model; the encrypted log data is also pre-stored on the blockchain. If the risk parameters are verified to be valid, a log audit report is generated based on the risk parameters and the attack graph.

2. The method according to claim 1, characterized in that, The method further includes: Based on the attack graph and / or the risk parameters, detect whether the model update triggering conditions are met; If the model update triggering condition is met, the smart contract is invoked to incrementally train the anomaly detection model using federated learning and privacy protection technologies, so as to update the version of the anomaly detection model; After the anomaly detection model completes its version update, the smart contract is invoked to store the updated version of the anomaly detection model.

3. The method according to claim 2, characterized in that, The step of calling the smart contract to store the updated version of the anomaly detection model includes: The hash value of the updated version of the anomaly detection model is determined based on the hash value of the training data used in this version update, the model parameters obtained after this version update, and the performance metrics of the anomaly detection model after this version update. The smart contract is invoked to write the hash value of the updated version of the anomaly detection model into the blockchain, thereby obtaining a blockchain-based notarized record of the updated version of the anomaly detection model.

4. The method according to claim 1, characterized in that, The invocation of the smart contract deployed on the blockchain verifies the validity of the risk parameters based on the pre-stored blockchain evidence of the log data and the current version of the anomaly detection model, including: Determine the hash value of the target log data, and the hash value of the current version of the anomaly detection model used when determining the risk parameters; The smart contract deployed on the blockchain is invoked to compare the hash value of the target log data with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and to compare the hash value of the current version of the anomaly detection model used when determining the risk parameter with the blockchain evidence of the current version of the anomaly detection model. The validity of the risk parameters is determined based on the comparison results.

5. The method according to claim 1, characterized in that, The step of determining the validity of the risk parameter based on the comparison results includes: If the hash value of the target log data is consistent with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and the hash value of the current version of the anomaly detection model used when determining the risk parameter is consistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, then the risk parameter is determined to be valid. If the hash value of the target log data is inconsistent with the blockchain evidence of the target log data included in the pre-stored blockchain evidence of the log data, and / or the hash value of the current version of the anomaly detection model used when determining the risk parameter is inconsistent with the blockchain evidence of the current version of the pre-stored anomaly detection model, the risk parameter is determined to be invalid.

6. The method according to claim 1, characterized in that, The attack graph constructed based on the target log data includes: The target log data is transformed into structured log data using a large language model; The structured log data is then used to construct a time series diagram. The attack graph is obtained by constructing the time sequence graph using a graph structure analysis model.

7. An audit data generation device, characterized in that, The device includes: The data acquisition module is used to respond to an audit query instruction carrying query conditions for log data and acquire target log data that meets the query conditions. The graph construction module is used to construct an attack graph based on the target log data. The risk parameter determination module is used to determine the risk parameters corresponding to the target log data through an anomaly detection model. The validity verification module is used to call the smart contract deployed on the blockchain to verify the validity of the risk parameters based on the blockchain notarization of the pre-stored log data and the blockchain notarization of the current version of the anomaly detection model; the blockchain also pre-stores the encrypted log data. The report generation module is used to generate a log audit report based on the risk parameters and the attack graph, provided that the risk parameters have been verified as valid.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the audit data generation method as described in any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the audit data generation method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the audit data generation method as described in any one of claims 1 to 6.

Images