Context reconstruction method and device for audit log, equipment and medium
By caching, preprocessing, feature extraction, and time-series learning model inference of Kubernetes audit logs, the problem of the inability to automatically reconstruct distributed logs is solved, and the complete recovery of operational semantics and efficient log analysis are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2026-01-06
- Publication Date
- 2026-04-17
AI Technical Summary
Existing technologies cannot automatically reconstruct fragmented Kubernetes audit logs, making it difficult to fully and accurately recover operational semantics and scope of impact.
By acquiring audit log data streams, caching and preprocessing them, extracting feature vector sequences, and using time-series learning models for inference, a context label sequence is output. Finally, clustering and context reconstruction are performed to generate a structured set of operation instances.
It enables automated context reconstruction of distributed logs, improving log readability and understandability, enhancing analysis efficiency, strengthening the ability to identify and locate complex operations, and simplifying the compliance audit and liability determination process.
Smart Images
Figure CN121880146A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and medium for context reconstruction of audit logs. Background Technology
[0002] Currently, Kubernetes (K8s) is widely used in production environments, and its audit logs record user and system API interactions over time. Existing practices often rely on rule-based security detection tools (such as Falco) or log aggregation and retrieval platforms (such as Elasticsearch and Splunk) for single-log-level retrieval and alerting. This makes it difficult to automatically correlate multiple logs triggered by the same initial operation (e.g., creating a Deployment) and distributed across resource objects and time series into a complete operation chain in the absence of explicit causal identification. Current technologies cannot automatically reconstruct the context of scattered logs, making it difficult to fully and accurately recover the operation semantics and scope of impact. Summary of the Invention
[0003] This invention provides a method, apparatus, device, and medium for context reconstruction of audit logs, in order to solve the technical problem that existing methods cannot perform automated context reconstruction of scattered logs, resulting in the difficulty in fully and accurately restoring the operational semantics and scope of impact.
[0004] Firstly, a method for context reconstruction of audit logs is provided, including: Obtain the audit log data stream and cache it to obtain the original log sequence; The original log sequence is preprocessed to obtain a structured log sequence; Feature extraction is performed on the structured log sequence to obtain a feature vector sequence; The feature vector sequence is input into a time-series learning model for inference, and a context label sequence is output. Cluster the context label sequence to obtain a set of operation instances; The set of operation instances is reconstructed in context, and the logs are concatenated in chronological order, and the context reconstruction result is output.
[0005] Secondly, a context reconstruction apparatus for audit logs is provided, including: The data caching module is used to acquire and cache the audit log data stream to obtain the original log sequence; The data processing module is used to preprocess the original log sequence to obtain a structured log sequence; The feature extraction module is used to extract features from the structured log sequence to obtain a feature vector sequence; The data inference module is used to input the feature vector sequence into the time-series learning model for inference and output the context label sequence; The clustering processing module is used to perform clustering processing on the context label sequence to obtain a set of operation instances; The reconstruction output module is used to reconstruct the context of the set of operation instances, concatenate the logs in chronological order, and output the context reconstruction result.
[0006] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the aforementioned audit log context reconstruction method.
[0007] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the aforementioned audit log context reconstruction method.
[0008] The aforementioned audit log context reconstruction method, apparatus, device, and medium implement a scheme that involves acquiring and caching the audit log data stream to obtain an original log sequence; preprocessing the original log sequence to obtain a structured log sequence; extracting features from the structured log sequence to obtain a feature vector sequence; inputting the feature vector sequence into a time-series learning model for inference to output a context label sequence; clustering the context label sequence to obtain an operation instance set; reconstructing the context of the operation instance set, concatenating the logs in chronological order, and outputting the context reconstruction result. In this invention, addressing the problem that existing methods cannot automatically reconstruct the context of scattered logs, resulting in difficulty in fully and accurately recovering the operation semantics and scope of impact, this invention solves the problem of scattered and unrelated logs by caching and preprocessing the audit log data stream, extracting a feature vector sequence, inputting it into a time-series learning model for inference to obtain a context label sequence, and finally reconstructing the context. Attached Figure Description
[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 This is a flowchart illustrating a context reconstruction method for audit logs according to an embodiment of the present invention; Figure 2 yes Figure 1A detailed implementation flow diagram of step S10 Figure 1 ; Figure 3 yes Figure 1 A detailed implementation flow diagram of step S20 Figure 2 ; Figure 4 yes Figure 1 A detailed implementation flow diagram of step S30 Figure 3 ; Figure 5 yes Figure 1 A detailed implementation flow diagram of step S40 Figure 4 ; Figure 6 yes Figure 1 A detailed implementation flow diagram of step S50 Figure 5 ; Figure 7 yes Figure 1 A detailed implementation flow diagram of step S60 Figure 6 ; Figure 8 This is a schematic diagram of a context reconstruction device for audit logs according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 10 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation
[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0012] Please see Figure 1 As shown, Figure 1 A flowchart illustrating a context reconstruction method for audit logs provided in an embodiment of the present invention includes the following steps: S10: Obtain the audit log data stream and cache it to obtain the original log sequence.
[0013] Connect to the Kubernetes audit backend and receive JSON format audit logs in real time via webhook or file reading. Perform preliminary parsing and buffering to obtain the log stream (i.e., the original log sequence, the same below).
[0014] Combination Figure 2As shown, step S10 specifically includes the following steps: S101: Access and parse the corresponding configuration to obtain the access method identifier.
[0015] After the system starts, it first accesses and parses the corresponding access configuration. The parsed items include at least the access method (webhook / file), authentication method, target path (such as callback URL or log directory), security and retry parameters, etc. The parsing result is fixed as the access method identifier, which is used to drive the creation of subsequent channels.
[0016] S102: The channel initializes the access method identifier to establish a callback endpoint and obtain the input channel.
[0017] The channel is initialized with an access method identifier to establish a callback endpoint. In webhook mode, an HTTP / HTTPS callback endpoint with two-way authentication capability is registered and the token / certificate is loaded. In file mode, a read-only file handle and a round-robin listener are established for the specified audit directory to obtain the input channel. In financial and medical scenarios, the least privilege access and audit private network entry and exit whitelist can be enabled according to the institution's security baseline.
[0018] S103: Incrementally read the input channel to obtain a log fragment stream.
[0019] Incremental reading is performed on the input channel. In webhook mode, the incoming request body is buffered into log fragments at the event granularity. In file mode, the newly added intervals are read according to the file growth offset (tailing). Both are uniformly converted into a log fragment stream arranged in the order of arrival, ensuring that bounded memory usage and replayability are maintained during peak log periods.
[0020] S104: Perform integrity reconstruction and format verification on the log fragment stream to obtain a normalized record stream.
[0021] The log fragment stream is subjected to integrity reorganization and format verification. First, the JSON fragments that cross packets or lines are reorganized based on the delimiter and structural boundaries. Then, syntax-level compliance verification and pattern-level field verification are performed to remove corrupted records and supplement necessary metadata (such as receiving timestamps and source identifiers) to output a normalized record stream.
[0022] S105: Use the normalized record stream to perform key-value mapping to obtain a cache entry sequence.
[0023] By using a normalized record stream for key-value mapping and standardization, key fields are mapped to a unified data structure for subsequent feature extraction and modeling. The mapping items at least cover core elements such as object reference API groups, resources, operation verbs, subject to group, and owner reference UID or associated objects, ultimately resulting in a cached entry sequence, which lays the foundation for building a fixed-length, quantifiable feature representation.
[0024] S106: Write the cache entry sequence into the cache queue in chronological order and maintain the offset index to obtain the original log sequence.
[0025] The cached entry sequence (a single record with key-value mapping and supplemented metadata) is written into the cache queue according to a stable sorting strategy of event timestamp and reception time, while maintaining a monotonically increasing offset index to form a raw log sequence that can be consumed sequentially and resumed from breakpoints. In deployments aimed at financial transactions and medical business continuity, the cache queue and offset index can be combined with available storage to achieve cross-instance migration to ensure the integrity and continuity of the audit chain.
[0026] S20: Perform data preprocessing on the original log sequence to obtain a structured log sequence.
[0027] The log stream undergoes normalization preprocessing, including missing field filling and nested JSON structure flattening, to unify the data structure and obtain a structured log sequence.
[0028] Combination Figure 3 As shown, step S20 specifically includes the following steps: S201: Perform syntax parsing on the original log sequence to obtain a compliant log sequence.
[0029] Syntax parsing of the original log sequence can be performed using JSON syntax validation and schema validation (SchemaCheck). Records that cross packages or lines are first reconstructed, and then entries with non-compliant syntax or broken field structures are removed to output a compliant log sequence.
[0030] S202: Complete the missing fields of the compliant log sequence to obtain the completed log sequence.
[0031] The compliant log sequence is filled with missing fields. For key fields of Kubernetes audit logs (such as objectRef.apiGroup, objectRef.resource, verb, user.groups, requestObject.metadata.ownerReferences[0].uid or responseObject.involvedObject), when they are missing or empty, the default value or placeholder is filled in according to the preset rules, and the receiving timestamp, source identifier and other meta information are added to obtain the completed log sequence (this completion process belongs to the "field filling" processing of the log stream).
[0032] S203: Perform hierarchical flattening processing on the completed log sequence to obtain a flattened log sequence.
[0033] The log sequence is flattened by expanding the nested JSON hierarchy into a single key name and maintaining the traceability of the key path, resulting in a flattened log sequence (this step corresponds to the unified structured processing of "nested JSON structure flattening").
[0034] S204: Determine the data type of the flattened log sequence, perform unified unit conversion, and output a normalized log sequence.
[0035] The system determines the data type of the flattened log sequence and performs a unified unit conversion, including explicit calibration of Boolean, numeric, string and timestamp types, as well as standardization of duration (e.g. ns→ms), capacity (e.g. B→MiB) and time (unified to UTC, ISO8601), and outputs a normalized log sequence.
[0036] S205: Arrange the time order of the normalized log sequence to obtain a time-aligned log sequence.
[0037] By organizing and standardizing the time sequence of logs, prioritizing event timestamps and using reception time as a secondary factor for stable sorting, and performing error correction and deduplication on out-of-order and duplicate records, a time-aligned log sequence can be obtained.
[0038] S206: Remap the key names of the time-aligned log sequence according to the preset data structure mapping table to obtain a structured log sequence.
[0039] The time-series aligned log sequences are remapped using a pre-defined data structure mapping table to create a structured key space for subsequent feature extraction, resulting in a structured log sequence.
[0040] It is important to note that in implementations targeting financial and healthcare scenarios, steps S202 and S204 can be enhanced based on the institution's security baseline and compliance requirements. For example, in financial transaction audits, mandatory completion and whitelist verification can be performed on the user.groups and namespace fields; in healthcare information systems, time zones and time formats can be standardized to meet cross-system audit traceability requirements; and controlled desensitization strategies can be applied to sensitive identifier fields (such as user-related identifiers or account-related identifiers) before proceeding to step S206 to ensure that audit compliance and privacy protection requirements are met without affecting subsequent feature extraction and context reconstruction.
[0041] S30: Extract features from the structured log sequence to obtain a feature vector sequence.
[0042] In the feature extraction stage, preset key features are extracted from each structured log and numerically encoded, including at least API groups, resources, operation verbs, user groups, and owner reference UIDs or associated objects, to construct a feature vector sequence.
[0043] Combination Figure 4 As shown, step S30 specifically includes the following steps: S301: Expand the structured log sequence to create a set of field records.
[0044] First, the structured log sequence is expanded, and the nested key path and its value of each log are extracted into a standardized set of field records. Event timestamps, source identifiers, and request / response key information are retained for subsequent connection.
[0045] S302: Extract the object reference field group from the field record set and perform dictionary mapping to obtain the object reference encoding set.
[0046] Extract the object reference field group (including objectRef.apiGroup, objectRef.resource, and objectRef.subresource) from the field record set, and perform stable integer ID mapping according to the offline constructed dictionary to obtain the object reference encoding set.
[0047] S303: Based on the object reference encoding set, the operation verb field of the corresponding record is introduced according to the record index and concatenated to obtain the operation enhancement encoding set.
[0048] Based on the record index, the operation verb field is joined to the object reference encoding set, and the same dictionary mapping is applied to the verb values (such as create, delete, patch, etc.) to obtain the operation enhanced encoding set.
[0049] S304: The operation enhancement code set is introduced into the subject group field of the corresponding record according to the record index and multi-value encoding is performed to obtain the subject enhancement code set.
[0050] We continue to introduce the user.groups field, which is the group to which the subject belongs, based on the record index. We use multi-hot / set encoding for its multi-value entries to obtain the subject enhanced encoding set. In the financial and medical compliance scenarios, we can incorporate institutional domains (such as transaction audit domains / clinical operation and maintenance domains) and role domains (such as auditors / transaction risk control, medical information department / medical and nursing subgroups) into the encoding space in a hierarchical manner to ensure semantic consistency across tenants and systems.
[0051] S305: The main enhanced coding set is introduced into the associated identification information of the corresponding record according to the record index and the identification mapping is performed to obtain the associated enhanced coding set.
[0052] The associated identification information is introduced by the record index and the identification mapping is performed: when there is an owner reference of the request object, requestObject.metadata.ownerReferences[0].uid can be extracted; otherwise, the associated object field responseObject.involvedObject of the response object is extracted. The two are converted into discrete codes through a unified identification mapping strategy, thereby obtaining an association-enhanced code set.
[0053] S306: Perform dimension alignment and concatenation on the association enhancement encoding set respectively to obtain a fixed-length feature vector set.
[0054] The association enhancement encoding set is dimensionally aligned and concatenated in a preset order to construct a fixed-length feature vector set. In this embodiment, a total of 39 key features are extracted to form a 39-dimensional fixed-length vector (the field order and value space are kept consistent during the training and inference stages).
[0055] S307: Perform numerical normalization on the set of fixed-length feature vectors to obtain a single log feature vector.
[0056] The fixed-length feature vector set is numerically normalized, meaning that discrete fields retain their sparse / dense encoding, while numerical fields (such as resource quotas and object version numbers) are scaled using interval scaling or logarithmic scaling to unify the units of measurement. Timestamp-related fields (such as the difference between event time and reception time) are converted into comparable standard metrics. Controlled desensitization mapping is applied to potentially sensitive domain identifiers (such as secondary group codes for financial account domains / medical user domains) before they enter the training / inference process, ultimately resulting in a single log feature vector. This approach satisfies industry privacy compliance without compromising feature stability.
[0057] S308: Encapsulate the single log feature vector in chronological order to obtain a feature vector sequence.
[0058] The normalized single log feature vector is encapsulated into a feature vector sequence according to the event time order, and the offset index within the sequence is recorded to support subsequent bidirectional association with context labels and instance clustering results, thus obtaining the feature vector sequence.
[0059] S40: Input the feature vector sequence into the time-series learning model for inference and output the context label sequence.
[0060] The feature vector sequence is input into a pre-trained temporal learning model (such as a model based on a recurrent neural network RNN) for inference. A bidirectional long short-term memory network (BiLSTM) is preferred. The sequence is mapped through an input layer, a bidirectional BiLSTM layer and a fully connected layer to output a context label sequence used to identify the type of the initial triggering event.
[0061] Combination Figure 5 As shown, step S40 specifically includes the following steps: S401: Align the feature vector sequence by length in chronological order to obtain the model input tensor.
[0062] The feature vectors obtained in step S30 are organized into sequences of equal length or batch processing according to the event time order to construct the model input tensor for inference by the time-series learning model; this serialized input is the input data form.
[0063] S402: Perform an embedding transformation on the input tensor of the model to obtain an embedding sequence.
[0064] Embedding transformation is performed on the model input tensor to map discrete codes (such as resource ID, sub-resource ID, verb ID, etc.) into dense vector representations. This can be regarded as the vectorization preprocessing of feature vectors by the input layer to obtain the embedding sequence.
[0065] S403: The embedded sequence is input into the forward and backward long short-term memory units respectively for processing to obtain the forward representation sequence and the backward representation sequence.
[0066] The embedded sequence is fed into the forward and backward long short-term memory units (two opposing time-series units in the BiLSTM structure) respectively, and the bidirectional dependency relationship of the log sequence is modeled in parallel to obtain the forward representation sequence and the backward representation sequence respectively.
[0067] S404: The forward and backward representation sequences are concatenated in the feature dimension and input into a fully connected mapping to obtain a temporal hidden state sequence.
[0068] The forward / backward representations are concatenated in the feature dimension and integrated into a fully connected mapping to output a temporal hidden state sequence.
[0069] S405: Perform classification mapping on the temporal hidden state sequence to obtain a label probability distribution sequence.
[0070] The output layer performs classification mapping on the temporal hidden state (classification on a time-by-time basis) to obtain the label probability distribution sequence, thereby enabling the prediction of the "initial trigger event type" to which each log belongs.
[0071] S406: Based on the preset encoding rules, the category index of the label probability distribution sequence is mapped to a quintuple label to obtain the context label sequence.
[0072] According to a preset encoding rule, the category index of the tag probability distribution sequence is mapped to a quintuple label L = (r, s, v, n, u), where r represents the resource ID, s represents the sub-resource ID, v represents the operation verb ID, n represents whether it is a namespace scope (Boolean identifier), and u represents whether it is a single object operation (Boolean identifier), thus forming a context tag sequence aligned in time order. This quintuple is used to uniquely identify the initial triggering event type to which the log belongs, and ensures that subsequent logs triggered by the same triggering operation receive consistent labels during the online inference phase (e.g., the "create namespace" scenario).
[0073] In practical implementation, offline supervised learning can be used in the model training phase, based on labeled K8s audit logs (clustered by operation type and assigned uniform labels). In the online phase, label sequences are generated according to the above steps S401-S406 and sent back to the subsequent clustering and context reconstruction stages to support compliance traceability and liability determination, including financial transaction audits and medical information system operation and maintenance audits.
[0074] S50: Perform clustering processing on the context label sequence to obtain a set of operation instances.
[0075] Based on the obtained context label sequence, the logs are clustered in two stages: first, they are grouped into temporary groups according to the same label, and then within the group, they are further subdivided and associated with the same instance according to the object reference UID, owner reference or associated object information to form a set of operation instances.
[0076] Combination Figure 6 As shown, step S50 specifically includes the following steps: S501: The context label sequences are indexed, aligned, and merged to obtain the labeled log sequences.
[0077] The context label sequence obtained in step S40 is indexed, aligned, and merged. Each log entry is recorded along with its time offset and source channel offset to obtain a traceable labeled log sequence.
[0078] S502: Group and aggregate the labeled log sequences according to context labels to obtain a temporary group set.
[0079] The labeled logs are grouped and aggregated based on the context label L to obtain a temporary group set; for example, all logs labeled "create namespace" are first put into the same group to ensure that similar triggering events are aggregated at the label level first.
[0080] S503: Extract the association identifier of the log record in the temporary group set, and form an association key together with the name identifier to obtain the association key table.
[0081] The association identifiers used for instance differentiation are extracted from each temporary group. These identifiers preferably include object reference identifiers (such as objectRef.uid), owner reference information (ownerReferences.uid), or related object information (responseObject.involvedObject). Together with the object name identifier (such as objectRef.name), they form the association key, generating an association key table. In the "Create Namespace" group, different created objects (such as ns-finance and ns-research) are distinguished by their names or UIDs and assigned to different keys.
[0082] S504: Perform same-key aggregation and transitive association merging on the association key table to obtain an association cluster set.
[0083] Same-key aggregation and transitive association merging are performed on the association key table. Same-key aggregation is used to converge logs with completely identical keys into the same candidate cluster. Transitive association merging is used to handle causal chains across resources. If the "related object UID of the response object" of log A is the same as the "object reference UID" of log B, then they are determined to belong to the same operation instance and merged into the same association cluster. For complex operations (such as creating a Deployment), the multi-hop relationship of Deployment→ReplicaSet→Pod is chained by passing ownerReferences.uid to obtain a continuous instance-level log chain (i.e., a set of association clusters).
[0084] S505: Sort the log records within each cluster in the associated cluster set according to the event time order to obtain the instance time sequence.
[0085] Logs within each associated cluster are stably sorted by event timestamps, and order compensation is performed based on the receiving time to output the instance time sequence, ensuring semantic consistency and replayability of actions within the instance.
[0086] S506: Generate instance identifiers using the instance time sequence, establish a bidirectional index between instances and logs, and output the set of operation instances.
[0087] Each instance time sequence is assigned a globally unique instance identifier (which can be constructed by combining the tag L with the start time / first UID hash). At the same time, a bidirectional index is established for "instance → log record" and "log record → instance". The set of operation instances is output for subsequent context concatenation and structured encapsulation. In financial and medical scenarios, the above process can accurately distinguish the operation key values under different business domains or department namespaces (such as ns-finance and ns-emr), which not only avoids cross-domain merging errors, but also facilitates tracing back the complete operation chain along the instance identifier in compliance audits.
[0088] S60: Reconstruct the context of the set of operation instances, concatenate the logs in chronological order, and output the context reconstruction result.
[0089] The logs within each operation instance are organized and structured according to the event time sequence, and output in a preset format (such as a JSON array) for consumption by upper-layer security analysis or auditing systems (i.e., context reconstruction results).
[0090] Combination Figure 7 As shown, step S60 specifically includes the following steps: S601: Perform record aggregation processing on the set of operation instances to obtain a set of instance records.
[0091] The operation instance set is processed by record aggregation. Based on the instance identifier, the log record list under the same instance name, instance start and end time, association key and source channel information are summarized to obtain the instance record set (the operation instance set can be obtained by two-stage clustering based on context label and association identifier).
[0092] S602: Sort the log records of each instance in the instance record set according to the event timestamp to obtain a time-series sorted set.
[0093] The log records of each instance in the instance record set are stably sorted according to the event timestamp, and the receipt time is used as the secondary arbiter for the time in which they are ranked, resulting in a time-series sorted set.
[0094] S603: Concatenate adjacent logs in the time sequence of the time sequence sorting set to obtain the instance time sequence concatenation sequence.
[0095] The temporal sequence of the sorted set is concatenated with the temporal order of adjacent logs. Based on the temporal sequence of two adjacent logs and the causal relationship within the instance, an ordered chain of "event-subsequent event" is generated, resulting in the instance temporal sequence concatenation sequence.
[0096] S604: Establish cross-log reference relationships based on the instance time-series concatenation sequence and generate instance-level indexes to obtain instance context indexes.
[0097] Establish cross-log reference relationships based on the instance time sequence, preferably with the endpoints of the edges being object reference identifiers (such as objectRef.uid), owner references (ownerReferences.uid), and associated object information (responseObject.involvedObject), to generate instance-level indexes (including three types of indexes: "instance → log record", "log record → instance", and "log record → log record"), thus obtaining the instance context index.
[0098] S605: Perform structured encapsulation on the instance context index to generate a structured context record set.
[0099] The instance context index is encapsulated in a structured manner to meet the needs of general consumption and audit traceability. Elements such as instance identifier, context tag, timeline, and cross-log references (edges) are organized into a unified set of data objects to generate a set of structured context records.
[0100] S606: Perform target formatting on the structured context record set to convert it into a preset structured array format to obtain the context reconstruction result.
[0101] The structured context record set is formatted into a preset structured array format (preferably a JSON array) to facilitate integration with Security Information and Event Management Systems (SIEM) and alarm platforms, or to perform persistent storage, resulting in the final context reconstruction. In the financial sector, this data can be directly fed into the Transaction Compliance Audit and Security Operations Center (SOC) for rapid attack chain tracing. In the healthcare sector, it can be integrated into the Hospital Information System (HIS / EMR) security audit module to support incident review and accountability.
[0102] As can be seen, this invention forms a closed loop through log acquisition and preprocessing, feature extraction, context label prediction, and two-stage clustering reconstruction, enabling the automated reconstruction of scattered Kubernetes audit logs into a traceable "operation storyline." Specifically, a pre-trained temporal learning model (preferably BiLSTM) is first used to predict the context label for each log entry. Then, based on the label and identifiers such as UID / owner reference / associated object, a two-stage clustering process is implemented, first by label, then by identifier. Finally, the initial triggering event and its causally related subsequent events are aggregated into a complete operation context and output as a structured result. This automatically reconstructs the operation context, significantly improving the readability and understandability of the logs. Simultaneously, by organizing and outputting massive amounts of logs according to context, periodic "noise" records are systematically isolated, allowing analysts or upper-level automated systems to directly focus on a small number of high-value operation instances, resulting in an order-of-magnitude improvement in analysis efficiency and a significantly improved signal-to-noise ratio.
[0103] In security and operations scenarios, the reconstructed complete context enables precise definition of complex, multi-step attack or failure modes, thereby improving the ability to identify and locate "low, slow, and small" penetration chains and cascading failure root causes, and enhancing the security and stability of critical industry systems such as finance. Furthermore, the structured context results provide a clear chain of evidence in the form of "who, when, what, and its subsequent impact," facilitating direct integration with SIEM / alarm platforms or audit systems, significantly simplifying compliance audits and liability determination processes in heavily regulated industries such as finance and insurance.
[0104] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0105] In one embodiment, an audit log context reconstruction apparatus is provided, which corresponds one-to-one with the audit log context reconstruction method described in the above embodiments. For example... Figure 8 As shown, the context reconstruction device for the audit log includes: a data caching module 100, a data processing module 200, a feature extraction module 300, a data inference module 400, a clustering processing module 500, and a reconstruction output module 600.
[0106] Detailed descriptions of each functional module are as follows: Data caching module 100 is used to acquire and cache audit log data streams to obtain the original log sequence; Data processing module 200 is used to preprocess the original log sequence to obtain a structured log sequence; The feature extraction module 300 is used to extract features from the structured log sequence to obtain a feature vector sequence; The data inference module 400 is used to input the feature vector sequence into the time-series learning model for inference and output the context label sequence; Clustering processing module 500 is used to perform clustering processing on the context label sequence to obtain a set of operation instances; The reconstruction output module 600 is used to reconstruct the context of the operation instance set, concatenate the logs in chronological order, and output the context reconstruction result.
[0107] In one embodiment, the data caching module 100 is specifically used for: Access and parse the corresponding configuration to obtain the access method identifier; The channel initializes the access method identifier to establish a callback endpoint, thus obtaining the input channel; Incremental reading is performed on the input channel to obtain a log fragment stream; The log fragment streams are subjected to integrity reconstruction and format verification respectively to obtain a normalized record stream; The normalized record stream is used for key-value mapping to obtain a sequence of cached entries; The cache entry sequence is written into the cache queue in chronological order and an offset index is maintained to obtain the original log sequence.
[0108] In one embodiment, the data processing module 200 is specifically used for: The original log sequence is parsed to obtain a compliant log sequence; The missing fields of the compliant log sequence are filled in to obtain the completed log sequence; The completed log sequence is then flattened hierarchically to obtain a flattened log sequence. The data type of the flattened log sequence is determined, and the unit is uniformly converted to output a normalized log sequence. The time order of the normalized log sequence is arranged to obtain a time-aligned log sequence; The time-aligned log sequence is remapped using a preset data structure mapping table to obtain a structured log sequence.
[0109] In one embodiment, the feature extraction module 300 is specifically used for: Expand the structured log sequence to create a set of field records; Extract the object reference field group from the field record set and perform dictionary mapping to obtain the object reference encoding set; Based on the object reference encoding set, the operation verb field of the corresponding record is introduced according to the record index and then concatenated and encoded to obtain the operation enhancement encoding set; The operation enhancement code set is introduced into the subject group field of the corresponding record according to the record index and multi-value encoding is performed to obtain the subject enhancement code set; The main enhanced coding set is introduced into the associated identification information of the corresponding record according to the record index and the identification mapping is performed to obtain the associated enhanced coding set; The association enhancement encoding set is dimensionally aligned and concatenated in a preset order to obtain a fixed-length feature vector set; The fixed-length feature vector set is numerically normalized to obtain a single log feature vector; The feature vectors of a single log entry are encapsulated in chronological order to obtain a sequence of feature vectors.
[0110] In one embodiment, the data inference module 400 is specifically used for: The feature vector sequence is length-aligned according to time order to obtain the model input tensor; The model input tensor is subjected to an embedding transformation to obtain an embedding sequence; The embedded sequences are input into the forward and backward long short-term memory units for processing, respectively, to obtain the forward representation sequence and the backward representation sequence. The forward and backward representation sequences are concatenated in the feature dimension and input into a fully connected mapping to obtain a temporal hidden state sequence; The temporal hidden state sequence is classified and mapped to obtain a label probability distribution sequence; Based on preset encoding rules, the category index of the label probability distribution sequence is mapped to a quintuple label to obtain the context label sequence.
[0111] In one embodiment, the clustering processing module 500 is specifically used for: The context label sequences are index-aligned and merged to obtain labeled log sequences; The labeled log sequences are grouped and aggregated according to context labels to obtain a temporary group set; Extract the association identifiers of log records from the temporary group set, and combine them with the name identifiers to form an association key, thus obtaining an association key table; Perform same-key aggregation and transitive association merging on the association key tables respectively to obtain an association cluster set; The log records within each cluster in the associated cluster set are sorted according to the event time order to obtain the instance time sequence; Instance identifiers are generated using the instance time sequence, and a bidirectional index between instances and logs is established to output a set of operation instances.
[0112] In one embodiment, the reconstruction output module 600 is specifically used for: The set of operation instances is subjected to record aggregation processing to obtain a set of instance records; The log records of each instance in the instance record set are sorted according to the event timestamp to obtain a time-series sorted set; The temporal sequence of adjacent logs is concatenated to obtain the instance temporal sequence concatenation sequence. Based on the instance time-series concatenation sequence, a cross-log reference relationship is established, and an instance-level index is generated to obtain the instance context index; The instance context index is structurally encapsulated to generate a set of structured context records; The structured context record set is formatted to convert it into a preset structured array format, thus obtaining the context reconstruction result.
[0113] Specific limitations regarding the context reconstruction apparatus for audit logs can be found in the limitations of the context reconstruction method for audit logs above, and will not be repeated here. Each module in the aforementioned context reconstruction apparatus for audit logs can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can invoke and execute the operations corresponding to each module.
[0114] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the server-side functions or steps of an audit log context reconstruction method.
[0115] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements client-side functions or steps of an audit log context reconstruction method.
[0116] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed, can perform the steps provided in the above embodiments.
[0117] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0118] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0119] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0120] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method for context reconstruction of an audit log, characterized by, include: Obtain the audit log data stream and cache it to obtain the original log sequence; The original log sequence is preprocessed to obtain a structured log sequence; Feature extraction is performed on the structured log sequence to obtain a feature vector sequence; The feature vector sequence is input into a time-series learning model for inference, and a context label sequence is output. Cluster the context label sequence to obtain a set of operation instances; The set of operation instances is reconstructed in context, and the logs are concatenated in chronological order, and the context reconstruction result is output.
2. The method of contextually reconstructing an audit log of claim 1, wherein, The process of acquiring and caching the audit log data stream to obtain the original log sequence includes: Access and parse the corresponding configuration to obtain the access method identifier; The channel initializes the access method identifier to establish a callback endpoint, thus obtaining the input channel; Incremental reading is performed on the input channel to obtain a log fragment stream; The log fragment streams are subjected to integrity reconstruction and format verification respectively to obtain a normalized record stream; The normalized record stream is used for key-value mapping to obtain a sequence of cached entries; The cache entry sequence is written into the cache queue in chronological order and an offset index is maintained to obtain the original log sequence.
3. The method of contextually reconstructing an audit log of claim 1, wherein, The step of preprocessing the original log sequence to obtain a structured log sequence includes: The original log sequence is parsed to obtain a compliant log sequence; The missing fields of the compliant log sequence are filled in to obtain the completed log sequence; The completed log sequence is then flattened hierarchically to obtain a flattened log sequence. The data type of the flattened log sequence is determined, and the unit is uniformly converted to output a normalized log sequence. The time order of the normalized log sequence is arranged to obtain a time-aligned log sequence; The time-aligned log sequence is remapped using a preset data structure mapping table to obtain a structured log sequence.
4. The method of contextually reconstructing an audit log of claim 1, wherein, The step of extracting features from the structured log sequence to obtain a feature vector sequence includes: Expand the structured log sequence to create a set of field records; Extract the object reference field group from the field record set and perform dictionary mapping to obtain the object reference encoding set; Based on the object reference encoding set, the operation verb field of the corresponding record is introduced according to the record index and then concatenated and encoded to obtain the operation enhancement encoding set; The operation enhancement code set is introduced into the subject group field of the corresponding record according to the record index and multi-value encoding is performed to obtain the subject enhancement code set; The main enhanced coding set is introduced into the associated identification information of the corresponding record according to the record index and the identification mapping is performed to obtain the associated enhanced coding set; The association enhancement encoding set is dimensionally aligned and concatenated in a preset order to obtain a fixed-length feature vector set; The fixed-length feature vector set is numerically normalized to obtain a single log feature vector; The feature vectors of a single log entry are encapsulated in chronological order to obtain a sequence of feature vectors.
5. The context reconstruction method for audit logs according to claim 1, characterized in that, The step of inputting the feature vector sequence into a time-series learning model for inference and outputting a context label sequence includes: The feature vector sequence is length-aligned according to time order to obtain the model input tensor; The model input tensor is subjected to an embedding transformation to obtain an embedding sequence; The embedded sequences are input into the forward and backward long short-term memory units for processing, respectively, to obtain the forward representation sequence and the backward representation sequence. The forward and backward representation sequences are concatenated in the feature dimension and input into a fully connected mapping to obtain a temporal hidden state sequence; The temporal hidden state sequence is classified and mapped to obtain a label probability distribution sequence; Based on preset encoding rules, the category index of the label probability distribution sequence is mapped to a quintuple label to obtain the context label sequence.
6. The context reconstruction method for audit logs according to claim 1, characterized in that, The clustering process of the context label sequence to obtain the operation instance set includes: The context label sequences are index-aligned and merged to obtain labeled log sequences; The labeled log sequences are grouped and aggregated according to context labels to obtain a temporary group set; Extract the association identifiers of log records from the temporary group set, and combine them with the name identifiers to form an association key, thus obtaining an association key table; Perform same-key aggregation and transitive association merging on the association key tables respectively to obtain an association cluster set; The log records within each cluster in the associated cluster set are sorted according to the event time order to obtain the instance time sequence; Instance identifiers are generated using the instance time sequence, and a bidirectional index between instances and logs is established to output a set of operation instances.
7. The method for context reconstruction of audit logs according to claim 1, characterized in that, The process of reconstructing the context of the set of operation instances, concatenating logs in chronological order, and outputting the context reconstruction result includes: The set of operation instances is subjected to record aggregation processing to obtain a set of instance records; The log records of each instance in the instance record set are sorted according to the event timestamp to obtain a time-series sorted set; The temporal sequence of adjacent logs is concatenated to obtain the instance temporal sequence concatenation sequence. Based on the instance time-series concatenation sequence, a cross-log reference relationship is established, and an instance-level index is generated to obtain the instance context index; The instance context index is structurally encapsulated to generate a set of structured context records; The structured context record set is formatted to convert it into a preset structured array format, thus obtaining the context reconstruction result.
8. A context reconstruction apparatus for audit logs, characterized in that, include: The data caching module is used to acquire and cache the audit log data stream to obtain the original log sequence; The data processing module is used to preprocess the original log sequence to obtain a structured log sequence; The feature extraction module is used to extract features from the structured log sequence to obtain a feature vector sequence; The data inference module is used to input the feature vector sequence into the time-series learning model for inference and output the context label sequence; The clustering processing module is used to perform clustering processing on the context label sequence to obtain a set of operation instances; The reconstruction output module is used to reconstruct the context of the set of operation instances, concatenate the logs in chronological order, and output the context reconstruction result.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the context reconstruction method for audit logs as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the context reconstruction method for audit logs as described in any one of claims 1 to 7.