Log anomaly detection method, device, storage medium and computer program product

Abstract

The application provides a log anomaly detection method, device, storage medium and computer program product, the method comprises: based on the model structure of long short-term memory network LSTM, parameterized quantum circuit is used to construct quantum model;Quantum encoding is carried out on the first log data, and the first quantum state data is obtained;The first quantum state data is processed based on the quantum model, and the first result representing the log anomaly state is obtained.Such, the embodiment of the application is based on quantum-classical neural network fusion strategy, the low parameter quantity, high expression capacity characteristics of parameterized quantum circuit are used, the quantum model capable of processing data with time sequence dependence characteristics is constructed, and the log anomaly detection is carried out based on the quantum model, the model parameter scale is reduced, and the log anomaly detection efficiency is improved.

Description

Technical Field

[0001] This application relates to data processing technology, and more particularly to a log anomaly detection method, device, storage medium, and computer program product. Background Technology

[0002] In the field of cybersecurity, log anomaly detection provides crucial monitoring and analysis capabilities, serving as a core technology for ensuring the secure and stable operation of information systems such as industrial systems and IoT devices. However, with increasing system complexity, massive growth in log data, and its strong temporal characteristics, higher demands are placed on the computational efficiency, resource consumption, and recognition accuracy of detection models.

[0003] Quantum machine learning algorithms, with their characteristics of parallel processing of quantum states and low-dimensional feature mapping, can provide stronger processing capabilities than classical machine learning algorithms in the field of large-scale data processing. However, most commonly used quantum machine learning algorithms adopt a feedforward structure, which makes it difficult to capture the temporal characteristics of log data and the long-term dependencies between data when processing log data. Therefore, commonly used quantum machine learning algorithms are poorly adapted to log anomaly detection scenarios and it is difficult to achieve efficient and stable anomaly log identification. Summary of the Invention

[0004] This application provides a log anomaly detection method, device, storage medium, and computer program product, which aim to improve processing efficiency and reduce model complexity for log anomaly detection scenarios.

[0005] The technical solution of this application embodiment is implemented as follows: In a first aspect, embodiments of this application provide a log anomaly detection method, the method comprising: Based on the model structure of Long Short-Term Memory (LSTM) network, a quantum model is constructed using Parameterized Quantum Circuit (PQC). The first log data is quantum-encoded to obtain the first quantum state data; The first quantum state data is processed based on the quantum model to obtain the first result characterizing the abnormal state of the log.

[0006] In some implementations, the LSTM-based model structure employs PQC to construct the quantum model, including: Based on quantum rotation gates and quantum entanglement gates, quantum gated units corresponding to each gated unit of LSTM are constructed respectively; The quantum model is constructed by connecting the quantum gated units based on the LSTM-based gated topology.

[0007] In some implementations, processing the first quantum state data based on the quantum model to obtain a first result characterizing the log's abnormal state includes: Extract second quantum state data representing a predetermined amount of historical log data from the first quantum state data; The second quantum state data is processed based on the quantum model to obtain the second result representing the hidden state output by the model. Based on the second result, the corresponding short-term anomaly score and long-term anomaly score are obtained; The first result is determined based on the short-term anomaly score and the long-term anomaly score.

[0008] In some implementations, the method further includes: Obtain the cumulative detection duration and the number of abnormal logs in the historical log data; Based on the cumulative detection duration, the first factor is determined; The second factor is determined based on the number of abnormal logs. The scoring weights are determined based on the first factor and the second factor; Determining the first result based on the short-term anomaly score and the long-term anomaly score includes: Based on the scoring weights, the short-term abnormal scores and the long-term abnormal scores are weighted to determine the first result.

[0009] In some implementations, based on the second result, a corresponding short-term anomaly score is obtained, including: The second result is subjected to expectation measurement and linear transformation to obtain second log data characterizing the log prediction result; Extract the third log data corresponding to the detection time from the first log data; Calculate the similarity between the second log data and the third log data, and determine the corresponding short-term anomaly score based on the first similarity result.

[0010] In some implementations, based on the second result, a corresponding long-term anomaly score is obtained, including: Obtain the historical hidden state set output by the quantum model; Cluster the set of historical hidden states and determine the central hidden state of each class; Calculate the similarity between the second result and each of the central hidden states, and determine the second similarity result with the highest similarity. Based on the second similarity result, the corresponding long-term anomaly score is determined.

[0011] In some implementations, the step of quantum encoding the first log data to obtain the first quantum state data includes: The first log data is preprocessed to obtain a log sequence carrying context information; Based on each feature vector in the log sequence, the corresponding qubit is encoded by x-axis, y-axis and z-axis angles to obtain the x-axis quantum state, y-axis quantum state and z-axis quantum state corresponding to each feature vector; The quantum states along the x-axis, y-axis, and z-axis corresponding to the feature vector are weighted and normalized to obtain the quantum state data corresponding to the feature vector. The first quantum state data is obtained based on the quantum state data corresponding to each of the aforementioned feature vectors.

[0012] In a second aspect, embodiments of this application provide an electronic device, the electronic device comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method described in the first aspect.

[0013] Thirdly, embodiments of this application provide a storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0014] Fourthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0015] In the technical solution of this application, a quantum model is constructed using a parameterized quantum circuit based on an LSTM model structure; the first log data is quantum-encoded to obtain the first quantum state data; the first quantum state data is processed based on the quantum model to obtain a first result characterizing the log anomaly state. Thus, this embodiment of the application, based on a quantum-classical neural network fusion strategy, utilizes the low parameter count and high expressive power characteristics of parameterized quantum circuits to construct a quantum model capable of processing data with time-dependent characteristics, and performs log anomaly detection based on this quantum model, reducing the model parameter scale while improving the efficiency of log anomaly detection. Attached Figure Description

[0016] Figure 1 This is a first flowchart illustrating the log anomaly detection method according to an embodiment of this application; Figure 2 This is a schematic diagram of the second process of the log anomaly detection method according to an embodiment of this application; Figure 3 This is a schematic diagram of the log anomaly detection device according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0017] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the field of cybersecurity, log anomaly detection is a core technical means to ensure the safe and stable operation of information systems such as industrial systems and IoT devices. By monitoring and analyzing the log data generated by the system operation, abnormal behaviors and abnormal states can be identified, thereby providing early warning and control of system security risks.

[0020] In related technologies, classic machine learning or deep learning algorithms are typically used to build log anomaly detection models. However, the accurate identification of abnormal logs by such models is highly dependent on a large amount of high-quality log data. Moreover, as system complexity increases, the scale of log data grows exponentially, which can easily lead to a surge in the number of model parameters, resulting in significant limitations in terms of processing efficiency and computational resource consumption.

[0021] With the rapid development of quantum computing, quantum machine learning algorithms, leveraging the characteristics of parallel quantum state processing and low-dimensional feature mapping, offer greater computational power than classical machine learning algorithms in large-scale data processing and optimization. However, commonly used quantum machine learning algorithms often employ feedforward structures, while log anomaly detection heavily relies on the analysis of log context and evolutionary relationships. When processing log data, it is difficult to capture the temporal characteristics of the log data and the long-term dependencies between data points. Therefore, commonly used quantum machine learning algorithms are poorly adapted to log anomaly detection scenarios, making it difficult to achieve efficient and stable anomaly log identification.

[0022] To address the aforementioned issues, this application provides a log anomaly detection method, aiming to improve processing efficiency and reduce model complexity for log anomaly detection scenarios.

[0023] For example, the log anomaly detection method provided in the embodiments of this application is as follows: Figure 1 As shown, the method includes: Step 101: Based on the model structure of long short-term memory networks, construct a quantum model using parameterized quantum circuits.

[0024] Step 102: Perform quantum encoding on the first log data to obtain the first quantum state data.

[0025] Step 103: Process the first quantum state data based on the quantum model to obtain the first result characterizing the abnormal state of the log.

[0026] Here, Long Short-Term Memory (LSTM) is a classic recurrent neural network suitable for time-series data processing, which can effectively capture long-term dependencies in sequence data. LSTM consists of multiple gating units, including input gate, forget gate and output gate. Each gating unit works together to maintain and update the cell state and hidden state during the LSTM operation.

[0027] The cell state is the core memory unit of LSTM, used to store historical feature information of sequence data for a long time and maintain the continuity of temporal correlation; the hidden state is used to output the feature representation of the current time step, pass it to the next time step and participate in the calculation.

[0028] The input gate controls the input level of feature information at the current moment, filters out valid new information and updates the cell state, and suppresses interference from redundant information. The forget gate controls the retention level of historical information, adaptively discards outdated invalid information in the cell state based on sequence association characteristics, and ensures the effectiveness of the memory unit. The output gate controls the output level of the cell state, filters out valid information based on the updated quantum cell state, generates and outputs the hidden state at the current moment, and passes it to the subsequent processing stage.

[0029] Specifically, LSTM is based on a serial topology where the forget gate, input gate, and output gate act sequentially on the cell state. The forget gate and input gate act on the cell state in parallel, and the output gate generates the current hidden state based on the updated cell state. When LSTM processes a task, the forget gate first selectively forgets the historical cell states, while the input gate simultaneously filters the current input features and generates candidate states. Together, they complete the update of the cell state. The output gate then generates and outputs the hidden state at the current time step based on the updated cell state.

[0030] Understandably, based on the functions of each gating unit and the gating topology mentioned above, when LSTM processes log anomaly detection tasks, it can effectively capture long-term temporal correlations and behavioral patterns in log sequences, retain key historical log features and filter redundant fluctuation information, thereby improving the accuracy and robustness of anomaly log identification.

[0031] Here, parameterized quantum circuits (PQCs) are quantum circuits containing trainable parameters, including quantum rotation gates and quantum entanglement gates. Controllable transformations and feature mappings of quantum states are achieved by adjusting trainable parameters such as the rotation angle of the quantum rotation gate, exhibiting inherent characteristics of low parameter count and high expressive power. Specifically, quantum rotation gates encode and transform feature information by tunably rotating the quantum state to change its amplitude or phase. Based on the difference in rotation axis, they are typically divided into x-axis quantum rotation gates, y-axis quantum rotation gates, and z-axis quantum rotation gates. Quantum rotation gates with different rotation axes can be tuned separately. The characteristics of quantum states in corresponding dimensions are described. Quantum entanglement gates are used to establish quantum correlations between different qubits, including controlled NOT gates (CNOT), controlled phase gates (CZ), controlled rotation gates (CRx / CRy / CRz), and swapping gates (iSWAP). Based on different types of quantum entanglement gates, amplitude inversion, phase modulation, and state swapping of quantum states can be realized, thereby enhancing the ability of quantum circuits to capture and express complex features. For example, the CNOT gate is a two-qubit operation, where two qubits include a control bit and a target bit. When the control bit is... When the target bit remains unchanged, the control bit remains unchanged. In the state of quantum NOT, a quantum NOT gate operation is performed on the target.

[0032] It is understandable that by training and optimizing the trainable parameters of PQC, high-dimensional features can be efficiently compressed and represented by the superposition and entanglement states of qubits, and complex data features can be extracted and modeled with fewer training parameters.

[0033] Here, the quantum model in this application embodiment is an LSTM quantum circuit, specifically a quantum circuit constructed based on the data processing logic of a long short-term memory network. It can perform temporal feature filtering, memory retention and information update processing on the input quantum state data, and update the corresponding quantum cell state and the output quantum hidden state.

[0034] For example, based on the LSTM model structure, a quantum model is constructed by: constructing quantum gate units corresponding to each gate unit of the LSTM based on quantum rotation gates and quantum entanglement gates; and connecting each quantum gate unit based on the LSTM gate topology to construct the quantum model.

[0035] Here, quantum gated units corresponding to each gated unit of the LSTM are constructed based on the x-axis quantum rotation gate, y-axis quantum rotation gate, z-axis quantum rotation gate, and CNOT gate. Among them, the constructed quantum gated units include the quantum forget gate corresponding to the LSTM forget gate, the quantum input gate corresponding to the LSTM input gate, and the quantum output gate corresponding to the LSTM output gate.

[0036] Here, each quantum gating unit consists of multiple alternating quantum rotation gates and quantum entanglement gates.

[0037] In some embodiments, the quantum gating unit is shown in the following equation:

[0038] in, These are trainable parameters, including the rotation angle of the x-axis quantum rotating gate. The rotation angle of the y-axis quantum rotating door and the rotation angle of the z-axis quantum rotating gate ; For x-axis quantum rotation gate, For y-axis quantum rotation gate, For z-axis quantum rotation gate, For qubits.

[0039] It should be noted that after the quantum state data is input into the quantum model, the quantum input gate modulates the amplitude and phase of the current quantum state data to complete the screening and encoding; the quantum forget gate, based on the same qubit system, adaptively modulates the historical features in the quantum cell state; the two work together to update the quantum cell state; the quantum output gate then adjusts the output according to the updated quantum cell state to obtain and output the quantum hidden state at the current moment, thus completing the quantum state data processing as a whole.

[0040] Understandably, compared to the corresponding LSTM model, the quantum model constructed in this application has a significantly reduced number of parameters, and by relying on the superposition and entanglement characteristics of qubits, the computational overhead in the time-series feature processing is greatly reduced.

[0041] It is understandable that, since PQC parameters are adjustable and can perform complex processing such as amplitude and phase control and inter-bit correlation constraints on quantum state data, the embodiments of this application can simulate the same gating logic as the LSTM gating unit by combining multiple quantum rotation gates and quantum entanglement gates and adjusting the internal trainable parameters. This can replace the gating unit structures such as input gates, forget gates and output gates in LSTM, and complete the dynamic control of cell state and hidden state based on quantum state evolution. While ensuring the capture of data timing and dependencies, it effectively reduces the overall number of model parameters and computational overhead.

[0042] Here, the first log data is the raw log data generated by the system, that is, the log data that needs to be detected for anomalies.

[0043] It is understood that, in this application embodiment, the first log data is quantum encoded, thereby mapping the first log data into a quantum state that can be directly processed by the quantum model.

[0044] It is understandable that after the first quantum state data representing the log data is input into the quantum model, the log historical features with temporal dependencies are extracted, the corresponding quantum cell states are maintained, and the quantum hidden state reflecting the current log behavior is output according to the log evolution law, thereby obtaining the first result representing the abnormal state of the log.

[0045] It is understood that the embodiments of this application are based on a quantum-classical neural network fusion strategy, which utilizes the low parameter quantity and high expressive power characteristics of parameterized quantum circuits to construct a quantum model that can process data with time-dependent characteristics, and performs log anomaly detection based on this quantum model, thereby reducing the model parameter size and improving the efficiency of log anomaly detection.

[0046] For example, quantum encoding of the first log data to obtain the first quantum state data includes: preprocessing the first log data to obtain a log sequence carrying context information; encoding the corresponding qubits according to the x-axis, y-axis, and z-axis angles based on each feature vector in the log sequence to obtain the x-axis quantum state, y-axis quantum state, and z-axis quantum state corresponding to each feature vector; performing weighted normalization processing on the x-axis quantum state, y-axis quantum state, and z-axis quantum state corresponding to the feature vector to obtain the quantum state data corresponding to the feature vector; and obtaining the first quantum state data based on the quantum state data corresponding to each feature vector.

[0047] In some embodiments, preprocessing the first log data to obtain a log sequence carrying context information includes: parsing the first log data to determine the corresponding constant data and variable data; replacing variable data based on wildcards to generate a log event template; sequentially performing log event template mapping, data cleaning, classification, and time-series sorting on the first log data to obtain sorted event logs; performing feature encoding on each event log to obtain a corresponding event vector sequence; performing context association processing on the event vector sequence to obtain a high-dimensional context sequence; and performing dimensionality reduction processing on the high-dimensional context sequence to obtain a log sequence carrying context information.

[0048] Here, the first log data consists of fixed text messages and dynamic parameters. The constant data includes fixed text messages, and the variable data includes dynamic parameters. When parsing and constructing the log event template, the dynamic parameters need to be replaced with wildcards such as "*" to achieve normalization of different dynamic parameters and eliminate the interference of dynamic parameter differences on subsequent feature encoding and time series analysis.

[0049] In some embodiments, the first log data is parsed using a log parsing tool to generate a log event template.

[0050] In some embodiments, the log parsing tool for parsing the first log data is Drain.

[0051] In some embodiments, in order to achieve unique identification and differentiation of events, a corresponding event identifier is assigned to each log event template.

[0052] Here, after mapping the first log data to log event templates, multiple initial log events with fixed structures are obtained.

[0053] Here, after obtaining the initial log events corresponding to the first log data, data cleaning is performed on each initial log event. The data cleaning includes at least one of the following: segmenting compound words, removing interfering symbols, and processing missing and redundant data.

[0054] In some embodiments, log events carry device information, and the cleaned log events are categorized based on the device information.

[0055] Here, log events carry timestamp information representing the time when the event occurred; based on the timestamp information carried by the log events, the embodiments of this application sort log events of the same category in chronological order to accurately express the temporal correlation between log events.

[0056] Here, log events are feature-encoded, converting the text-based log events into vector-represented feature vectors to meet the model's subsequent feature extraction and processing needs. For example, a log event e consisting of n features, after feature encoding, is represented as a sequence of event vectors. .

[0057] In some embodiments, the event logs are feature-encoded based on the Bidirectional Encoder Representation Model (BERT) to obtain the corresponding event vector sequence.

[0058] It should be noted that the event vector sequence maintains the same temporal order as the aforementioned log events.

[0059] It should be noted that context association processing is performed on the event vector sequence to mine the temporal dependency and context association features between adjacent events based on log attributes; wherein, in order to constrain the context scope and avoid computational redundancy caused by global traversal, the embodiments of this application use a sliding window to extract the context event corresponding to each event vector from the sorted event vector sequence.

[0060] Here, the sliding window is used to limit the maximum number of context events and / or the longest time interval.

[0061] In some embodiments, the sliding window uses the L event vectors preceding the current event vector as corresponding context events to form a context set; for the event vector sequence e i After performing context association processing, a high-dimensional context feature sequence is obtained. .

[0062] It should be noted that, due to the limited number of qubits available in a quantum computer, the dimensionality of its encodeable features is typically lower than the dimensionality of the high-dimensional context feature sequence. Therefore, after obtaining the high-dimensional context feature sequence, it is necessary to perform dimensionality reduction through linear transformation to compress the feature dimension to a range that a quantum computer can process. The log sequence obtained after dimensionality reduction is shown in the following equation:

[0063] Where W is the weight matrix and b is the bias vector, both of which are trainable parameters.

[0064] Here, the feature vectors in the log sequence correspond one-to-one with the qubits of the quantum computer.

[0065] It should be noted that in related technologies, when mapping feature vectors to quantum states, a single encoding method is typically used, such as angular encoding of the feature vectors based on x-axis rotation to obtain the corresponding quantum state data. The applicant's research has found that this single encoding method, utilizing only a single rotation dimension to express feature information, easily leads to insufficient feature representation capability and low feature discrimination of the quantum state data. Therefore, in this embodiment, an xyz-axis cyclic encoding strategy is adopted to perform multi-dimensional angular encoding of the feature vectors to improve the feature representation capability and feature discrimination of the quantum state data.

[0066] Encoding the qubit along the x-axis angle includes rotating the state of the qubit through an x-axis encoding rotation gate.

[0067] Here, the parameters (i.e., rotation angle) of the x-axis encoded revolving door are determined based on the feature vector.

[0068] In some embodiments, the x-axis encoded rotary gate is as follows:

[0069] in, Let X be the feature vector, and let X be the Pauli-X gate.

[0070] It is understandable that after the state of a qubit is rotated through an x-axis encoding rotation gate, it is mapped to generate an x-axis quantum state corresponding to the feature vector. The x-axis quantum state is specifically:

[0071] Encoding the y-axis angle of the qubit includes rotating the state of the qubit through a y-axis encoding rotation gate.

[0072] Here, the parameters (i.e., rotation angle) of the y-axis encoded revolving door are determined based on the feature vector.

[0073] In some embodiments, the y-axis encoded rotary gate is as follows:

[0074] in, Let Y be the feature vector, and Y be the Pauli-Y gate.

[0075] It is understandable that after the state of a qubit is rotated through a y-axis encoding rotation gate, it is mapped to generate a y-axis quantum state corresponding to the feature vector. The y-axis quantum state is specifically as follows:

[0076] Encoding the qubits along the z-axis angle includes rotating the state of the qubits using a z-axis encoding rotation gate.

[0077] Here, the parameters (i.e., rotation angle) of the z-axis encoded rotating door are determined based on the feature vector.

[0078] In some embodiments, the z-axis encoded rotary gate is as follows:

[0079] in, Let Z be the feature vector, and Z be the Pauli-Z gate.

[0080] It is understandable that after the state of a qubit is rotated through a z-axis encoding rotation gate, it is mapped to generate a z-axis quantum state corresponding to the feature vector. The z-axis quantum state is specifically:

[0081] For example, weighted normalization is performed on the x-axis quantum state, y-axis quantum state, and z-axis quantum state corresponding to the feature vector to obtain the quantum state data corresponding to the feature vector, including: weighted normalization is performed on the x-axis quantum state, y-axis quantum state, and z-axis quantum state corresponding to the feature vector based on the trained axis weights to obtain the quantum state data corresponding to the feature vector.

[0082] Here, the x-axis quantum state corresponds to the x-axis weight α. x The y-axis quantum state corresponds to the y-axis weight α. y The z-axis quantum state corresponds to the z-axis weight α. z The axis weights represent the contribution of the corresponding quantum state to the normalized quantum state data, and are used as learnable parameters.

[0083] In some embodiments, the weighted normalization of the quantum states of each axis is performed as shown in the following equation:

[0084] in, For quantum state data, For normalization functions; in some embodiments, This is the Sigmoid function.

[0085] It is understood that the embodiments of this application, based on multi-axis cyclic rotation encoding and dynamic axis weight adjustment, can improve the feature representation capability of quantum state data and adaptively adjust the feature contribution of each rotation axis.

[0086] In some embodiments, the method further includes: training the quantum model based on sample data to obtain a trained quantum model.

[0087] The sample data consists of quantum state data characterizing log events and is labeled with corresponding log anomaly states.

[0088] It is understood that after constructing the quantum model, the embodiments of this application train the quantum model based on sample data; during the training process, the trainable parameters of the quantum model are continuously optimized and adjusted so that the trained quantum model can accurately identify abnormal logs in the log data.

[0089] Accordingly, the first quantum state data is processed based on the quantum model, including: processing the first quantum state data based on the trained quantum model.

[0090] Understandably, the training efficiency is greatly improved because the number of trainable parameters for quantum models is significantly smaller than that of classic log anomaly detection models such as LSTM.

[0091] This application does not limit the specific training method of the quantum model; in some embodiments, the trainable parameters of the quantum model are adjusted and updated based on a classical optimizer and parameter shifting rules. The classical optimizer can be the Adam optimizer.

[0092] For example, processing the first quantum state data based on a quantum model to obtain a first result representing the log anomaly state includes: extracting second quantum state data representing a set number of historical log data from the first quantum state data; processing the second quantum state data based on the quantum model to obtain a second result representing the hidden state output by the model; obtaining corresponding short-term anomaly scores and long-term anomaly scores based on the second result; and determining the first result based on the short-term anomaly scores and long-term anomaly scores.

[0093] Here, the quantum state data in the first quantum state data are ordered based on the temporal order.

[0094] Here, the quantity is set based on the sliding window used to extract historical log data.

[0095] Here, historical log data is determined based on the detection time.

[0096] It is understandable that if the length of the sliding window is L, when checking whether the log event at the current moment is abnormal, the extracted second quantum state data is the L quantum state data before the quantum state data corresponding to the current moment.

[0097] Understandably, log anomalies have significant temporal and contextual dependencies. Whether a current log event is abnormal usually depends on the temporal variation pattern of consecutive log events over a certain historical period. Therefore, a set amount of second quantum state data needs to be extracted as a basis for detection during the detection process.

[0098] It is understandable that after the first quantum state data is input into the quantum model, the quantum output gate of the quantum model outputs the second result, which is the quantum hidden state corresponding to the current detection time, representing the quantum state prediction result obtained based on the temporal change pattern of continuous log events.

[0099] It should be noted that related technologies typically perform log anomaly detection based on only a single time scale, that is, they only focus on the regular changes in short-term log data and lack consideration for the overall trend of log event evolution, which can easily lead to one-sided anomaly judgment and a high false alarm rate. In order to improve the accuracy of log anomaly detection, this application provides a joint detection mechanism that combines short-term anomalies with long-term anomalies. During detection, it not only considers the abrupt changes in recent logs, but also introduces the stable patterns of long-term logs as a reference, thereby achieving comprehensive judgment across multiple time scales.

[0100] In some embodiments, obtaining a corresponding short-term anomaly score based on the second result includes: performing expectation measurement and linear transformation on the second result to obtain second log data representing the log prediction result; extracting third log data corresponding to the detection time from the first log data; calculating the similarity between the second log data and the third log data, and determining the corresponding short-term anomaly score based on the first similarity result.

[0101] In some embodiments, performing a quantum measurement on the second result includes: performing a desired measurement on the second result based on a Pauli operator to obtain a third result.

[0102] The third result is a classical numerical representation of the hidden state.

[0103] Here, the Pauli operator is a fundamental operator for quantum measurement, used to convert quantum states into computable classical expectation values.

[0104] Understandably, quantum states cannot be directly used as input to classical models. They need to be measured using Pauli operators to map quantum state information into classical probabilities or numerical forms for subsequent anomaly scoring calculations.

[0105] Understandably, after obtaining the third result, a linear transformation is performed on the third result to obtain the second log data that represents the log prediction result.

[0106] In some embodiments, linear transformation processing of the third result includes: linear transformation processing of the third result based on a fully connected layer.

[0107] Here, the fully connected layer is a classic neural network structure used to achieve linear mapping and dimension adaptation of features.

[0108] It is understandable that the second log data is the log result predicted based on the changing patterns of the extracted short-term historical log data; by calculating the similarity between the second log data and the real log data corresponding to the current detection time (i.e., the third log data), a short-term anomaly score reflecting whether the log data at the current time is abnormal can be obtained.

[0109] It should be noted that the embodiments of this application do not specifically limit the calculation method of the first similarity result; in some embodiments, the similarity between the second log data and the third log data is calculated based on Euclidean distance to obtain the first similarity result.

[0110] In one application example, the second log data characterizing the log prediction result is shown in the following formula:

[0111] in, This is the set of trainable parameters for the quantum model.

[0112] In this example, the short-term anomaly score is determined based on Euclidean distance quantization; the larger the calculated distance result, the higher the probability of an anomaly in the log. The first similarity result represents the distance result. As shown in the following formula:

[0113] In some embodiments, obtaining a corresponding long-term anomaly score based on the second result includes: acquiring a set of historical hidden states output by the quantum model; clustering the set of historical hidden states and determining the central hidden states of each cluster; calculating the similarity between the second result and each central hidden state and determining the second similarity result with the highest similarity; and determining the corresponding long-term anomaly score based on the second similarity result.

[0114] The historical hidden state set is the historical set formed by the second result output by the model after the first quantum state data is input into the quantum model.

[0115] It is understood that, during the log anomaly detection process, the quantum model in this application stores all the output quantum hidden states to form a historical hidden state set, which is used to characterize the long-term pattern features of the log data.

[0116] In some embodiments, clustering the set of historical hidden states includes: performing quantum K-means clustering on the set of historical hidden states.

[0117] Here, quantum k-means clustering is a clustering method suitable for quantum computing. By utilizing the inner product and superposition properties of quantum states to calculate the similarity between states, it is possible to efficiently classify quantum state data and determine the cluster centers of each class.

[0118] The quantum state inner product is obtained by calculating the amplitude information of the quantum state and is used to measure the similarity between two quantum states. The larger the inner product value, the closer the quantum states are.

[0119] The cluster center is the mean state of all quantum states in each cluster, representing the overall distribution characteristics of quantum state data within that cluster. Iterative updates minimize the state differences between the quantum states within each cluster and the central quantum state.

[0120] In some embodiments, the location of cluster centers is optimized by using a variable quantum eigenvalue solver (VQE) to minimize the distance between intra-cluster quantum states and cluster centers, thereby achieving optimal updating of cluster centers.

[0121] Here, the central hidden state is the cluster center after the historical hidden state set is classified.

[0122] Understandably, by performing quantum K-means clustering on the historical hidden state set, quantum state data with similar characteristics can be grouped into one category, and the core features of each category can be extracted, thereby quickly capturing the overall distribution pattern of quantum state data.

[0123] Accordingly, the similarity between the second result and each central hidden state is calculated, including: calculating the quantum state inner product between the second result and each central hidden state.

[0124] Understandably, the second similarity result represents the highest similarity between the current hidden state and the centers of various clusters, indicating the degree of matching between the current hidden state and historical long-term features. The smaller the second similarity result, the higher the probability of log anomalies.

[0125] In one application example, all quantum hidden states output by the quantum model. Quantum K-means clustering is performed, and the quantum hidden state at detection time t is calculated using a swap test. With the center hidden state The distance, that distance The similarity is expressed as follows:

[0126] in, It is the square of the quantum state inner product.

[0127] Correspondingly, the second similarity result As shown in the following formula:

[0128] Where K is the preset number of clusters.

[0129] For example, determining a first result based on short-term anomaly scores and long-term anomaly scores includes: weighting the short-term anomaly scores and long-term anomaly scores based on their respective weights to determine the first result.

[0130] It is understood that the embodiments of this application combine short-term real-time differences with long-term pattern characteristics for weighted fusion, which can simultaneously capture instantaneous anomalies in logs and global anomalies that deviate from historical patterns, thereby improving the comprehensiveness and accuracy of anomaly detection.

[0131] In some embodiments, the method further includes: obtaining the cumulative detection duration and the number of abnormal logs in historical log data; determining a first factor based on the cumulative detection duration; determining a second factor based on the number of abnormal logs; and determining a scoring weight based on the first factor and the second factor.

[0132] It should be noted that, in order to improve the accuracy of log anomaly detection, the embodiments of this application dynamically adjust the scoring weights corresponding to short-term anomaly scores and long-term anomaly scores during the detection process.

[0133] Here, the cumulative detection duration is the cumulative time for the quantum model to process the first quantum state data; correspondingly, the first factor determined based on the cumulative detection duration represents the time decay term, and the first factor gradually decreases as the detection duration increases.

[0134] Here, the second factor represents the anomaly density term, specifically the proportion of abnormal logs in a set amount of historical log data; correspondingly, the more abnormal logs there are in a short period of time, the larger the second factor becomes.

[0135] Specifically, the scoring weights of short-term abnormality scores are negatively correlated with the first factor and positively correlated with the second factor; the scoring weights of long-term abnormality scores are positively correlated with the first factor and negatively correlated with the second factor.

[0136] It is understandable that by simultaneously introducing time decay and anomaly density terms to dynamically adjust the weights, the influence of short-term anomaly scores can be enhanced during periods of frequent anomalies, highlighting real-time mutation characteristics; while during periods of stable operation and fewer anomalies, the influence of long-term anomaly scores can be enhanced, strengthening historical pattern constraints. This enables the anomaly detection strategy to adaptively adjust with the actual operating state of the system, effectively balancing detection sensitivity and anti-interference capability, and improving detection reliability in complex scenarios.

[0137] In some embodiments, the scoring weight of short-term abnormal scores is within a preset threshold range.

[0138] Understandably, in order to avoid extreme values ​​of weights that could affect the stability of the detection results, upper and lower limits are imposed on the scoring weights of short-term anomaly scores by setting a preset threshold range, making the anomaly detection process more stable and reliable.

[0139] In one application example, the scoring weights for short-term anomaly ratings. As shown in the following formula:

[0140] in, These are the base weights used to determine the proportion of short-term anomaly scores at the initial time. This is the time decay coefficient, which determines the rate at which the weight decreases over time. This represents the time decay term, which is close to the initial detection period. As the sliding window moves and the cumulative detection time increases, This gradually reduces the weight of short-term abnormal scores, thereby increasing the weight of long-term abnormal scores. is the anomaly density coefficient, used to adjust the increase in the scoring weight of short-term anomalies on short-term anomaly scores; L is the sliding window length, i.e., the set number. This represents the number of exception logs in the sliding window. This indicates the abnormal density item. If there are many abnormalities in the short term, the short-term abnormality score is weighted to enable timely response.

[0141] Among them, the scoring weight of short-term abnormality scores Falling Inside.

[0142] In this example, the first result The specific formula is as follows:

[0143] in, For short-term abnormality scoring, For long-term abnormality scoring.

[0144] For example, the method further includes: if the first result is greater than a set threshold, then determining that the log event corresponding to the first result is an abnormal log.

[0145] In some embodiments, the threshold is set to 0.95.

[0146] Understandably, by processing the first quantum state data based on the trained quantum model, we can obtain the first result corresponding to each log event arranged in chronological order. By comparing the first result with a set threshold, abnormal log events can be identified quickly and accurately.

[0147] The log anomaly detection method flow of this application embodiment is as follows: Figure 2 As shown, performing log anomaly detection based on the above method can reduce training and computational overhead while ensuring the accuracy of log anomaly detection and improving processing efficiency.

[0148] In order to implement the method of the embodiments of this application, the embodiments of this application also provide a log anomaly detection device, which corresponds to the above-described log anomaly detection method, and the steps in the above-described log anomaly detection method embodiments are also fully applicable to the embodiments of this device.

[0149] like Figure 3 As shown, the log anomaly detection device provided in this application embodiment includes a construction module 301, an encoding module 302, and a detection module 303. The construction module 301 is used to construct a quantum model based on an LSTM model structure using parameterized quantum circuits; the encoding module 302 is used to perform quantum encoding on the first log data to obtain first quantum state data; the detection module 303 is used to process the first quantum state data based on the quantum model to obtain a first result characterizing the log anomaly state.

[0150] In some embodiments, the construction module 301 is specifically used to: construct quantum gated units corresponding to each gated unit of the LSTM based on quantum rotation gates and quantum entanglement gates; and connect each quantum gated unit based on the LSTM gated topology to construct a quantum model.

[0151] In some embodiments, the detection module 303 is specifically used to: extract second quantum state data representing a set number of historical log data from the first quantum state data; process the second quantum state data based on a quantum model to obtain a second result representing the hidden state output by the model; obtain the corresponding short-term anomaly score and long-term anomaly score based on the second result; and determine the first result based on the short-term anomaly score and long-term anomaly score.

[0152] In some embodiments, the detection module 303 is further configured to: obtain the cumulative detection duration and the number of abnormal logs in the historical log data; determine a first factor based on the cumulative detection duration; determine a second factor based on the number of abnormal logs; and determine a scoring weight based on the first factor and the second factor.

[0153] In some embodiments, the detection module 303 is specifically used to: perform weighted processing on short-term abnormal scores and long-term abnormal scores based on scoring weights to determine a first result.

[0154] In some embodiments, the detection module 303 is specifically used to: perform expected measurement and linear transformation processing on the second result to obtain second log data representing the log prediction result; extract the third log data corresponding to the detection time from the first log data; calculate the similarity between the second log data and the third log data, and determine the corresponding short-term anomaly score based on the first similarity result.

[0155] In some embodiments, the detection module 303 is specifically used to: acquire the historical hidden state set output by the quantum model; cluster the historical hidden state set and determine the central hidden state of each class; calculate the similarity between the second result and each central hidden state and determine the second similarity result with the highest similarity; and determine the corresponding long-term anomaly score based on the second similarity result.

[0156] In some embodiments, the encoding module 302 is specifically used for: preprocessing the first log data to obtain a log sequence carrying context information; encoding the corresponding qubits according to the x-axis, y-axis and z-axis angles based on each feature vector in the log sequence to obtain the x-axis quantum state, y-axis quantum state and z-axis quantum state corresponding to each feature vector; performing weighted normalization processing on the x-axis quantum state, y-axis quantum state and z-axis quantum state corresponding to the feature vector to obtain the quantum state data corresponding to the feature vector; and obtaining the first quantum state data based on the quantum state data corresponding to each feature vector.

[0157] It should be noted that the log anomaly detection device provided in the above embodiments is only illustrated by the division of the above program modules when performing log anomaly detection. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the log anomaly detection device and the log anomaly detection method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0158] Based on the hardware implementation of the above program modules, and in order to implement the log anomaly detection method of this application embodiment, this application embodiment also provides an electronic device, such as... Figure 4As shown, electronic device 400 includes at least one processor 401, memory 402, user interface 403, and at least one network interface 404. The various components in electronic device 400 are coupled together via bus system 405. It can be understood that bus system 405 is used to implement communication between these components. In addition to a data bus, bus system 405 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 4 The general designated all buses as Bus System 405.

[0159] The user interface 403 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0160] The memory 402 in this embodiment is used to store various types of data to support the operation of the electronic device 400. Examples of such data include any computer program used to operate on the electronic device 400.

[0161] The log anomaly detection method disclosed in this application can be applied to or implemented by the processor 401. The processor 401 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the log anomaly detection method can be completed by the integrated logic circuitry of the hardware in the processor 401 or by instructions in software form. The processor 401 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 401 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, which is located in the memory 402. The processor 401 reads the information in the memory 402 and, in conjunction with its hardware, completes the steps of the log anomaly detection method provided in the embodiments of this application.

[0162] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned log anomaly detection method.

[0163] It is understood that memory 402 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), EEPROM, ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM). The memory 402 described in this application embodiment is intended to include, but is not limited to, these and any other suitable types of memory.

[0164] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 402 storing a computer program. This computer program can be executed by the processor 401 of the electronic device 400 to complete the steps described in the log anomaly detection method of this application embodiment. The computer-readable storage medium can be a ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0165] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 401 of an electronic device 400 to perform the steps described in the method of this application embodiment.

[0166] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0167] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0168] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting log anomalies, characterized in that, The method includes: Based on the model structure of the Long Short-Term Memory (LSTM) network, a quantum model is constructed using parameterized quantum circuits. The first log data is quantum-encoded to obtain the first quantum state data; The first quantum state data is processed based on the quantum model to obtain the first result characterizing the abnormal state of the log.

2. The method according to claim 1, characterized in that, The LSTM-based model structure employs a quantum model construction method, including: Based on quantum rotation gates and quantum entanglement gates, quantum gated units corresponding to each gated unit of LSTM are constructed respectively; The quantum model is constructed by connecting the quantum gated units based on the LSTM-based gated topology.

3. The method according to claim 1, characterized in that, The process of processing the first quantum state data based on the quantum model to obtain a first result characterizing the abnormal state of the log includes: Extract second quantum state data representing a predetermined amount of historical log data from the first quantum state data; The second quantum state data is processed based on the quantum model to obtain the second result representing the hidden state output by the model. Based on the second result, the corresponding short-term anomaly score and long-term anomaly score are obtained; The first result is determined based on the short-term anomaly score and the long-term anomaly score.

4. The method according to claim 3, characterized in that, The method further includes: Obtain the cumulative detection duration and the number of abnormal logs in the historical log data; Based on the cumulative detection duration, the first factor is determined; The second factor is determined based on the number of abnormal logs. The scoring weights are determined based on the first factor and the second factor; Determining the first result based on the short-term anomaly score and the long-term anomaly score includes: Based on the scoring weights, the short-term abnormal scores and the long-term abnormal scores are weighted to determine the first result.

5. The method according to claim 3, characterized in that, Based on the second result, the corresponding short-term anomaly score is obtained, including: The second result is subjected to expectation measurement and linear transformation to obtain second log data characterizing the log prediction result; Extract the third log data corresponding to the detection time from the first log data; Calculate the similarity between the second log data and the third log data, and determine the corresponding short-term anomaly score based on the first similarity result.

6. The method according to claim 3, characterized in that, Based on the second result, the corresponding long-term anomaly score is obtained, including: Obtain the historical hidden state set output by the quantum model; Cluster the set of historical hidden states and determine the central hidden state of each class; Calculate the similarity between the second result and each of the central hidden states, and determine the second similarity result with the highest similarity. Based on the second similarity result, the corresponding long-term anomaly score is determined.

7. The method according to claim 1, characterized in that, The process of quantum encoding the first log data to obtain the first quantum state data includes: The first log data is preprocessed to obtain a log sequence carrying context information; Based on each feature vector in the log sequence, the corresponding qubit is encoded by x-axis, y-axis and z-axis angles to obtain the x-axis quantum state, y-axis quantum state and z-axis quantum state corresponding to each feature vector; The quantum states along the x-axis, y-axis, and z-axis corresponding to the feature vector are weighted and normalized to obtain the quantum state data corresponding to the feature vector. The first quantum state data is obtained based on the quantum state data corresponding to each of the aforementioned feature vectors.

8. An electronic device, characterized in that, The electronic device includes: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method according to any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images