Abnormality detection method and related apparatus, electronic device, and storage medium

Abstract

This application discloses an anomaly detection method, related apparatus, electronic device, and storage medium. The anomaly detection method includes: extracting access strings at various times based on real-time network logs of a terminal device in a network environment; performing a sliding window operation on the access strings at each time time to obtain the access sequence at the current time; performing time-series modeling on the access sequence at the current time to obtain the feature sequence at the current time; decoding the feature representations of the access strings in the feature sequence to obtain a first probability; and determining whether an anomaly is triggered at the current time based on the first probabilities of each access string in the access sequence at the current time. This scheme improves the accuracy of network access anomaly detection.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to an anomaly detection method and related devices, electronic equipment and storage media. Background Technology

[0002] In the internet field, how to accurately identify abnormal access behavior in real time has become a core issue in protecting network security and user rights.

[0003] However, both traditional manual review mechanisms and rule engines have significant limitations. The former is constrained by human resources and susceptible to subjective biases, while the latter struggles to handle large-scale user scenarios and is prone to misjudgments when the anomaly type is not covered by the rule engine. Therefore, improving the accuracy of network access anomaly detection has become an urgent problem to be solved. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide an anomaly detection method and related devices, electronic equipment, and storage media, which can improve the accuracy of network access anomaly detection.

[0005] To address the aforementioned technical problems, the first aspect of this application provides an anomaly detection method, comprising: extracting access strings at various times based on real-time network logs of a terminal device in a network environment; wherein the access strings include a target domain name and the target service to which the target domain name belongs; performing a sliding window based on the access strings at various times to obtain an access sequence at the current time; wherein the window length of the sliding window covers multiple times; performing time-series modeling based on the access sequence at the current time to obtain a feature sequence at the current time; wherein the feature sequence contains feature representations of each access string in the access sequence; decoding the feature representations of the access strings in the feature sequence to obtain a first probability; wherein the first probability represents the accuracy of mapping the decoded feature representations of the access strings to a target vocabulary, the target vocabulary being the vocabulary of the target service in the access strings; and determining whether an anomaly is triggered at the current time based on the first probability of each access string in the access sequence at the current time.

[0006] To address the aforementioned technical problems, a second aspect of this application provides an anomaly detection device, comprising: a log extraction module, a sequence sliding module, a time-series modeling module, a feature decoding module, and an anomaly determination module. The log extraction module is used to sequentially extract access strings at various times based on real-time network logs from a terminal device in a network environment; wherein the access strings include a target domain name and the target service to which the target domain name belongs. The sequence sliding module is used to slide a window based on the access strings at various times to obtain the access sequence at the current time; wherein the window length of the sliding window covers multiple times. The time-series modeling module is used to perform time-series modeling based on the access sequence at the current time to obtain a feature sequence at the current time; wherein the feature sequence contains feature representations of each access string in the access sequence. The feature decoding module is used to decode the feature representations of the access strings in the feature sequence to obtain a first probability; wherein the first probability represents the accuracy of mapping the decoded feature representations of the access strings to a target vocabulary, where the target vocabulary is the vocabulary of the target service in the access strings. The anomaly determination module is used to determine whether an anomaly is triggered at the current time based on the first probability of each access string in the access sequence at the current time.

[0007] To address the aforementioned technical problems, a third aspect of this application provides an electronic device comprising at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor executes the program instructions to implement the anomaly detection method described in the first aspect.

[0008] To address the aforementioned technical problems, a fourth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the anomaly detection method of the first aspect described above.

[0009] The above scheme, based on the real-time network logs of the terminal device in the network environment, sequentially extracts the access strings at each time point. Each access string includes the target domain name and the target service to which the target domain name belongs. Then, a sliding window is applied based on the access strings at each time point to obtain the access sequence at the current time point. The window length of the sliding window covers multiple time points. Temporal modeling is then performed based on the access sequence at the current time point to obtain the feature sequence at the current time point. The feature sequence contains the feature representations of each access string in the access sequence. Decoding is then performed based on the feature representations of the access strings in the feature sequence to obtain a first probability. The first probability represents the accuracy of mapping the decoded feature representations of the access strings to the target vocabulary, where the target vocabulary is the vocabulary of the target service in the access strings. Furthermore, based on each access string in the access sequence at the current time point... The first probability is used to determine whether an anomaly is triggered at the current moment. Firstly, since the access strings at each moment within the sliding window are used as a whole for temporal modeling to obtain a feature sequence, the feature representation of each access string at any moment can be dynamically generated depending on its context during the temporal modeling process. That is, the feature representation differs across access sequences due to variations in context, which helps to capture the personalized features of network access behavior as accurately as possible during decoding. Secondly, because the decoding process maps back to the vocabulary of the target service in the access string, and since the vocabulary of the target service in the access string can reflect normal behavior patterns to a certain extent, the first probability obtained through decoding helps to quantify the difference between the input access sequence and the normal behavior pattern as much as possible. Therefore, it can improve the accuracy of network access anomaly detection. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating an embodiment of the anomaly detection method of this application;

[0011] Figure 2 This is a schematic diagram of a process of an embodiment of the anomaly detection method of this application;

[0012] Figure 3 This is a schematic diagram of the framework of an embodiment of the anomaly detection device of this application;

[0013] Figure 4 This is a schematic diagram of the framework of an embodiment of the electronic device of this application;

[0014] Figure 5 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0015] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0016] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0017] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the slash " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper indicates two or more objects.

[0018] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the anomaly detection method of this application. It should be noted that the process operations in this embodiment can be executed by an electronic device with computing capabilities or related equipment containing an electronic device. The specific structure and type of the electronic device and related equipment containing the electronic device are not limited herein. Specifically, this embodiment may include the following steps:

[0019] Step S11: Based on the real-time network logs of the terminal device in the network environment, extract the access strings at each time point in sequence.

[0020] In this embodiment of the disclosure, the access string may include the target domain name and the target service to which the target domain name belongs. For example, the target domain name can be extracted from the "target domain name" column in a real-time network log. Furthermore, the target service to which the target domain name belongs includes at least one type: social applications, financial applications, and web applications. Exemplarily, the target service can be represented by a string indicating the service name. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of a process of an embodiment of the anomaly detection method of this application. Figure 2 As shown, s1 and s2 represent two access strings, which can each contain the target domain name and its associated target service (e.g., access string s1 contains target domain name 1 and target service 1, and access string s2 contains target domain name 2 and target service 2). Of course, Figure 2 The two access strings s1 and s2 are just one possible example of access strings in actual applications. Other possible cases are not limited here, nor will they be listed one by one.

[0021] In one implementation scenario, the terminal device may include, but is not limited to, smartphones, tablets, personal computers, etc., without specifying the specific type of terminal device.

[0022] In one implementation scenario, under network conditions, as various applications running on the terminal device perform normal tasks (such as reading novels, chatting, or shopping) or abnormal tasks (such as being controlled by a virus to automatically perform tasks like money transfers or file transfers), network logs will be continuously generated and updated. These are real-time network logs, which can contain access behaviors at various moments, such as timestamps of the access behaviors, the aforementioned target domain names and their respective target services, etc. Specific content is not limited here, nor will examples be provided. Based on this, the target domain names and their respective target services at each moment can be extracted sequentially according to the timestamps, forming access strings for each moment. As the real-time network logs are updated in real time, the access strings for each moment are also continuously updated. For example, after obtaining access strings s1 and s2, access strings s3 (not shown), s4 (not shown), ..., s2 can be generated as the real-time network logs are updated. t (Not shown).

[0023] Step S12: Perform a sliding window based on the access strings at each time point to obtain the access sequence at the current time point.

[0024] In this embodiment, the window length of the sliding window can cover multiple time points. For example, the window length of the sliding window can cover N time points. That is, unless there are special circumstances, the access sequence at the current time point can contain access strings from N time points. Of course, in practical applications, during the sliding window process, individual time points may be missing due to factors such as jitter, meaning that the actual number of access strings contained in the access sequence at the current time point may be less than N. For ease of description, the total number of access strings contained in the access sequence at the current time point can be denoted as K (K is not greater than N), where the access sequence at the current time point can be denoted as S=[s1,s2,…,s…]. K ].

[0025] In one implementation scenario, the sliding window's step size can be set to 1, meaning it slides for one time step at a time (i.e., when acquiring the access sequence at the next time step, the oldest access string in the current time step's access sequence will be popped and added to the latest access string in the current time step's access sequence). The access string at the next time step is still based on the aforementioned access sequence [s1, s2, ..., s...). K For example, the access sequence at the next moment becomes [s2, s3, ..., s]. K+1 Of course, the sliding step size of the sliding window can also be set to other values, such as 2, 3, etc. The specific value of the sliding step size is not limited here, nor will it be listed one by one.

[0026] In one implementation scenario, the access sequence can be updated according to a target frequency, such as once per second. That is, one second after the current moment, a sliding window can be applied again based on the access strings extracted from real-time network logs to arrive at the new access sequence for the current moment. Of course, the target frequency for access sequence updates can also be set to other values, such as once every two seconds, once every three seconds, etc. Here, the target frequency for access sequence updates is not limited, and examples will not be provided.

[0027] In one implementation scenario, as a possible approach, the access sequence at the current moment can include the access string at the current moment, and further, it can include access strings from several moments prior to the current moment. Taking the current moment as 't' as an example, the access sequence at the current moment can be denoted as [s...]. t-K+1 ,s t-K+2 ,…,s t Of course, the above example is only one possible example of the access sequence at the current moment under this situation. Other possible situations are not limited here, nor will they be listed one by one.

[0028] In another implementation scenario, as a different possible approach, distinct from the aforementioned implementation, the access sequence at the current moment can include the access string at the current moment, and further, it can include access strings from several moments after the current moment. Taking the current moment as 't' as an example, the access sequence at the current moment can be denoted as [s...]. t ,s t+1 ,…,s t+K-1 Of course, the above example is only one possible example of the access sequence at the current moment under this situation. Other possible situations are not limited here, nor will they be listed one by one.

[0029] In another implementation scenario, as yet another possible implementation method, differing from the aforementioned implementation methods, the access sequence at the current moment can include the access string at the current moment. Furthermore, the access sequence at the current moment can also include access strings from several moments before the current moment and access strings from several moments after the current moment. Taking the current moment as 't' as an example, the access sequence at the current moment can be denoted as [s]. t-(k-1) / 2 ,s t-（K-1） / 2+1 ,…,s t ,…,s t+(K-1) / 2 Of course, the above example is only one possible example of the access sequence at the current moment under this situation. Other possible situations are not limited here, nor will they be listed one by one.

[0030] Step S13: Perform time series modeling based on the access sequence at the current time to obtain the feature sequence at the current time.

[0031] In this embodiment of the disclosure, the feature sequence may include feature representations of each access string in the access sequence. It should be noted that, for any access string, its feature representation may include not only its own feature information, but also feature information of feature strings at other positions in the feature sequence. Here, the coverage of the feature information of the feature representation of the access string is not limited.

[0032] In one implementation scenario, as a possible approach, the access sequence at the current time step can be time-series modeled based on a multi-layer Transformer encoder to obtain the feature sequence at the current time step. Specifically, feature embedding can first be performed on each access string in the access sequence at the current time step to obtain the embedding representation of each access string. Then, the embedding representation of each access string in the access sequence at the current time step is processed by the multi-layer Transformer encoder to obtain the feature sequence at the current time step, i.e., the feature representation of each access string in the access sequence.

[0033] In a specific implementation scenario, to achieve feature embedding, a trainable embedding matrix E can be initialized, which can be represented as a matrix of length |V| with embedding dimension d. model A real matrix, where |V| can represent the total number of accessed strings in the access sequence. Based on this, for an access sequence S=[s1,s2,…,s…] K In this regard, feature embedding can be performed based on the embedding matrix E to obtain the embedding representation of each access string [x1, x2, ..., x]. K ], where x t =Es t Of course, the above example is only one possible example of feature embedding in practical applications. Other possible implementation methods are not limited here, nor will they be listed one by one.

[0034] In a specific implementation scenario, for a multi-layer Transformer encoder, each Transformer layer can sequentially include: a multi-head attention network and a feedforward neural network. Each Transformer layer can be configured with residual connections and layer normalization. The multi-head attention network ensures that each position in the access sequence simultaneously attends to all other positions in the access sequence, thereby directly and effectively capturing long-range dependencies. The multi-head attention mechanism enhances expressive power by repeating the operation H times (i.e., the number of heads) and concatenating and projecting the outputs from each iteration. The feedforward neural network can employ a fully connected network structure to perform non-linear transformations on the representation of each position to further extract features. For ease of description, the high-order sequence representation, i.e., the feature sequence, obtained after processing by the multi-layer Transformer encoder through its L-layer Transformer can be represented as:

[0035] ,

[0036] In the above formula, the superscript L indicates that it is output by the Lth layer Transformer, and the subscripts 1, 2, ..., K indicate the access string to which the feature belongs (e.g., the subscript 1 indicates that the feature belongs to the first access string).

[0037] In another implementation scenario, as a possible alternative, distinct from the aforementioned implementation, feature embedding can be performed on each access string in the access sequence to obtain an embedded representation of each access string. Then, based on these embedded representations, a first encoding and a second encoding are performed to obtain coded representations of each access string. The first encoding is performed in ascending order of time, and the second encoding is performed in ascending order of time. Finally, feature extraction is performed based on these coded representations to obtain the feature sequence for the current time. This approach, by using the first encoding in ascending order of time and the second encoding in ascending order of time, effectively captures the past and future context of each access string, thus improving the representational power of the coded representation.

[0038] In a specific implementation scenario, the specific method of feature embedding can be found in the aforementioned description of feature embedding of the access string using the initial embedding matrix E, which will not be repeated here.

[0039] In a specific implementation scenario, after obtaining the embedded representation of the access string, a first encoding and a second encoding can be performed on the embedded representation based on technologies such as Bi-GRU (Bidirectional Gated Recurrent Unit) and Bi-LSTM (Bidirectional Long Short-Term Memory) to obtain the encoded representation of the access string. Taking the implementation of the first and second encoding using Bi-GRU as an example, for the t-th access string in the access sequence, after performing the first encoding, the encoded representation can be obtained. :

[0040] ,

[0041] In the above formula, GRU represents the gated recurrent unit of a Bi-GRU. This represents the encoded representation of the (t-1)th access string after performing the first encoding, meaning that performing the first encoding can effectively capture information based on past context. Information Similarly, for the t-th access string in the access sequence, after performing the second encoding, the encoded representation can be obtained. :

[0042] ,

[0043] In the above formula, GRU represents the gated recurrent unit of a Bi-GRU. This represents the encoded representation of the (t-1)th access string after performing the second encoding, meaning that performing the second encoding can effectively capture information based on future context. Information Based on this, it is possible to effectively capture information based on past context by performing the first encoding. Information Executing the second encoding can effectively capture future context-based information. Information By performing fusion (e.g., concatenation), the final encoded representation h of the t-th access string is obtained. t :

[0044] ,

[0045] In the above formula, [.||.] represents the concatenation operation. Therefore, it can be seen that for any access string at time t, its final encoded representation can contain not only its own string s. tThe feature information also incorporates the contextual influence of the entire access sequence S. Of course, the above example is merely one possible illustration of using Bi-GRU to implement the first and second encodings; other possible methods are not limited here, nor will they be listed in detail.

[0046] In a specific implementation scenario, after obtaining the final encoded representation of each access string in the access sequence, feature extraction can be performed to obtain the feature sequence at the current time. Specifically, positional encoding can be performed on each access string in the access sequence to obtain the positional representation of each access string. For example, for the access sequence, a sine positional encoding function and a cosine positional encoding function can be used to generate the position matrix. Where K represents the total number of accessed strings in the access sequence, and d model The feature dimension representing the positional representation, that is, for the entire access sequence, its positional representation can be characterized as K. d model A real matrix of size d; in other words, for any access string in the access sequence, its position representation can be characterized as d. model The feature vector of size. Furthermore, for the specific process of positional encoding, please refer to the technical details of positional encoding functions such as the sine positional encoding function and the cosine positional encoding function, which will not be elaborated here. After this, the encoded representations and positional representations of the same access strings in the access sequence can be fused (e.g., added) to obtain the initial representation of the corresponding access strings in the access sequence. For ease of description, the overall encoded representation of the access sequence after the aforementioned first and second encodings can be represented as Z. (0) Then the initial representation H of the entire access sequence. (0) It can be characterized as:

[0047] H (0) =Z (0) +P

[0048] In the above formula, H (0) Z represents the initial representation of the entire access sequence. (0)Let P represent the overall encoded representation of the access sequence after the first and second encodings described above, and let P represent the overall positional representation of the access sequence. Based on this, attention processing can be performed on the initial representations of each access string in the access sequence to obtain the feature sequence at the current time step. For example, the initial representations of each access string in the access sequence can be processed using a multi-layer Transformer encoder to obtain the feature sequence at the current time step; details can be found in the aforementioned descriptions and will not be repeated here. This method, by performing positional encoding on each access string in the access sequence to obtain the positional representation of each access string, and fusing the encoded and positional representations of the same access string in the access sequence to obtain the initial representation of the corresponding access string, and then performing attention processing on the initial representations of each access string in the access sequence to obtain the feature sequence at the current time step, can ensure, as accurately as possible, the capture of the relative positional relationships of each access string in the access sequence.

[0049] Step S14: Decode the first probability based on the feature representation of the accessed string in the feature sequence.

[0050] In this embodiment of the disclosure, the first probability is characterized by the accuracy of mapping the feature representation of the access string to the target vocabulary, where the target vocabulary is the vocabulary of the target service in the access string. It should be noted that each application service has a corresponding vocabulary under normal access behavior. The higher the first probability of decoding based on the feature representation to map back to the target vocabulary, the greater the likelihood that the access string represents the target service under normal access behavior; conversely, the lower the first probability of decoding based on the feature representation to map back to the target vocabulary, the less likely the access string represents the target service under normal access behavior.

[0051] In one implementation scenario, to improve decoding efficiency, a sequence reconstruction model can be pre-trained to achieve the aforementioned temporal modeling and decoding operations. This sequence reconstruction model could include Bi-GRU, Bi-LSTM, etc., for implementing bidirectional encoding, a multi-layer Transformer encoder for extracting features from the encoded representation, and a decoder for implementing the decoding operation. The decoder can include, but is not limited to, linear layers; the network structure of the decoder is not limited here. For ease of description, we will still use the aforementioned access sequence S=[s1,s2,…,s…]. K For example, taking the feature extraction using an L-layer Transformer encoder, its feature sequence H... (L) It can be represented as:

[0052] ,

[0053] In the above formula, This represents the feature representation of the first accessed string in the feature sequence; others can be deduced similarly, and will not be elaborated further here. Based on this, a decoder can be used to extract the feature sequence H. (L) Mapping back to the target service's vocabulary (i.e., the target vocabulary), we obtain the logits output o of the t-th access string in the access sequence. t for:

[0054] ,

[0055] In the above formula, This represents the feature representation of the t-th access string in the feature sequence. , These represent the weight and bias parameters of the decoder, respectively. It should be noted that the logits output o of the t-th access string in the access sequence... t This can be viewed as the feature representation of the t-th access string mapping back to the accurate score of the vocabulary of its target service. Based on this, the above logits output can be further transformed into a probability distribution using the Softmax function, such as the logits output o of the t-th access string. t It can be converted to the first probability using the Softmax function. :

[0056] ,

[0057] In one implementation scenario, to maximize the accuracy of the sequence reconstruction model, sample access sequences generated under normal access behavior and sample access sequences generated under abnormal access behavior can be pre-collected, and these sample access sequences can be labeled with sample tags (e.g., ...). Figure 2As shown, the sequence reconstruction model can be trained in a supervised manner. For the former, each sample access string in the sample access sequence can be labeled with a normal label (e.g., "1"), and for the latter, each sample access string in the sample access sequence can be labeled with an abnormal label (e.g., "0"). Based on this, temporal modeling of the sample access sequence can be performed based on the sequence reconstruction model, which may include bidirectional encoding, feature extraction, and decoding prediction (see the aforementioned descriptions for details, which will not be repeated here). This will yield the predicted probability of the sample access string (the specific meaning of which can be found in the aforementioned description of the first probability, which will also not be repeated here). The predicted probability of the sample string and the sample label can be measured using a loss function such as cross-entropy to obtain the training loss. Based on the training loss, the network parameters of the sequence reconstruction model can be adjusted, thereby forcing the sequence reconstruction model to learn the vocabulary space of relevant services under normal access behavior during training. Of course, the above example is only one possible example of training a sequence reconstruction model in practical applications. Other possible training methods are not limited here, nor will they be listed one by one.

[0058] Step S15: Based on the first probability of each access string in the access sequence at the current moment, determine whether an exception is triggered at the current moment.

[0059] In one implementation scenario, as a possible approach, after obtaining the first probability of each access string in the access sequence at the current time, the second probability of the access sequence at the current time can be obtained by fusing (e.g., averaging) the first probabilities of each access string in the access sequence at the current time. For ease of description, the aforementioned access sequence S=[s1,s2,…,s] will still be used. K For example, the second probability score(S) can be expressed as:

[0060] ,

[0061] In the above formula, This represents the t-th access string s in the access sequence. t The first probability is given, where K represents the total number of accessed strings in the access sequence. Based on this, the second probability of the access sequence at the current moment can be used to determine whether an exception is triggered at the current moment. For example, it can be checked whether the second probability of the access sequence at the current moment meets the triggering condition (e.g., the second probability is lower than a probability threshold, the second probability is not higher than a probability threshold, etc.). If so, it can be determined that an exception is triggered at the current moment; otherwise, it can be determined that no exception is triggered at the current moment.

[0062] In another implementation scenario, as a possible alternative, distinct from the aforementioned implementation, a second probability of the access sequence at the current moment can be obtained by fusing the first probabilities of each access string in the access sequence at the current moment. This second probability is then fused with the second probabilities of the access sequences at the current moment and several historical moments prior to the current moment to obtain the target probability at the current moment. Based on this, it can be determined whether an anomaly is triggered at the current moment, at least based on the target probability. This approach, by further combining the second probabilities of the access sequences at several historical moments prior to the current moment to obtain the target probability at the current moment, and thereby determining whether an anomaly is triggered, can minimize the impact of noise and effectively reduce instantaneous false alarms (i.e., accidental anomaly triggers).

[0063] In a specific implementation scenario, the target probability can be obtained by fusing (e.g., averaging) the second probabilities of the current time and the access sequences of the previous 2, 3 or more historical time moments. Here, the specific number of historical time moments is not limited, nor will examples be given one by one.

[0064] In a specific implementation scenario, after obtaining the target probability, it can be determined whether an anomaly should be triggered at the current moment based on whether the target probability at the current moment meets the triggering conditions; alternatively, it can be determined whether an anomaly should be triggered at the current moment based on whether the target probabilities at the current moment and several historical moments prior to the current moment all meet the triggering conditions. It should be noted that the triggering conditions include any of the following: the target probability is lower than a probability threshold, or the target probability is not higher than a probability threshold.

[0065] In another implementation scenario, as yet another possible implementation method, distinct from the aforementioned implementation methods, access sequences at various moments during normal access can be pre-acquired and used as reference sequences. For example, a large number of access sequences at various moments under different behavioral patterns during normal access (e.g., chatting, shopping, reading, etc.) can be collected and used as reference sequences. Based on this, time-series modeling can be performed on each reference sequence using the aforementioned sequence reconstruction model for time-series modeling to obtain the feature sequences of each reference sequence. For details, please refer to the aforementioned description of the feature sequences at the current moment; further elaboration is omitted here. In this way, clustering can be performed based on the feature sequences of each reference sequence to obtain several cluster sets. Each cluster set contains the feature sequences of at least one reference sequence. Different cluster sets represent feature sets under different behavioral patterns during normal access. Then, detection is performed based on each cluster set and the feature sequence at the current moment to obtain the detection result. The detection result is characterized by whether the feature sequence at the current moment can be classified into any cluster set. In response to the detection result that the feature sequence at the current moment cannot be classified into any cluster set, the aforementioned step of decoding based on the feature representation of the access string in the feature sequence can be performed to obtain the first probability, so as to further determine whether an anomaly is triggered at the current moment based on the first probability. The above method, by pre-constructing cluster sets representing different behavioral patterns during normal access, and deciding whether to further perform decoding operations based on whether the feature sequence at the current moment can be classified into any cluster set, helps to further improve the efficiency of anomaly detection.

[0066] In a specific implementation scenario, after obtaining each cluster set, it is possible to determine whether the current feature sequence can be assigned to a cluster set based on the feature distance between the current feature sequence and the cluster center of each cluster set. For example, if the feature distance is less than (or not greater than) the distance threshold, it can be determined that the current feature sequence can be assigned to a cluster set; otherwise, if the feature distance is not less than (or greater than) the distance threshold, it can be determined that the current feature sequence cannot be assigned to a cluster set.

[0067] In a specific implementation scenario, if the detection results, including the feature sequence at the current moment, can be categorized into a cluster set, it can be directly determined that no anomaly has been triggered at the current moment.

[0068] In a specific implementation scenario, if the detection result indicates that the current feature sequence cannot be classified into any cluster set, after determining whether an anomaly has been triggered at the current moment based on the first probability of each access string in the access sequence, a new cluster set can be created based on the current feature sequence in response to the determination that no anomaly has been triggered at the current moment. In other words, although the current feature sequence cannot be classified into any cluster set, since decoding and prediction can determine that no anomaly needs to be triggered, the current feature sequence can be considered a new behavior pattern during normal access. In this case, to facilitate subsequent anomaly detection of new access sequences, a new cluster set can be created based on the current feature sequence. This way, when a feature sequence exhibiting this behavior pattern reappears, it can be directly classified into the new cluster set without further decoding and prediction.

[0069] The above scheme, based on the real-time network logs of the terminal device in the network environment, sequentially extracts the access strings at each time point. Each access string includes the target domain name and the target service to which the target domain name belongs. Then, a sliding window is applied based on the access strings at each time point to obtain the access sequence at the current time point. The window length of the sliding window covers multiple time points. Temporal modeling is then performed based on the access sequence at the current time point to obtain the feature sequence at the current time point. The feature sequence contains the feature representations of each access string in the access sequence. Decoding is then performed based on the feature representations of the access strings in the feature sequence to obtain a first probability. The first probability represents the accuracy of mapping the decoded feature representations of the access strings to the target vocabulary, where the target vocabulary is the vocabulary of the target service in the access strings. Furthermore, based on each access string in the access sequence at the current time point... The first probability is used to determine whether an anomaly is triggered at the current moment. Firstly, since the access strings at each moment within the sliding window are used as a whole for temporal modeling to obtain a feature sequence, the feature representation of each access string at any moment can be dynamically generated depending on its context during the temporal modeling process. That is, the feature representation differs across access sequences due to variations in context, which helps to capture the personalized features of network access behavior as accurately as possible during decoding. Secondly, because the decoding process maps back to the vocabulary of the target service in the access string, and since the vocabulary of the target service in the access string can reflect normal behavior patterns to a certain extent, the first probability obtained through decoding helps to quantify the difference between the input access sequence and the normal behavior pattern as much as possible. Therefore, it can improve the accuracy of network access anomaly detection.

[0070] Please see Figure 3 , Figure 3This is a schematic diagram of the framework of an embodiment of the anomaly detection device of this application. The anomaly detection device 30 includes: a log extraction module 31, a sequence sliding module 32, a time-series modeling module 33, a feature decoding module 34, and an anomaly determination module 35. The log extraction module 31 is used to extract access strings at various times based on real-time network logs from the terminal device in a network environment; wherein the access strings include the target domain name and the target service to which the target domain name belongs. The sequence sliding module 32 is used to slide a window based on the access strings at various times to obtain the access sequence at the current time; wherein the window length of the sliding window covers multiple times. The time-series modeling module 33 is used to perform time-series modeling based on the access sequence at the current time to obtain the feature sequence at the current time; wherein the feature sequence contains the feature representations of each access string in the access sequence. The feature decoding module 34 is used to decode the feature representations of the access strings in the feature sequence to obtain a first probability; wherein the first probability represents the accuracy of mapping the decoded feature representations of the access strings to a target vocabulary, where the target vocabulary is the vocabulary of the target service in the access strings. The anomaly determination module 35 is used to determine whether an anomaly is triggered at the current time based on the first probability of each access string in the access sequence at the current time.

[0071] In the above scheme, the anomaly detection device 30 extracts access strings at various times based on the real-time network logs of the terminal device in the network environment. Each access string includes the target domain name and the target service to which the target domain name belongs. Then, a sliding window is used to obtain the access sequence at the current time, with the window length covering multiple times. Temporal modeling is then performed based on the access sequence at the current time to obtain the feature sequence at the current time. This feature sequence contains the feature representations of each access string in the access sequence. The feature representations of the access strings in the feature sequence are then decoded to obtain a first probability. This first probability represents the accuracy of mapping the decoded feature representations of the access strings to the target vocabulary, where the target vocabulary is the vocabulary of the target service in the access strings. Furthermore, based on the access sequence at the current time, each access string... The first probability of a string is used to determine whether an anomaly is triggered at the current moment. Firstly, since the access strings at each moment within the sliding window are treated as a whole for temporal modeling to obtain a feature sequence, the feature representation of each access string at any given moment can be dynamically generated based on its context. This means that the feature representation differs across access sequences due to varying contexts, helping to capture the personalized features of network access behavior as accurately as possible during decoding. Secondly, the decoding process maps back to the vocabulary of the target service in the access string. Since the vocabulary of the target service in the access string can reflect normal behavior patterns to some extent, the first probability obtained through decoding helps to quantify the difference between the input access sequence and the normal behavior pattern as much as possible. Therefore, the accuracy of network access anomaly detection can be improved.

[0072] In some disclosed embodiments, the anomaly determination module 35 includes a first fusion submodule, used to fuse based on the first probability of each access string in the access sequence at the current time to obtain a second probability of the access sequence at the current time; the anomaly determination module 35 includes a second fusion submodule, used to fuse based on the second probabilities of the access sequences at the current time and several historical times before the current time to obtain a target probability at the current time; the anomaly determination module 35 includes an anomaly determination submodule, used to determine whether an anomaly is triggered at the current time, at least based on the target probability at the current time.

[0073] In some disclosed embodiments, the anomaly determination submodule is specifically used to determine whether an anomaly is triggered at the current time based on whether the target probability at the current time meets the triggering condition; or, to determine whether an anomaly is triggered at the current time based on whether the target probabilities at the current time and several historical times before the current time all meet the triggering condition; wherein, the triggering condition includes any of the following: the target probability is lower than the probability threshold, or the target probability is not higher than the probability threshold.

[0074] In some disclosed embodiments, the time-series modeling module 33 includes a feature embedding submodule, used to perform feature embedding based on each access string in the access sequence to obtain an embedded representation of each access string in the access sequence; the time-series modeling module 33 includes a bidirectional encoding submodule, used to perform a first encoding and a second encoding based on the embedded representation of each access string in the access sequence to obtain an encoded representation of each access string in the access sequence; wherein, the first encoding is characterized by encoding in the order from late to early time, and the second encoding is characterized by encoding in the order from early to late time; the time-series modeling module 33 includes a feature extraction submodule, used to perform feature extraction based on the encoded representation of each access string in the access sequence to obtain a feature sequence at the current time.

[0075] In some disclosed embodiments, the feature extraction submodule includes a position encoding unit, used to perform position encoding on each access string in the access sequence to obtain the position representation of each access string in the access sequence; the feature extraction submodule includes a representation fusion unit, used to fuse the encoded representation and position representation of the same access string in the access sequence to obtain the initial representation of the corresponding access string in the access sequence; the feature extraction submodule includes an attention processing unit, used to perform attention processing on the initial representation of each access string in the access sequence to obtain the feature sequence at the current time.

[0076] In some disclosed embodiments, the feature sequence at the current moment is obtained by performing time-series modeling on the access sequence at the current moment using a sequence reconstruction model. The anomaly detection device 30 includes a reference acquisition module, used to acquire the access sequences at each moment during normal access, which are respectively used as reference sequences. The time-series modeling module 33 is also used to perform time-series modeling on each reference sequence based on the sequence reconstruction model to obtain the feature sequences of each reference sequence. The anomaly detection device 30 includes a feature clustering module, used to perform clustering based on the feature sequences of each reference sequence to obtain several cluster sets. The cluster sets contain the feature sequences of at least one reference sequence. The anomaly detection device 30 includes a classification detection module, used to perform detection based on each cluster set and the feature sequence at the current moment to obtain a detection result. The detection result is characterized as whether the feature sequence at the current moment can be classified into any cluster set. The feature decoding module 34 is specifically used to perform a step of decoding based on the feature representation of the access string in the feature sequence to obtain a first probability in response to the detection result including that the feature sequence at the current moment cannot be classified into any cluster set.

[0077] In some disclosed embodiments, when the detection result includes a feature sequence at the current moment that cannot be classified into any cluster set, the anomaly detection device 30 includes a cluster creation module for creating a new cluster set based on the feature sequence at the current moment in response to determining that no anomaly has been triggered at the current moment.

[0078] In some disclosed embodiments, the anomaly determination module 35 is further configured to directly determine that no anomaly has been triggered at the current time in response to the detection result including the feature sequence at the current moment being able to be classified into a cluster set.

[0079] In some publicly available embodiments, the target service to which the target domain belongs includes at least one type of social application, financial application, or web application.

[0080] In some disclosed embodiments, the access sequence at the current moment includes the access string at the current moment, and the access sequence at the current moment also includes at least one of the following: the access string at several moments before the current moment, and the access string at several moments after the current moment.

[0081] Please see Figure 4 , Figure 4 This is a schematic diagram of a framework of an embodiment of the electronic device of this application. The electronic device 40 includes at least a memory 41 and a processor 42 coupled to each other. The memory 41 stores at least program instructions, and the processor 42 is used to execute the program instructions to implement the steps in any of the above-described anomaly detection method embodiments. For details, please refer to the foregoing disclosed embodiments, which will not be repeated here.

[0082] Specifically, processor 42 controls itself and memory 41 to implement the steps in any of the above-described anomaly detection method embodiments. Processor 42 can also be referred to as a CPU (Central Processing Unit). Processor 42 may be an integrated circuit chip with signal processing capabilities. Processor 42 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 42 can be implemented using integrated circuit chips.

[0083] In the above scheme, the electronic device 40 extracts access strings at various times based on the real-time network logs of the terminal device in the network environment. Each access string includes the target domain name and the target service to which the target domain name belongs. Then, a sliding window is used based on the access strings at each time to obtain the access sequence at the current time. The window length of the sliding window covers multiple time points. Temporal modeling is then performed based on the access sequence at the current time to obtain the feature sequence at the current time. The feature sequence contains the feature representations of each access string in the access sequence. Decoding is performed based on the feature representations of the access strings in the feature sequence to obtain a first probability. The first probability represents the accuracy of mapping the decoded feature representations of the access strings to the target vocabulary, where the target vocabulary is the vocabulary of the target service in the access strings. Furthermore, based on each access string in the access sequence at the current time... The first probability of the string determines whether an anomaly is triggered at the current moment. Firstly, because temporal modeling is performed using the access strings at each moment within the sliding window as a whole to obtain the feature sequence, the feature representation of the access string at any given moment can be dynamically generated based on its context. This means that the feature representation differs across access sequences due to varying contexts, helping to capture the personalized features of network access behavior as accurately as possible during decoding. Secondly, because the decoding process maps back to the vocabulary of the target service in the access string, which can reflect normal behavior patterns to some extent, the first probability obtained through decoding helps to quantify the difference between the input access sequence and the normal behavior pattern as much as possible. Therefore, it can improve the accuracy of network access anomaly detection.

[0084] Please see Figure 5 , Figure 5 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 50 stores program instructions 51 that can be executed by a processor. The program instructions 51 are used to implement the steps in any of the above-described embodiments of the anomaly detection method.

[0085] The above scheme involves a computer-readable storage medium 50 extracting access strings at various times based on real-time network logs from a terminal device in a networked environment. Each access string includes the target domain name and the target service to which it belongs. A sliding window is then used to obtain the access sequence at the current time, with the window length covering multiple times. Temporal modeling is then performed based on the access sequence at the current time to obtain a feature sequence. This feature sequence contains feature representations of each access string in the access sequence. Decoding is then performed on the feature representations of the access strings in the feature sequence to obtain a first probability. This first probability represents the accuracy of mapping the decoded feature representations of the access strings to a target vocabulary, where the target vocabulary is the vocabulary of the target service in the access strings. Furthermore, based on the access sequence at the current time... The first probability of each access string determines whether an anomaly is triggered at the current moment. Firstly, since temporal modeling is performed using access strings at various times within a sliding window as a whole to obtain a feature sequence, the feature representation of each access string at any given time moment can be dynamically generated based on its context. This means that the feature representation differs across access sequences due to varying contexts, helping to capture the personalized features of network access behavior as accurately as possible during decoding. Secondly, because the decoding process maps back to the vocabulary of the target service in the access string, which can reflect normal behavior patterns to some extent, the first probability obtained through decoding helps to quantify the difference between the input access sequence and the normal behavior pattern as much as possible. Therefore, it can improve the accuracy of network access anomaly detection.

[0086] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0087] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0089] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0090] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0091] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0092] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. An anomaly detection method characterized by, include: Based on the real-time network logs of the terminal device in the network environment, the access strings at each time point are extracted sequentially; wherein, the access string includes the target domain name and the target service to which the target domain name belongs; A sliding window is used to obtain the access sequence at the current time based on the access strings at each time point; wherein the window length of the sliding window covers multiple time points; Temporal modeling is performed based on the access sequence at the current moment to obtain the feature sequence at the current moment; wherein, the feature sequence contains the feature representation of each access string in the access sequence; wherein, the feature sequence at the current moment is obtained by temporal modeling of the access sequence at the current moment by a sequence reconstruction model; Obtain the access sequence at each time point during normal access, and use them as reference sequences respectively; Based on the sequence reconstruction model, time series modeling is performed on each of the reference sequences to obtain the feature sequences of each of the reference sequences; Clustering is performed based on the feature sequences of each of the reference sequences to obtain several cluster sets; wherein, each cluster set contains the feature sequence of at least one of the reference sequences; Detection is performed based on each of the cluster sets and the feature sequence at the current moment to obtain a detection result; wherein, the detection result is characterized by whether the feature sequence at the current moment can be classified into any of the cluster sets; In response to the detection result including that the feature sequence at the current moment cannot be classified into any of the cluster sets, decoding is performed based on the feature representation of the access string in the feature sequence to obtain a first probability. Based on the first probability of each access string in the access sequence at the current moment, it is determined whether an anomaly is triggered at the current moment. The first probability represents the accuracy of the feature representation decoding mapping to the target vocabulary, where the target vocabulary is the vocabulary of the target service in the access string. The higher the first probability of decoding based on the feature representation to map back to the target vocabulary, the greater the likelihood that the access string is generated by the target service under normal access behavior. Conversely, the lower the first probability of decoding based on the feature representation to map back to the target vocabulary, the less likely that the access string is generated by the target service under normal access behavior. In response to the detection result including the fact that the feature sequence at the current moment can be classified into the cluster set, it is directly determined that no anomaly has been triggered at the current moment.

2. The method of claim 1, wherein, Determining whether an exception is triggered at the current moment based on the first probability of each access string in the access sequence at the current moment includes: The second probability of the access sequence at the current moment is obtained by fusing the first probabilities of each access string in the access sequence at the current moment. The target probability at the current moment is obtained by fusing the second probabilities of the access sequences at the current moment and several historical moments prior to the current moment. Based at least on the target probability at the current moment, determine whether an anomaly is triggered at the current moment.

3. The method of claim 2, wherein, Determining whether an anomaly is triggered at the current moment, based at least on the target probability at the current moment, includes: Whether an anomaly is triggered at the current moment is determined based on whether the target probability at the current moment meets the triggering condition; or, whether an anomaly is triggered at the current moment is determined based on whether the target probabilities at the current moment and several historical moments prior to the current moment all meet the triggering condition; wherein, the triggering condition includes any one of the following: the target probability is lower than a probability threshold, or the target probability is not higher than a probability threshold.

4. The method of claim 1, wherein, The time-series modeling based on the access sequence at the current moment, to obtain the feature sequence at the current moment, includes: Based on feature embedding of each of the access strings in the access sequence, an embedded representation of each of the access strings in the access sequence is obtained; Based on the embedding representation of each of the access strings in the access sequence, a first encoding and a second encoding are performed respectively to obtain the encoded representation of each of the access strings in the access sequence; wherein, the first encoding is characterized by encoding in order from late to early time, and the second encoding is characterized by encoding in order from early to late time; Feature extraction is performed based on the encoded representation of each access string in the access sequence to obtain the feature sequence at the current time.

5. The method of claim 4, wherein, The feature extraction based on the encoded representation of each access string in the access sequence to obtain the feature sequence at the current time includes: Position encoding is performed on each of the access strings in the access sequence to obtain the position representation of each of the access strings in the access sequence; The initial representation of the access string corresponding to the access string in the access sequence is obtained by fusing the encoded representation and positional representation of the same access string in the access sequence. Attention processing is performed on the initial representations of each of the access strings in the access sequence to obtain the feature sequence at the current time.

6. The method of claim 1, wherein, If the detection result includes the fact that the feature sequence at the current moment cannot be classified into any of the cluster sets, after determining whether an anomaly is triggered at the current moment based on the first probability of each of the access strings in the access sequence at the current moment, the method further includes: In response to determining that no exception has been triggered at the current moment, a new cluster set is created based on the feature sequence at the current moment.

7. The method according to any one of claims 1 to 6, characterized in that, The target service to which the target domain belongs includes at least one type of social application, financial application, or web application. And / or, the access sequence at the current moment includes the access string at the current moment, and the access sequence at the current moment also includes at least one of the following: access strings at several moments before the current moment, and access strings at several moments after the current moment.

8. An abnormality detection device characterized by comprising: include: The log extraction module is used to extract access strings at various times based on the real-time network logs of the terminal device in the network environment; wherein, the access string includes the target domain name and the target service to which the target domain name belongs; The sequence sliding module is used to slide a window based on the access strings at each time step to obtain the access sequence at the current time step; the window length of the sliding window covers multiple time steps. A time-series modeling module is used to perform time-series modeling based on the access sequence at the current time to obtain the feature sequence at the current time; wherein, the feature sequence includes the feature representation of each access string in the access sequence; wherein, the feature sequence at the current time is obtained by performing time-series modeling on the access sequence at the current time by a sequence reconstruction model; The reference acquisition module is used to acquire the access sequence at each time point during normal access, and use them as reference sequences respectively; The sequence reconstruction module is used to perform time-series modeling on each of the reference sequences based on the sequence reconstruction model to obtain the feature sequences of each of the reference sequences. A feature clustering module is used to cluster based on the feature sequences of each of the reference sequences to obtain several cluster sets; wherein, the cluster set contains the feature sequence of at least one of the reference sequences; The detection module is used to perform detection based on each of the cluster sets and the feature sequence at the current time to obtain a detection result; wherein, the detection result is characterized by whether the feature sequence at the current time can be classified into any of the cluster sets; A feature decoding module is configured to, in response to the detection result including the feature sequence at the current moment not being able to be classified into any of the cluster sets, perform decoding based on the feature representation of the access string in the feature sequence to obtain a first probability, and determine whether an anomaly is triggered at the current moment based on the first probability of each access string in the access sequence at the current moment; wherein, the first probability represents: the accuracy of the feature representation decoding of the access string mapping to the target vocabulary, the target vocabulary being the vocabulary of the target service in the access string; the higher the first probability of decoding based on the feature representation to map back to the target vocabulary, the greater the probability that the access string is generated by the target service under normal access behavior; the lower the first probability of decoding based on the feature representation to map back to the target vocabulary, the smaller the probability that the access string is generated by the target service under normal access behavior. The clustering creation module is used to directly determine that no anomaly has been triggered at the current time in response to the detection result including the fact that the feature sequence at the current moment can be classified into the cluster set.

9. An electronic device, comprising: It includes at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the anomaly detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The system stores program instructions that can be executed by a processor, the program instructions being used to implement the anomaly detection method according to any one of claims 1 to 7.

Images