Method and device for analyzing abnormal operation

Abstract

A method and device for analyzing operational anomalies, used to analyze operational behaviors, promptly detect operational anomalies, and issue alerts. The method includes: for each action during a user's operation, predicting multiple predicted behaviors that the user might perform next based on the user's current behavior in their current state, as well as a first risk value for each predicted behavior; if the user's actual next action matches one of the predicted actions, and the first risk value of the predicted action is greater than a first risk threshold, marking the actual action as suspicious; and if multiple suspicious actions occur during the user's operation, determining that the user's operation is abnormal.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a method and device for analyzing abnormal operations. Background Art

[0002] With the rapid development of the internet, various cyberattacks are emerging one after another. Social engineering attacks are a common and effective means of attack. After gaining access to a user's personal terminal, attackers will immediately search and browse useful information within the account to conduct subsequent expansion and penetration attacks. Email systems are particularly vulnerable to information leaks. By exploiting system interface vulnerabilities, attackers can log into any mailbox and access sensitive information. For example, by browsing email content, they can obtain important information such as account names and passwords. Furthermore, attackers can forge the content of historical emails in an email account and send simulated phishing emails to other users.

[0003] Therefore, there is an urgent need for a solution to analyze operational behaviors, detect operational anomalies in a timely manner, and issue alarms. Summary of the Invention

[0004] The present application provides a method and device for analyzing operation anomalies, which are used to analyze operation behaviors, detect operation anomalies in a timely manner and issue alarms.

[0005] In the first aspect, the present application provides a method for analyzing operation abnormalities, which includes: for each behavior during the user's operation, predicting multiple predicted behaviors that the user may perform in the next step based on the current behavior of the user in the current state, and a first risk value for each predicted behavior; if the actual behavior of the user in the next step matches one of the predicted behaviors, and the first risk value of the predicted behavior is greater than a first risk threshold, the actual behavior is marked as suspicious behavior; when multiple suspicious behaviors occur during the user's operation, it is determined that the user's operation is abnormal.

[0006] In the above technical solution, each action during the user's operation is simulated and deduced to predict the user's next possible action and the first risk value of each predicted action in advance. When the user's next actual action occurs, the risk value of the actual action is immediately matched and the first risk threshold is used to determine whether the actual action is suspicious. If the user's operation repeatedly exhibits suspicious behavior, the user's operation is determined to be abnormal and an alarm is issued. This allows for timely detection and interception before more private information is accessed by the attacker or before any file transfer operation is performed.

[0007] In one possible design, the predicting of multiple predicted behaviors that may occur next by the user based on the current behavior in the current state of the user, as well as the first risk value of each predicted behavior, includes: reasoning about the current behavior in the current state of the user to obtain multiple predicted behaviors that may occur next; in any traversal process, selecting the predicted behavior of this traversal from the multiple predicted behaviors, and updating the risk value of the predicted behavior based on the simulated behavior chain of the predicted behavior of this traversal; the simulated behavior chain is obtained by multiple expansions of subsequent behaviors of the predicted behavior of this traversal; after the simulated behavior chain of multiple predicted behaviors meets the traversal requirements, obtaining the first risk value of each predicted behavior.

[0008] In one possible design, the simulated behavior chain is obtained in the following manner: the current behavior in the current state is used as the root node; each time the expansion is performed, the first behavior sub-node with the largest risk value is selected from the root node to determine whether the first behavior sub-node is fully expanded; the risk value is determined by a first parameter value and a second parameter value; the second parameter value is positively correlated with the risk probability of the behavior sub-node and negatively correlated with the number of times the behavior sub-node is visited; the risk probability is obtained through a strategy neural network; the first behavior sub-node is any one of a plurality of predicted behaviors; if the first behavior sub-node is not fully expanded, the subsequent plurality of second behavior sub-nodes of the first behavior sub-node are simulated through the strategy neural network to obtain a simulated behavior chain of the predicted behavior.

[0009] In one possible design, the risk value of the predicted behavior is updated, including: for each second behavior sub-node in the simulated behavior chain of the predicted behavior, obtaining a simulation result of the second behavior sub-node through a fast strategy neural network, and obtaining a third parameter value of the second behavior sub-node through a value neural network; the simulation result indicates whether the second behavior sub-node operates abnormally; based on the simulation result of each second behavior sub-node and the third parameter value of each second behavior sub-node, updating the first parameter value of the first behavior sub-node of the simulated behavior chain; and updating the risk value of the predicted behavior based on the first parameter value of the simulated behavior chain.

[0010] In the above technical solution, the risk value of the predicted behavior finally obtained is not only related to the current behavior, but also to the previous and next behaviors during the operation process, which can make the prediction result more accurate.

[0011] In one possible design, the method further includes: training the value neural network by the following method: for each state and behavior in the training data, executing L steps of behavior after the state and behavior through the policy neural network; selecting the behavior with the highest risk probability to execute in each step of the L steps; randomly executing a behavior in the L+1 step; executing M steps of behavior after the L+1 step through the enhanced policy neural network to obtain a simulation result; selecting the behavior with the highest risk probability to execute in each step of the M steps; and using the simulation result as the label of the training data to train the value neural network.

[0012] In one possible design, the user's current state includes the current page where the user is staying and the content of the current page; before inferring the current behavior of the user in the current state through the risk prediction model to obtain multiple predicted behaviors that may occur next, it also includes: converting the current behavior of the user in the current state into a feature vector; wherein, the content of the current page is converted into a feature vector through word embedding.

[0013] In the above technical solution, the user's current state includes the current page where the user is staying and the content of the current page. The content of the current page is unstructured data with a large spatial dimension. Therefore, word embedding is used to convert the content of the page into a feature vector, which can not only reduce the spatial dimension of the content, but also take into account the connection between the content context.

[0014] In one possible design, when multiple suspicious behaviors occur during the user's operation, the user's operation is determined to be abnormal, including: accumulating the first risk value of each suspicious behavior during the user's operation to obtain a second risk value; the first risk value of the suspicious behavior is the first risk value of the corresponding predicted behavior; if the second risk value is greater than a second risk threshold, the user's operation is determined to be abnormal.

[0015] In a second aspect, an embodiment of the present application provides an operation abnormality analysis device, comprising:

[0016] A prediction module is configured to predict, for each behavior during a user operation, a plurality of predicted behaviors that the user may perform next based on the current behavior of the user in the current state, and a first risk value of each predicted behavior;

[0017] a processing module, configured to mark the actual behavior as suspicious if the user's next actual behavior matches one of the predicted behaviors and the first risk value of the predicted behavior is greater than a first risk threshold;

[0018] The determination module is used to determine that the user's operation is abnormal when multiple suspicious behaviors occur during the user's operation.

[0019] In a possible design, the prediction module is also used to infer the current behavior of the user in the current state to obtain multiple predicted behaviors that may occur in the next step; in any traversal process, the predicted behavior of this traversal is selected from the multiple predicted behaviors, and the risk value of the predicted behavior is updated according to the simulated behavior chain of the predicted behavior of this traversal; the simulated behavior chain is obtained by multiple expansions of the subsequent behaviors of the predicted behavior of this traversal; after the simulated behavior chains of multiple predicted behaviors meet the traversal requirements, the first risk value of each predicted behavior is obtained.

[0020] In one possible design, the prediction module is also used to obtain the simulated behavior chain in the following manner: taking the current behavior in the current state as the root node; at each expansion, selecting the first behavior sub-node with the largest risk value starting from the root node, and judging whether the first behavior sub-node is completely expanded; the risk value is determined by a first parameter value and a second parameter value; the second parameter value is positively correlated with the risk probability of the behavior sub-node, and negatively correlated with the number of times the behavior sub-node is visited; the risk probability is obtained through a strategy neural network; the first behavior sub-node is any one of a plurality of predicted behaviors; if the first behavior sub-node is not completely expanded, the subsequent plurality of second behavior sub-nodes of the first behavior sub-node are simulated through the strategy neural network to obtain a simulated behavior chain of the predicted behavior.

[0021] In one possible design, the prediction module is further configured to obtain, for each second behavior sub-node in the simulated behavior chain of the predicted behavior, a simulation result of the second behavior sub-node using a fast strategy neural network, and obtain a third parameter value of the second behavior sub-node using a value neural network; the simulation result indicates whether the second behavior sub-node has an abnormal operation;

[0022] updating the first parameter value of the first behavior sub-node of the simulated behavior chain according to the simulation result of each second behavior sub-node and the third parameter value of each second behavior sub-node;

[0023] The risk value of the predicted behavior is updated according to the first parameter value of the simulated behavior chain.

[0024] In one possible design, the device also includes a training module for training the value neural network by the following method: for each state and behavior in the training data, the strategy neural network executes L steps of behavior after the state and behavior; each step in the L steps of behavior selects the behavior with the highest risk probability to execute; a behavior is randomly executed in the L+1 step; the enhanced strategy neural network executes M steps of behavior after the L+1 step to obtain a simulation result; each step in the M steps of behavior selects the behavior with the highest risk probability to execute; and the simulation result is used as the label of the training data to train the value neural network.

[0025] In one possible design, the user's current state includes the current page where the user is staying and the content of the current page; the device also includes a preprocessing module for converting the user's current behavior in the current state into a feature vector before predicting multiple predicted behaviors that the user may perform next based on the user's current behavior in the current state; wherein the content of the current page is converted into a feature vector through word embedding.

[0026] In one possible design, the determination module is also used to accumulate the first risk value of each suspicious behavior during the user operation to obtain a second risk value; the first risk value of the suspicious behavior is the first risk value of the corresponding predicted behavior; if the second risk value is greater than a second risk threshold, the user's operation is determined to be abnormal.

[0027] In a third aspect, an embodiment of the present application further provides a computing device, including:

[0028] a memory for storing program instructions;

[0029] The processor is used to call the program instructions stored in the memory and execute the method described in any possible design of the first aspect according to the obtained program instructions.

[0030] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing computer-readable instructions. When a computer reads and executes the computer-readable instructions, the method described in any possible design of the first aspect above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0032] Figure 1A schematic diagram of a system architecture applicable to an embodiment of the present application;

[0033] Figure 2 A flowchart of a method for analyzing abnormal operation provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a process for predicting multiple predicted behaviors that a user may take next, and a first risk value for each predicted behavior, provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of a process for updating the risk value of a predicted behavior provided in an embodiment of the present application;

[0036] Figure 5 A schematic diagram of the structure of an abnormal operation analysis device provided in an embodiment of the present application;

[0037] Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] To make the objectives, technical solutions, and advantages of this application more clear, this application will be further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0039] In the embodiments of the present application, "a plurality" refers to two or more. Terms such as "first" and "second" are used only for the purpose of distinguishing descriptions and should not be understood as indicating or implying relative importance or order.

[0040] Figure 1 A schematic diagram of a system architecture applicable to the embodiment of the present application is shown in FIG. Figure 1 As shown, the system architecture includes at least a user terminal device 101, an operation analysis system 102, and an operation and maintenance terminal device 103. The user terminal device 101 and the operation analysis system 102, and the operation analysis system 102 and the operation and maintenance terminal device 103 can be connected directly or indirectly via wired or wireless communication, which is not specifically limited in this application.

[0041] There can be one or more user terminal devices 101. User terminal devices 101 can be used by internal personnel of a company or enterprise for work purposes. User terminal devices 101 can also be installed with certain applications or provide certain services. User terminal devices 101 collect every current user operation and send it to the operation analysis system, which then performs inference analysis on each user operation. User terminal devices 101 can be smartphones, tablet computers, laptop computers, desktop computers, smart home appliances, smart in-vehicle devices, etc.

[0042] The operation analysis system 102 obtains each user's operation on the user terminal device 101, performs reasoning analysis on each user's operation, and then sends the reasoning analysis results and alarm information of abnormal operations to the operation and maintenance terminal device 103. The alarm analysis system 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0043] There can be one or more operation and maintenance terminal devices 103, which are used to receive the reasoning and analysis results and abnormal operation warning information provided by the operation analysis system 102. The operation and maintenance terminal device 103 can be a smartphone, tablet computer, laptop computer, desktop computer, etc. The reasoning and analysis results stored in the operation and maintenance terminal device 103 allow operation and maintenance personnel to promptly respond to abnormal operation warning information and take timely defensive measures or intercept information before attackers can access more information or before file streams are transferred.

[0044] It should be noted that the above Figure 1 The system architecture shown is only an example and is not specifically limited in the embodiments of the present application.

[0045] The technical solution of the present application is applicable to scenarios where historical behavior chains can be accurately recorded, such as analyzing mailbox systems, cloud behaviors, analyzing user behaviors on Web sites, analyzing user command operations on Linux hosts, etc. The technical solution of the present application is illustrated below using an enterprise's internal mailbox system as an example.

[0046] Figure 2 The flowchart of a method for analyzing an operational anomaly provided in an embodiment of the present application is shown as an example. Figure 2 As shown, the method includes the following steps:

[0047] Step 201: For each behavior during the user operation process, based on the current behavior of the user in the current state, predict multiple predicted behaviors that the user may perform next, and the first risk value of each predicted behavior.

[0048] In the embodiments of the present application, the user operation process refers to starting from the behavior after the user logs in to the mailbox and ending when the user logs out of the mailbox or there is no operation within N minutes. The login information and operation content records of each user can be obtained through the mail log server, and log preprocessing is performed to convert the current behavior of the user in the current state into a feature vector.

[0049] The current state of the user includes the current page where the user stays and the content of the current page. For the mailbox system, the current page where the user stays may specifically include the category where the user is located, as well as the major state category and minor state category under the category. Among them, the category may be the inbox, outbox, draft box, folder, etc. For the inbox, the major state category may refer to the page number of the inbox, and the minor state category may refer to the number of the email on the page number of the inbox. The content of the current page is the content of the email (including the email body and email attachments), and the content of the current page can be converted into a feature vector through word embedding. Exemplarily, the format of the feature vector corresponding to the current state of the user can be defined as (inbox, outbox, draft box, folder, major state category, minor state category, details, attachments). For example, if the user stays on the first email on the first page of the inbox, it is represented as (1, 0, 0, 0, 1, 1, 0.3, 0.2). Among them, 0.3 refers to the word embedding of the email body, and 0.2 refers to the word embedding of the attachment name / attachment content. The word embedding can be obtained through methods such as TF-IDF or one-hot encoding, and the present application does not specifically limit the number of bits of the feature vector of the word embedding.

[0050] The current behavior of the user includes the behavior and the content of the behavior. For the mailbox system, the behavior may be writing a letter, searching, sending, viewing, turning pages, etc. For the behavior of writing a letter, the content of the behavior is the content of the letter; for the search behavior, the content of the behavior is the content of the search. The content of the behavior can be converted into a feature vector through word embedding. Exemplarily, the format of the feature vector corresponding to the current behavior of the user can be defined as (writing a letter, searching, sending, viewing, turning pages, content). For example, if the user searches for "secret", it is represented as (0, 1, 0, 0, 0, 0.4), where 0.4 refers to the word embedding of "secret".

[0051] Then, the feature vector corresponding to the current behavior of the user in the current state is the combination of the feature vector of the current state and the feature vector of the current behavior. For example, when the user searches for "secret" in the first email on the first page of the inbox, it can be represented as (1, 0, 0, 0, 1, 1, 0.3, 0.2, 0, 1, 0, 0, 0, 0.4).

[0052] In step 201, the implementation method for predicting multiple possible next behaviors of the user and the first risk value of each predicted behavior based on the current behavior of the user in the current state is as Figure 3 shown, and specifically includes the following steps:

[0053] Step 301: Reason about the current behavior of the user in the current state to obtain multiple possible next behaviors.

[0054] For example, if the current behavior of the user in the current state is that the user searches for "secret" in the first email on the first page of the inbox, reasoning about this behavior obtains multiple possible next behaviors, such as searching for "username", "password", "system", "do not" etc. in the state of the page where "secret" is searched.

[0055] Step 302: In any traversal process, select the predicted behavior for this traversal from multiple predicted behaviors, and update the risk value of the predicted behavior according to the simulated behavior chain of the predicted behavior for this traversal.

[0056] In step 302, the simulated behavior chain is obtained by expanding the subsequent behaviors of the predicted behavior for this traversal multiple times, that is, the simulated behavior chain includes multiple steps of simulated behaviors subsequent to the predicted behavior. Exemplarily, the simulated behavior chain can be obtained in the following way: Use the current behavior in the current state as the root node. In each expansion, start from the root node and select the first behavior child node with the largest risk value, and determine whether the first behavior child node is fully expanded, where the first behavior child node is any one of the multiple predicted behaviors. If the first behavior child node is not fully expanded, then use the policy neural network to simulate the subsequent multiple second behavior child nodes of the first behavior child node to obtain the simulated behavior chain of the predicted behavior.

[0057] The risk value represents the degree of security threat of this behavior. The risk value r can be determined by the first parameter value Q(s, a) and the second parameter value U(P), that is

[0058] r = Q(s, a) + U(P)

[0059] where the first parameter value Q(s, a) is the simulated risk of executing the current behavior a in the current state s of the user, and the initial value is 0;

[0060] That is, the second parameter value U(P) is positively correlated with the risk probability P(s, a) of the behavioral child node and negatively correlated with the number of times N(s, a) the behavioral child node is visited. The risk probability P(s, a) is obtained through the policy neural network. In the actual calculation process, Among them, the value of the coefficient b1 can be set according to experience.

[0061] It can be understood that if the risk value r of a certain behavioral child node A is always the largest, the behavioral child node is continuously expanded, then the number of times N(s, a) the behavioral child node is visited will increase more and more, and the second parameter value U(P) will become smaller and smaller. Subsequently, the risk value r will also become smaller. Then, after multiple expansions, there will be other behavioral child nodes B with larger risk values. At this time, the child node B should be expanded, so as to discover new branches and avoid excessive errors caused by the algorithm stopping expansion too quickly as the number of expansion times increases.

[0062] Taking the example that the user searches for "secret" in the first email on the first page of the inbox as the root node, the multiple predicted behaviors that may occur next: searching for "username", "password", "system", and "do not" are used as the first behavioral child nodes respectively. Suppose that at the first expansion, the risk value of the child node of searching for "username" is the largest, and the child node of searching for "username" has not been fully expanded. Then, through the policy neural network, the subsequent multiple second behavioral child nodes of the child node of searching for "username" are simulated to obtain the simulated behavior chain of the child node of searching for "username". Suppose that at the nth expansion, the risk value of the child node of searching for "system" is the largest, and the child node of searching for "system" has not been fully expanded. Then, through the policy neural network, the subsequent multiple second behavioral child nodes of the child node of searching for "system" are simulated to obtain the simulated behavior chain of the child node of searching for "system".

[0063] In step 302, the implementation method for updating the risk value of the predicted behavior is as Figure 4 shown, specifically including the following steps:

[0064] Step 401: For each second behavioral child node in the simulated behavior chain of the predicted behavior, obtain the simulation result of the second behavioral child node through the fast policy neural network and obtain the third parameter value of the second behavioral child node through the value neural network. Among them, the simulation result indicates whether the second behavioral child node operates abnormally.

[0065] In step 401, through the fast policy neural network, the subsequent multiple behaviors that may be executed by the second behavior are quickly simulated until the simulation reaches the end of this mailbox operation, and the simulation result Z of whether this operation is an abnormal operation is obtained L , where this operation is marked as abnormal as Z L=1, indicating that this mailbox operation may have the risk of information leakage; this operation is normal and marked as Z L = 0. The value neural network is used to evaluate the second child node to obtain the third parameter value v(S L ).

[0066] Step 402: Update the first parameter value of the first behavior sub-node of the simulated behavior chain according to the simulation result of each second behavior sub-node and the third parameter value of each second behavior sub-node.

[0067] In step 402, the simulation result Z of the child node in the second row can be used first. L and the second line is the third parameter value v(S L ), calculate the comprehensive value v all (S L ),Right now

[0068] v all (S L )=b2v(S L )+b3Z L

[0069] The values of coefficients b2 and b3 can be set based on experience, and b2 and b3 can be the same or different. For example, v all (S L )=0.5v(S L )+0.5Z L .

[0070] Then through v all (S L ) Update the first parameter value Q(s, a) of the first behavior child node of the simulation behavior chain, that is,

[0071]

[0072] in,

[0073] Step 403: Update the risk value of the predicted behavior according to the first parameter value of the simulated behavior chain.

[0074] Since the risk value is determined by the first parameter value Q(s, a) and the second parameter value U(P), after the first parameter value Q(s, a) of the predicted behavior is updated, the risk value of the predicted behavior needs to be further updated.

[0075] Step 303: After the simulated behavior chains of the multiple predicted behaviors meet the traversal requirements, a first risk value of each predicted behavior is obtained.

[0076] In step 303, the traversal requirements can be set according to actual needs. For example, it can be set to end after the total number of traversals reaches a set number, or it can be set to end after the number of traversals for each predicted behavior reaches a set number.

[0077] Step 202: If the actual behavior of the user's next step matches one of the predicted behaviors and the first risk value of the predicted behavior is greater than the first risk threshold, then mark the actual behavior as a suspicious behavior.

[0078] In step 202, the first risk threshold can be set based on experience. After obtaining multiple predicted behaviors that may occur for the user's next step and the first risk of each predicted behavior through step 201, the actual behavior of the user's next step is matched with the multiple predicted behaviors that may occur. There may be three matching situations. Situation 1: The actual behavior of the user's next step matches one of the predicted behaviors, and the first risk value of the predicted behavior is greater than the first risk threshold. For this situation, mark the actual behavior of the user's next step as a suspicious behavior and record the first risk value of this suspicious behavior. Among them, the first risk value of the suspicious behavior is the first risk value of the corresponding predicted behavior. Situation 2: The actual behavior of the user's next step matches one of the predicted behaviors, but the first risk value of the predicted behavior is less than the first risk threshold. Situation 3: The actual behavior of the user's next step does not match one of the predicted behaviors. For situations 2 and 3, it can be considered that the actual behavior of the user's next step is not a suspicious behavior and this behavior does not need special attention.

[0079] For example, the current behavior of the user in the current state is that the user searches for "secret" in the first email on the first page of the inbox, and obtains multiple predicted behaviors that may occur for the next step and the first risk value of each predicted behavior. For example, the first risk value for searching for "username" is 0.5, the first risk value for searching for "password" is 0.6, the first risk value for searching for "system" is 0.3, and the first risk value for searching for "do not" is 0.25. Assume the first risk threshold is 0.4. If the actual behavior of the user's next step is to search for "username", which matches the above predicted behavior of searching for "username", and the first risk value of searching for "username" (0.5) is greater than the first risk threshold (0.4), then mark the search for "password" as a suspicious behavior. If the actual behavior of the user's next step is to search for "do not", which matches the above predicted behavior of searching for "do not", but the first risk value of searching for "do not" (0.25) is less than the first risk threshold (0.4), then this behavior does not need to be marked. If the actual behavior of the user's next step is to exit the email page and does not match the above predicted behaviors, then this behavior does not need to be marked.

[0080] Step 203: When there are multiple suspicious behaviors during the user's operation process, determine that the user's operation is abnormal.

[0081] For example, the first risk value of each suspicious behavior during the user operation can be accumulated to obtain a second risk value. If the second risk value is greater than a second risk threshold, the user operation is determined to be abnormal. The second risk threshold can be set based on experience.

[0082] Example 1: Assume that the second risk threshold is set to 1. During the user operation, the only suspicious behavior recorded is the search for "secret" (the first risk value is 0.55). Then the second risk value is 0.55. At this time, the second risk value is less than the second risk threshold, and the alarm triggering condition for abnormal operation is temporarily not met.

[0083] Example 2: Assume that the second risk threshold is set to 1. During the user operation, the search for "password" (first risk value is 0.55) and the search for "user name" (first risk value is 0.5) have been recorded as suspicious behaviors. The first risk value of searching for "password" and the first risk value of searching for "user name" are added together to obtain a second risk value of 1.55. At this time, the second risk value is greater than the second risk threshold, then the user's operation is determined to be abnormal, and an abnormal operation alarm is issued.

[0084] Example 3: Assume that the second risk threshold is set to 1.2. During the user's operation, the user continuously checks emails containing the word "Confidential" in the content. For example, the user searches for "Confidential" in the first email on the first page of the inbox, searches for "Confidential" in the second email on the first page of the inbox, and searches for "Confidential" in the third email on the first page of the inbox. The above three searches for "Confidential" are all recorded as suspicious behaviors, and the corresponding first risk values are 0.55, 0.55, and 0.6, respectively. The first risk values of the three searches for "Confidential" are accumulated to obtain a second risk value of 1.7. At this time, the second risk value is greater than the second risk threshold, so the user's operation is determined to be abnormal, and an abnormal operation alarm is issued.

[0085] Each training sample of the policy neural network, fast policy neural network, and enhanced policy neural network used in the embodiments of the present application is a feature vector (s, a) corresponding to the state s and the behavior a. The label corresponding to each training sample is whether a data security incident occurs, 1 if a data security incident occurs, and 0 if no data security incident occurs. Among them, the number of training samples of the fast policy neural network is the same as that of the policy neural network, but the number of nodes of the fast policy neural network is less than that of the policy neural network, and the structure of the neural network is simpler, so the simulation speed is faster. The network structure of the enhanced policy neural network is consistent with that of the policy neural network, but the training samples of the enhanced policy neural network have been data enhanced to expand the training set.

[0086] Furthermore, the value neural network can be trained as follows:

[0087] Step a: For each state and action in the training data, the policy neural network executes the L steps of behavior following the state and action. In each of the L steps, the action with the highest risk probability is selected for execution.

[0088] Step b: In step L+1, a random action is executed.

[0089] Step c: Execute the M steps of behavior after the L+1 step through the enhanced policy neural network to obtain the simulation results. In each of the M steps of behavior, the behavior with the highest risk probability is selected for execution.

[0090] Step d: Use the simulation results as labels of training data to train the value neural network.

[0091] This application provides a method for analyzing operational anomalies. It simulates and deduces each action during a user's operation to predict the user's next possible action and the first risk value of each predicted action in advance. This allows the user to immediately match the risk value of the actual action after the user's next action occurs, and determine whether the actual action is suspicious based on the first risk threshold. If suspicious actions occur multiple times during a user's operation, the user's operation is determined to be abnormal and an alarm is issued. This allows the attacker to detect and intercept the attack before more private information is accessed or before any file transfer operation is performed.

[0092] Based on the same technical concept, Figure 5 An example of an analysis device for abnormal operation provided by an embodiment of the present application is shown. Figure 5 As shown, the apparatus 500 includes:

[0093] Prediction module 501, configured to predict, for each action during a user operation, multiple predicted actions that the user may take next based on the current action in the user's current state, and a first risk value for each predicted action;

[0094] Processing module 502, configured to mark the actual behavior as suspicious if the user's next actual behavior matches one of the predicted behaviors and the first risk value of the predicted behavior is greater than a first risk threshold;

[0095] The determination module 503 is configured to determine that the user's operation is abnormal when multiple suspicious behaviors occur during the user's operation.

[0096] In one possible design, the prediction module 501 is also used to infer the current behavior of the user in the current state to obtain multiple predicted behaviors that may occur in the next step; in any traversal process, the predicted behavior of this traversal is selected from the multiple predicted behaviors, and the risk value of the predicted behavior is updated according to the simulated behavior chain of the predicted behavior of this traversal; the simulated behavior chain is obtained by multiple expansions of the subsequent behaviors of the predicted behavior of this traversal; after the simulated behavior chains of multiple predicted behaviors meet the traversal requirements, the first risk value of each predicted behavior is obtained.

[0097] In one possible design, the prediction module 501 is also used to obtain the simulated behavior chain in the following manner: taking the current behavior in the current state as the root node; at each expansion, selecting the first behavior sub-node with the largest risk value starting from the root node, and judging whether the first behavior sub-node is fully expanded; the risk value is determined by a first parameter value and a second parameter value; the second parameter value is positively correlated with the risk probability of the behavior sub-node, and negatively correlated with the number of times the behavior sub-node is visited; the risk probability is obtained through a strategy neural network; the first behavior sub-node is any one of a plurality of predicted behaviors; if the first behavior sub-node is not fully expanded, the subsequent plurality of second behavior sub-nodes of the first behavior sub-node are simulated through the strategy neural network to obtain a simulated behavior chain of the predicted behavior.

[0098] In one possible design, the prediction module 501 is also used to obtain a simulation result of each second behavior sub-node in the simulated behavior chain of the predicted behavior through a fast strategy neural network, and obtain a third parameter value of the second behavior sub-node through a value neural network; the simulation result indicates whether the second behavior sub-node operates abnormally; based on the simulation result of each second behavior sub-node and the third parameter value of each second behavior sub-node, the first parameter value of the first behavior sub-node of the simulated behavior chain is updated; and the risk value of the predicted behavior is updated based on the first parameter value of the simulated behavior chain.

[0099] In one possible design, the device also includes a training module 504, which is used to train the value neural network by the following method: for each state and behavior in the training data, the strategy neural network executes L steps of behavior after the state and behavior; each step in the L steps of behavior selects the behavior with the highest risk probability to execute; a behavior is randomly executed in the L+1 step; the enhanced strategy neural network executes M steps of behavior after the L+1 step to obtain a simulation result; each step in the M steps of behavior selects the behavior with the highest risk probability to execute; and the simulation result is used as the label of the training data to train the value neural network.

[0100] In one possible design, the user's current state includes the current page where the user is staying and the content of the current page; the device also includes a preprocessing module 501, which is used to convert the current behavior of the user in the current state into a feature vector before predicting multiple predicted behaviors that the user may perform next based on the current behavior of the user in the current state; wherein the content of the current page is converted into a feature vector through word embedding.

[0101] In one possible design, the determination module 502 is also used to accumulate the first risk value of each suspicious behavior during the user operation to obtain a second risk value; the first risk value of the suspicious behavior is the first risk value of the corresponding predicted behavior; if the second risk value is greater than the second risk threshold, the user's operation is determined to be abnormal.

[0102] Based on the same technical concept, the embodiment of the present application provides a computing device, such as Figure 6 As shown, it includes at least one processor 601 and a memory 602 connected to the at least one processor. The specific connection medium between the processor 601 and the memory 602 is not limited in the embodiment of the present application. Figure 6 For example, the processor 601 and the memory 602 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, and the like.

[0103] In an embodiment of the present application, the memory 602 stores instructions that can be executed by at least one processor 601. The at least one processor 601 can execute the above-mentioned data synchronization method between data centers by executing the instructions stored in the memory 602.

[0104] Among them, the processor 601 is the control center of the computing device, which can use various interfaces and lines to connect various parts of the computer device, and perform resource settings by running or executing instructions stored in the memory 602 and calling data stored in the memory 602.

[0105] Optionally, processor 601 may include one or more processing units. Processor 601 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, and the modem processor primarily processes wireless communications. It is understood that the modem processor may not be integrated into processor 601. In some embodiments, processor 601 and memory 602 may be implemented on the same chip. In some embodiments, they may also be implemented on separate chips.

[0106] The processor 601 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0107] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 602 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 602 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0108] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer-executable program. The computer-executable program is used to enable a computer to execute the data synchronization method between data centers listed in any of the above methods.

[0109] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0110] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0111] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0113] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0114] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for analyzing abnormal operation, characterized in that: The method comprises: For each behavior during the user operation process, multiple predicted behaviors that may occur in the next step of the user are predicted based on the current behavior of the user in the current state, as well as the first risk value of each predicted behavior, including: reasoning about the current behavior of the user in the current state to obtain multiple predicted behaviors that may occur in the next step; in any traversal process, the predicted behavior of this traversal is selected from the multiple predicted behaviors, and the risk value of the predicted behavior is updated according to the simulated behavior chain of the predicted behavior of this traversal; the simulated behavior chain is obtained by multiple expansions of subsequent behaviors of the predicted behavior of this traversal; after the simulated behavior chain of multiple predicted behaviors meets the traversal requirements, the first risk value of each predicted behavior is obtained; the simulated behavior chain is obtained in the following manner: with the current The current behavior in the previous state is used as the root node; each time the first behavior subnode with the largest risk value is selected from the root node, and it is determined whether the first behavior subnode is fully expanded; the risk value is determined by a first parameter value and a second parameter value; the first parameter value is the simulated risk of the user performing the current behavior in the current state; the second parameter value is positively correlated with the risk probability of the behavior subnode and negatively correlated with the number of times the behavior subnode is visited; the risk probability is obtained through a policy neural network; the first behavior subnode is any one of a plurality of predicted behaviors; if the first behavior subnode is not fully expanded, the subsequent plurality of second behavior subnodes of the first behavior subnode are simulated by the policy neural network to obtain a simulated behavior chain of the predicted behavior; If the user's next actual behavior matches one of the predicted behaviors, and the first risk value of the predicted behavior is greater than a first risk threshold, the actual behavior is marked as suspicious behavior; If multiple suspicious behaviors occur during the user's operation, it is determined that the user's operation is abnormal.

2. The method according to claim 1, characterized in that The updating of the risk value of the predicted behavior includes: For each second behavior sub-node in the simulated behavior chain of the predicted behavior, a simulation result of the second behavior sub-node is obtained through a fast strategy neural network, and a third parameter value of the second behavior sub-node is obtained through a value neural network; the simulation result indicates whether the second behavior sub-node has an abnormal operation; updating the first parameter value of the first behavior sub-node of the simulated behavior chain according to the simulation result of each second behavior sub-node and the third parameter value of each second behavior sub-node; The risk value of the predicted behavior is updated according to the first parameter value of the simulated behavior chain.

3. The method according to claim 2, characterized in that The method further comprises: The value neural network is trained by the following method: For each state and behavior in the training data, the strategy neural network executes L steps of behavior after the state and behavior; each step in the L steps of behavior selects the behavior with the highest risk probability to execute; In the L+1 step, a random action is executed; The enhanced policy neural network executes M steps of behavior after the L+1 step to obtain a simulation result; each step of the M steps of behavior selects the behavior with the highest risk probability to execute; The simulation results are used as labels of the training data to train the value neural network.

4. The method according to claim 1, wherein The user's current status includes the current page where the user is staying and the content of the current page; Before predicting multiple predicted behaviors that the user may take next step based on the current behavior of the user in the current state, the method further includes: The current behavior of the user in the current state is converted into a feature vector; wherein the content of the current page is converted into a feature vector through word embedding.

5. The method according to claim 1, wherein When multiple suspicious behaviors occur during the user's operation, determining that the user's operation is abnormal includes: Accumulating the first risk value of each suspicious behavior during the user operation to obtain a second risk value; the first risk value of the suspicious behavior is the first risk value of the corresponding predicted behavior; If the second risk value is greater than a second risk threshold, it is determined that the user's operation is abnormal.

6. An analysis device for abnormal operation, characterized in that: include: A prediction module is configured to predict, for each behavior during a user operation, a plurality of predicted behaviors that the user may perform next based on the current behavior of the user in the current state, and a first risk value of each predicted behavior; a processing module, configured to mark the actual behavior as suspicious if the user's next actual behavior matches one of the predicted behaviors and the first risk value of the predicted behavior is greater than a first risk threshold; The prediction module is further configured to infer the current behavior of the user in the current state to obtain multiple predicted behaviors that may occur next; During any traversal process, a predicted behavior for this traversal is selected from the multiple predicted behaviors, and the risk value of the predicted behavior is updated based on a simulated behavior chain of the predicted behavior for this traversal; the simulated behavior chain is obtained by multiple expansions of subsequent behaviors of the predicted behavior for this traversal; after the simulated behavior chains of the multiple predicted behaviors meet the traversal requirements, a first risk value of each predicted behavior is obtained; The prediction module is further configured to obtain the simulated behavior chain by taking the current behavior in the current state as a root node; selecting the first behavior subnode with the largest risk value from the root node each time the behavior chain is expanded, and determining whether the first behavior subnode is fully expanded. The risk value is determined by a first parameter value and a second parameter value; the first parameter value is the simulated risk of executing the current behavior in the current state of the user; the second parameter value is positively correlated with the risk probability of the behavior sub-node and negatively correlated with the number of times the behavior sub-node is visited; the risk probability is obtained through a policy neural network; the first behavior sub-node is any one of multiple predicted behaviors; if the first behavior sub-node is not fully expanded, the policy neural network is used to simulate multiple subsequent second behavior sub-nodes of the first behavior sub-node to obtain a simulated behavior chain of the predicted behavior; The determination module is used to determine that the user's operation is abnormal when multiple suspicious behaviors occur during the user's operation.

7. A computing device, characterized in that include: a memory for storing program instructions; A processor is configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 5 according to the obtained program instructions.

8. A computer-readable storage medium, characterized in that The method comprises computer-readable instructions, which, when read and executed by a computer, enable the method according to any one of claims 1 to 5 to be implemented.

Images