Data processing method and device, equipment and storage medium

Abstract

The invention provides a data processing method and device, equipment and a storage medium, and relates to the technical field of computers. The data processing method comprises the steps of determining first risk category distribution of a target environment and sensitivity of each risk category to abnormal interaction behaviors based on first interaction data of the target environment and all interaction objects; determining a time window value meeting a target condition based on the sensitivity; determining second risk category distribution of the target interaction object based on second interaction data of the target interaction object with the target environment in the recent time window; determining the target similarity of the first risk category distribution and the second risk category distribution based on the sensitivity and the difference between the proportions of the risk categories in the first risk category distribution and the second risk category distribution; and if the target similarity meets a second threshold, determining that the second interaction data in the recent time window is abnormal.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, device, equipment and storage medium. Background Art

[0002] Currently, there is no clear industry standard for risk detection of abnormal data transfer, abnormal network traffic, and other similar network risk behaviors. The commonly used solution is to set a fixed threshold and determine whether the relevant data meets the fixed threshold. However, the above solution cannot dynamically determine a reasonable fixed threshold based on the company's situation, lacks flexibility, and cannot determine a relatively stable time window for risk detection, resulting in low accuracy of the detection results. In addition, the fixed threshold may cause an alarm storm, thereby covering up the real abnormal risk. Summary of the invention

[0003] The present disclosure provides a data processing method, device, equipment and storage medium to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present disclosure, a data processing method is provided, including: determining a first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior based on first interaction data between a target environment and all interaction objects; determining a time window value that satisfies a target condition based on the sensitivity, wherein the target condition characterizes that the similarity of the risk category distribution of the first interaction data in every two time windows satisfies a first threshold; determining a second risk category distribution of the target interaction object based on second interaction data between the target interaction object and the target environment in a recent time window; determining a target similarity between the first risk category distribution and the second risk category distribution based on the sensitivity and the difference between the proportion of the risk category in the first risk category distribution and the second risk category distribution; and determining that an abnormality exists in the second interaction data in the recent time window if the target similarity satisfies a second threshold.

[0005] In one possible implementation, determining the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior includes: inputting the first interaction data into a risk classification model, outputting the first risk category distribution of the target environment and the sensitivity of each risk category through the risk classification model, the risk classification model being trained based on training interaction data and risk categories corresponding to the training interaction data; wherein the risk categories corresponding to the training interaction data are obtained by clustering the training interaction data based on weight values ​​of features in the training interaction data and the difference between feature values ​​of every two training interaction data.

[0006] In one possible implementation, determining the time window value that meets the target condition based on the sensitivity includes: under the initial value of the time window, determining the risk category distribution of each time window based on the first interaction data in each time window; determining the similarity of the risk category distribution of every two time windows based on the difference in the proportion of each risk category in the risk category distribution of every two time windows and the sensitivity; if the similarities all meet the first threshold, determining the initial value as the time window value.

[0007] In one possible implementation, a data processing method also includes: if there is a similarity that does not meet the first threshold, adjusting the initial value, and determining the similarity of the risk category distribution of each two time windows under the adjusted initial value until the similarities all meet the first threshold.

[0008] In one possible implementation, determining the target similarity between the first risk category distribution and the second risk category distribution includes: determining the proportion of unclassified risk categories in the second risk category distribution, the unclassified risk category proportion being the proportion of risk categories in the second risk category distribution that do not belong to the first risk category distribution; determining the target similarity based on the sensitivity, the difference between the proportions of the risk category in the first risk category distribution and the second risk category distribution, and the unclassified risk category proportion.

[0009] In one possible implementation, a data processing method also includes: determining the historical similarity between the risk category distribution of the target interaction object in the historical time window and the first risk category distribution; detecting the abnormal similarity in the similarity sequence composed of the target similarity and the historical similarity based on an anomaly detection algorithm, and determining that there is an anomaly in the second interaction data within the time window corresponding to the abnormal similarity.

[0010] In one possible implementation, a data processing method also includes at least one of the following: if there is an abnormality in the second interaction data, an alarm message is generated, and the alarm message includes the cause of the abnormality of the second interaction data; if there is an abnormality in the second interaction data, the interaction behavior between the target interaction object and the target environment is suspended.

[0011] According to a second aspect of the present disclosure, a data processing device is provided, including: a category determination module, for determining, based on first interaction data between the target environment and all interaction objects, a first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior; a window determination module, for determining, based on the sensitivity, a time window value that satisfies a target condition, wherein the target condition characterizes that the similarity of the risk category distribution of the first interaction data in every two time windows satisfies a first threshold; the category determination module is also used to determine, based on second interaction data between the target interaction object and the target environment in a recent time window, a second risk category distribution of the target interaction object; a similarity determination module, for determining, based on the sensitivity and the difference between the proportion of the risk category in the first risk category distribution and the second risk category distribution, a target similarity of the first risk category distribution and the second risk category distribution; an anomaly determination module, for determining, if the target similarity satisfies the second threshold, that the second interaction data in the recent time window is abnormal.

[0012] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0016] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0019] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0020] Figure 1A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 1 ;

[0021] Figure 2 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 1 ;in,

[0022] Figure 2 A shows a schematic diagram of risk category distribution of a data processing method according to an embodiment of the present disclosure;

[0023] Figure 2 B shows a sensitivity distribution diagram of a data processing method according to an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 2 ;

[0025] Figure 4 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 2 ;

[0026] Figure 5 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 3 ;

[0027] Figure 6 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 3 ;

[0028] Figure 7 A schematic diagram of the structure of a data processing device according to an embodiment of the present disclosure is shown;

[0029] Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0030] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0031] Figure 1 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 1 ,like Figure 1 As shown, a data processing method includes:

[0032] Step S101: determining a first risk category distribution of the target environment and a sensitivity of each risk category to abnormal interaction behavior based on first interaction data between the target environment and all interaction objects.

[0033] In this embodiment, the target environment is a work environment where network risk behaviors may exist, such as positions, systems, platforms, and work types, and the interactive objects are objects that interact with the target environment, such as employees at positions, personnel performing system operations, and users performing data interactions with the platform, etc. Based on the first interaction data between the target environment and all its interactive objects, the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interactive behaviors are determined, wherein the first risk category distribution includes the risk categories that may exist in the target environment and the proportion of each risk category. The lower the sensitivity of the risk category, the fewer abnormal interactive behaviors there are, and the corresponding risk tolerance is lower, that is, it cannot tolerate large amounts of data or traffic transfer. In one example, the first interaction data can be clustered to obtain the risk categories that may exist in the target environment and the proportion of each risk category, and the median of each risk category is determined as the sensitivity of the risk category.

[0034] Figure 2 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 1 ,in, Figure 2 A shows a schematic diagram of risk category distribution of a data processing method according to an embodiment of the present disclosure, Figure 2 B shows a sensitivity distribution diagram of a data processing method according to an embodiment of the present disclosure. Figure 2 As shown in A, the first risk category distribution of a certain position may include risk categories C1, C2, C3, C4, C5 and C6, and their corresponding proportions are 13.0%, 6.0%, 36.0%, 29.0%, 9.0% and 7.0% respectively; Figure 2 As shown in B, Figure 2 The corresponding sensitivities of risk categories C1, C2, C3, C4, C5 and C6 in A are S1, S2, S3, S4, S5 and S6, and the corresponding proportions of each sensitivity are 31.0%, 24.0%, 19.0%, 15.0%, 7.0% and 4.0%, respectively.

[0035] Step S102: Determine a time window value that meets the target condition based on the sensitivity.

[0036] In this embodiment, it is also necessary to determine the time window value for detecting the interaction data, and the time window value needs to meet the target condition, which indicates that the similarity of the risk category distribution of the first interaction data in every two time windows meets the first threshold, that is, the risk category distribution of the first interaction data in every two time windows is relatively stable. In one example, the time window value can be determined starting from 1, that is, the first interaction data generated every day is determined as data in the same time window, thereby dividing the first interaction data into multiple time windows, and based on the sensitivity of each risk category calculated in step S101, the similarity of the risk category distribution of the first interaction data in every two time windows is determined. If the similarity meets the first threshold, the time window value is determined to be 1.

[0037] Step S103: determining a second risk category distribution of the target interactive object based on second interaction data between the target interactive object and the target environment in a recent time window.

[0038] In this embodiment, the target interaction object is an interaction object for which it is necessary to determine whether there is an interaction data anomaly. The second interaction data between the target interaction object and the target environment in a recent time window can be selected, and the second risk category distribution of the target interaction object can be determined based on the second interaction data. The process of determining the second risk category distribution is similar to the process of determining the first risk category distribution, and will not be repeated here.

[0039] Step S104: determining the target similarity between the first risk category distribution and the second risk category distribution based on the difference in sensitivity and the proportion of the risk category in the first risk category distribution and the second risk category distribution.

[0040] In this embodiment, the first risk category distribution of the target environment is the average risk category distribution of the target environment over a period of time, and the second risk category distribution is the risk category distribution of the target interactive object in a recent time window. If the second risk category distribution deviates far from the first risk category distribution, it proves that there is an anomaly in the data of the target interactive object interacting with the target environment in the most recent time window. Therefore, it is necessary to determine the target similarity of the first risk category distribution and the second risk category distribution to characterize the degree to which the second risk category distribution deviates from the first risk category.

[0041] In this embodiment, the target similarity between the first risk category distribution and the second risk category distribution may be determined based on the difference between the sensitivity calculated in step S101 and the proportion of the risk category in the first risk category distribution and the second risk category distribution. The target similarity may be determined based on the following formula 1:

[0042] Similarity=1-∑(Si×|Pi-Oi|) Formula 1

[0043] Among them, Similarity is the target similarity, Si is the sensitivity of each risk category i, Pi is the proportion of risk category i in the first risk category distribution, and Oi is the proportion of risk category i in the second risk category distribution.

[0044] Step S105: If the target similarity satisfies the second threshold, it is determined that the second interaction data in the most recent time window is abnormal.

[0045] In this embodiment, if the target similarity meets the second threshold, that is, the target similarity is small, proving that the second risk category distribution deviates far from the first risk category distribution, then it is determined that the second interaction data of the target interaction object in the most recent time window is abnormal, and there may be abnormal data transfer, abnormal network traffic, and other similar network risk behaviors.

[0046] In the present disclosure, the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interactive behavior are first determined, and then the second risk category distribution of the target interactive object in the most recent time window is determined. Based on the difference in sensitivity and the proportion of the risk category in the first risk category distribution and the second risk category distribution, the target similarity of the first risk category distribution and the second risk category distribution is determined. If the target similarity is small, it is determined that the second interactive data in the most recent time window is abnormal. In this way, the reasonable value of the time window can be automatically determined to ensure the accuracy of the time window, and there is no need to set a fixed threshold or too much prior knowledge. Dynamic anomaly identification can be performed according to changes in interactive data, thereby avoiding false alarms and alarm storms, and further improving the accuracy of anomaly detection results.

[0047] In another embodiment, the step S101 of “determining the first risk category distribution of the target environment and the sensitivity of each risk category to the abnormal interaction behavior” includes:

[0048] The first interaction data is input into a risk classification model, and the risk classification model outputs the first risk category distribution of the target environment and the sensitivity of each risk category. The risk classification model is trained based on the training interaction data and the risk categories corresponding to the training interaction data; wherein the risk categories corresponding to the training interaction data are obtained by clustering the training interaction data based on the weight values ​​of the features in the training interaction data and the difference between the feature values ​​of every two training interaction data.

[0049] In this embodiment, the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interactive behavior can be determined based on the risk classification model. The risk classification model is trained based on the training interaction data and the risk categories corresponding to the training interaction data. The first interaction data is input into the risk classification model, and the risk classification model can output the first risk category distribution of the target environment and the sensitivity of each risk category.

[0050] In one possible implementation, the training process of the risk classification model is as follows: obtain training interaction data, which can be interaction data corresponding to multiple target environments and interaction objects, perform feature selection on the training interaction data, such as selecting "file / traffic transmission size per unit time", "number of files / traffic transmission per unit time", "data confidentiality level", "interaction object rank", etc. as features, and normalize the selected features, and then use a set-based decision tree model to evaluate the importance of each feature, convert it into a weight value of each feature, and the sum of the weight values ​​of all features is 1; then the training interaction data after feature selection can be clustered based on the HDBSCAN (Hierachical Density-Based Spatial Clustering of Applications with Noise, density-based high-dimensional spatial clustering applications) algorithm. In one example, the distance between the training interaction data can be calculated based on the following formula 2:

[0051] metric(X,Y)=∑(Wi×|Xi-Yi|) Formula 2

[0052] Among them, metric(X,Y) is the distance between training interaction data X and Y, Wi is the weight of feature i, Xi is the value of feature i in training interaction data X, and Yi is the value of feature i in training interaction data Y. Using the above formula 2 to calculate the distance between training interaction data can better reflect the importance of each feature in risk classification, thereby improving the accuracy of clustering results.

[0053] After obtaining multiple risk categories of the training interaction data based on the HDBSCAN algorithm, the training interaction data and the risk categories corresponding to the training interaction data can be input into a predetermined initial model for training to obtain a risk classification model. After classifying the interaction data, the risk classification model can determine the proportion of each risk category, and determine the median of each risk category as the sensitivity of the risk category. In one example, the value corresponding to the training interaction data can be determined based on the weight Wi of feature i and the value of feature i in the training interaction data, and the median of the values ​​corresponding to all training interaction data in each risk category can be determined as the sensitivity of the risk category. It should be emphasized that the features and feature weights used by the risk classification model can also be updated iteratively based on the warning of abnormal interaction data.

[0054] Figure 3 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 2 ,like Figure 2 As shown, a data processing method includes:

[0055] Step S201: determining a first risk category distribution of the target environment and a sensitivity of each risk category to abnormal interaction behavior based on first interaction data between the target environment and all interaction objects.

[0056] The specific implementation details of step S201 are similar to those of step S101 and will not be repeated here.

[0057] Step S202, under the initial value of the time window, based on the first interaction data in each time window, determine the risk category distribution of each time window.

[0058] In this embodiment, in order to determine the time window value for detecting interaction data, the initial value of the time window can be determined first. For example, the initial value can be 1, that is, the first interaction data generated each day is determined as data within the same time window, thereby dividing the first interaction data into multiple time windows, and then determining the risk category distribution of the first interaction data in each time window.

[0059] Step S203: determining the similarity of the risk category distributions between every two time windows based on the difference and sensitivity of the proportion of each risk category in the risk category distributions between every two time windows.

[0060] In this embodiment, in order to ensure that the risk category distribution of the first interaction data in all time windows is relatively stable, it is also necessary to determine the similarity of the risk category distribution of every two time windows based on the difference and sensitivity of the proportion of each risk category in the risk category distribution of every two time windows. In one example, the similarity of the risk category distribution of every two time windows can be determined based on the following formula three:

[0061] Similarity (M,N) =1-∑(Si×|Mi-Ni|) Formula 3

[0062] Among them, Similarity (M,N) is the similarity between time window M and time window N, Si is the sensitivity of each risk category i, Mi is the proportion of risk category i in the risk category distribution of time window M, and Ni is the proportion of risk category i in the risk category distribution of time window N.

[0063] Figure 4 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 2 ,like Figure 4 As shown in the figure, the first interaction data of n days is taken to form a time window sequence (T1, T2, T3, ..., T n ), where T1 is the first time window, T2 is the second time window, and so on. If the initial value of the time window is 3, the similarity of the two symmetrical window subsequences Seq can be calculated with the midpoint of the sequence as the symmetry axis. For example, the time window subsequence Seq 1-X (T1, T2, T3) risk category distribution and time window subsequence Seq 1-y (T n-2 , T n-1 , T n )’s risk category distribution similarity1, calculate the time window subsequence Seq 2-X Risk category distribution and time window subsequence Seq (T2, T3, T4) 2-y (T n-3 , T n-2 , T n-1 )’s risk category distribution similarity Similarity2, and so on, ultimately obtaining a similarity sequence (Similarity1, Similarity2, Similarity3, …).

[0064] If the similarities all meet the first threshold, step S204 is executed to determine the initial value as the time window value.

[0065] If there is a similarity that does not meet the first threshold, step S205 is executed to adjust the initial value, and under the adjusted initial value, the similarity of the risk category distribution of every two time windows is determined until the similarities all meet the first threshold.

[0066] In this embodiment, if the similarities in the similarity sequence all meet the first threshold, it proves that the risk category distribution of the first interaction data in all time windows is relatively stable, and the initial value can be determined as the time window value; if there is a similarity that does not meet the first threshold, the initial value is adjusted, for example, the initial value is increased by 1, and then the similarity of the risk category distribution of every two time windows is re-determined under the adjusted initial value, until all similarities meet the first threshold, and the adjusted initial value at this time is determined as the time window value. In this way, the value of the time window that guarantees the stability of the risk category distribution of all time windows can be automatically determined, the accuracy of the time window is guaranteed, and the accuracy of the subsequent abnormality detection results for the second interaction data in the time window is further improved.

[0067] Step S206: determining a second risk category distribution of the target interactive object based on second interaction data between the target interactive object and the target environment in a recent time window.

[0068] Step S207: determining the target similarity between the first risk category distribution and the second risk category distribution based on the difference in sensitivity and the proportion of the risk category in the first risk category distribution and the second risk category distribution.

[0069] Step S208: If the target similarity satisfies the second threshold, it is determined that the second interaction data in the most recent time window is abnormal.

[0070] The specific implementation details of step S206 to step S208 are similar to those of step S103 to step S105 and are not repeated here.

[0071] Figure 5 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 3 ,like Figure 5 As shown, a data processing method includes:

[0072] Step S301: determining a first risk category distribution of the target environment and a sensitivity of each risk category to abnormal interaction behavior based on first interaction data between the target environment and all interaction objects.

[0073] Step S302: Determine the time window value that meets the target condition based on the sensitivity.

[0074] Step S303: determining a second risk category distribution of the target interactive object based on second interaction data between the target interactive object and the target environment in a recent time window.

[0075] The specific implementation details of step S301 to step S303 are similar to those of step S101 to step S103 and are not repeated here.

[0076] Step S304: determine the proportion of unclassified risk categories in the second risk category distribution.

[0077] In this embodiment, the proportion of unclassified risk categories in the second risk category distribution is the proportion of risk categories in the second risk category distribution that do not belong to the first risk category distribution. Figure 2 For example, if the first risk category distribution of a certain position includes risk categories C1, C2, C3, C4, C5 and C6, their corresponding proportions are 13.0%, 6.0%, 36.0%, 29.0%, 9.0% and 7.0% respectively; the second risk category distribution of an employee in this position includes risk categories C1, C2, C3 and C7, and their corresponding proportions are 15.0%, 6.0%, 38.0% and 41.0% respectively. Since risk category C7 does not belong to the risk category in the first risk category distribution, the proportion of risk category C7 (41.0%) is taken as the proportion of unclassified risk categories.

[0078] Step S305: determining the target similarity based on the sensitivity, the difference between the proportions of the risk categories in the first risk category distribution and the second risk category distribution, and the proportion of unclassified risk categories.

[0079] In this embodiment, the first risk category distribution of the target environment is the average risk category distribution of the target environment over a period of time, and the second risk category distribution is the risk category distribution of the target interactive object in a recent time window. If the second risk category distribution deviates far from the first risk category distribution, it proves that there is an anomaly in the data of the target interactive object interacting with the target environment in the most recent time window. Therefore, it is necessary to determine the target similarity of the first risk category distribution and the second risk category distribution to characterize the degree to which the second risk category distribution deviates from the first risk category.

[0080] In this embodiment, the target similarity between the first risk category distribution and the second risk category distribution can be determined based on the sensitivity calculated in step S101, the difference between the proportions of the risk category in the first risk category distribution and the second risk category distribution, and the proportion of the unclassified risk category. The target similarity can be determined based on the following formula 4:

[0081] Similarity=(1-∑(Si×|Pi-Oi|))×(1-D) Formula 4

[0082] Among them, Similarity is the target similarity, Si is the sensitivity of each risk category i, Pi is the proportion of risk category i in the first risk category distribution, Oi is the proportion of risk category i in the second risk category distribution, and D is the proportion of unclassified risk categories in the second risk category distribution.

[0083] It should be emphasized that in order to ensure the stability of the algorithm, when using Formula 4 to calculate the target similarity, it is necessary to convert the sum of the sensitivity and proportion of all risk categories involved in the first risk category distribution to 1. For example, if the first risk category distribution of a position includes risk categories C1, C2, C3, C4, C5 and C6, their corresponding proportions are 13.0%, 6.0%, 36.0%, 29.0%, 9.0% and 7.0% respectively; the second risk category distribution of an employee in this position includes risk categories C1, C2, C3, C4, C5 and C6, and their corresponding proportions are 13.0%, 6.0%, 36.0%, 29.0%, 9.0% and 7.0% respectively. 2. C3 and C7, whose corresponding proportions are 15.0%, 6.0%, 38.0% and 41.0% respectively. When calculating the target similarity, all risk categories involved in the first risk category distribution include C1, C2 and C3. Therefore, it is necessary to convert the sum of the proportions of C1, C2 and C3 in the first risk category distribution to 1, that is, transform 13.0%, 6.0% and 36.0% so that the sum of the three transformed proportions is 1. Accordingly, it is also necessary to convert the sum of the sensitivities of C1, C2 and C3 to 1.

[0084] Step S306: If the target similarity satisfies the second threshold, it is determined that the second interaction data in the most recent time window is abnormal.

[0085] The specific implementation details of step S306 are similar to those of step S105 and will not be repeated here.

[0086] In the present disclosure, when determining the target similarity between the first risk category distribution and the second risk category distribution, the proportion of unclassified risk categories in the second risk category distribution is introduced, which can more comprehensively evaluate the similarity between the behavior of the target interactive object and the characteristics of its corresponding target environment. For example, the target interactive object accesses some new files or traffic, and these behaviors may not be classified, but as long as the proportion of unclassified risk categories is within a certain threshold, it is considered to be normal behavior. If the proportion of unclassified risk categories is too high, it is considered to be abnormal behavior, which further improves the accuracy of the anomaly detection results.

[0087] In another embodiment, a data processing method further includes:

[0088] Determine the historical similarity between the risk category distribution of the target interactive object in the historical time window and the first risk category distribution; detect the abnormal similarity in the similarity sequence composed of the target similarity and the historical similarity based on the anomaly detection algorithm, and determine whether the second interaction data in the time window corresponding to the abnormal similarity has an anomaly.

[0089] In this embodiment, after determining the time window value that meets the target condition, the second interaction data between the target interaction object and the target environment in multiple historical time windows can also be determined based on the time window value, where the historical time windows are multiple time windows before the most recent time window, and the risk category distribution of the second interaction data in multiple historical time windows is determined, and then the historical similarity between the risk category distribution of the second interaction data in multiple historical time windows and the first risk category distribution is determined respectively, and the multiple historical similarities and the target similarity corresponding to the most recent time window form a similarity sequence, and the abnormal similarity in the similarity sequence can be detected based on the abnormal detection algorithm, and it is determined that the second interaction data in the time window corresponding to the abnormal similarity has an abnormality, wherein the abnormal detection algorithm can be an isolation forest algorithm and a K nearest neighbor algorithm. Thus, the abnormal similarity is determined from the similarity sequence between the risk category distribution of the target interaction object in multiple different time windows and the first risk category distribution, and the second interaction data corresponding to the abnormal similarity has an abnormality, which can avoid the problem of false alarm in the detection process of a single time window. For example, if only a single time window is detected, the time window may be determined as abnormal, but from the perspective of multiple time windows, a slight fluctuation of the time window may not be determined as abnormal.

[0090] Figure 6 A schematic diagram showing a data processing method according to an embodiment of the present disclosure Figure 3 ,like Figure 6 As shown, the horizontal axis is the time window and the vertical axis is the similarity. If the anomaly detection algorithm detects that the similarity at point a and point b is abnormal, it can be determined that there is an anomaly in the second interaction data within the time window corresponding to point a and point b.

[0091] In another embodiment, a data processing method further includes at least one of the following:

[0092] If the second interaction data is abnormal, an alarm message is generated, and the alarm message includes the abnormal reason of the second interaction data; if the second interaction data is abnormal, the interaction behavior between the target interaction object and the target environment is suspended.

[0093] In this embodiment, if it is determined that there is an abnormality in the second interaction data of the target interactive object in a certain time window, an alarm information can be generated to prompt the target interactive object that there is an abnormality in the interaction behavior and the cause of the abnormality, so that the target interactive object can promptly record or handle the abnormality based on the alarm information; the interaction behavior between the target interactive object and the target environment can also be automatically suspended. For example, if the target interactive object transmits a file in the target environment, if the file or traffic is too large during the transmission process, resulting in abnormal interaction behavior, the file transmission is automatically suspended to avoid the expansion of the abnormal interaction behavior.

[0094] Figure 7 A schematic diagram of the structure of a data processing device according to an embodiment of the present disclosure is shown. Figure 7 As shown, a data processing device includes:

[0095] A category determination module 10 is used to determine the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior based on the first interaction data between the target environment and all interaction objects; a window determination module 11 is used to determine the time window value that meets the target condition based on the sensitivity, and the target condition characterizes that the similarity of the risk category distribution of the first interaction data in every two time windows meets the first threshold; the category determination module 10 is also used to determine the second risk category distribution of the target interaction object based on the second interaction data of the target interaction object with the target environment in the most recent time window; a similarity determination module 12 is used to determine the target similarity of the first risk category distribution and the second risk category distribution based on the sensitivity and the difference in the proportion of the risk category in the first risk category distribution and the second risk category distribution; an anomaly determination module 13 is used to determine that there is an anomaly in the second interaction data in the most recent time window if the target similarity meets the second threshold.

[0096] In one embodiment, the category determination module 10 is also used to: input the first interaction data into a risk classification model, output the first risk category distribution of the target environment and the sensitivity of each risk category through the risk classification model, the risk classification model is trained based on the training interaction data and the risk categories corresponding to the training interaction data; wherein the risk categories corresponding to the training interaction data are obtained by clustering the training interaction data based on the weight values ​​of the features in the training interaction data and the difference between the feature values ​​of every two training interaction data.

[0097] In one embodiment, the window determination module 11 is also used to: determine the risk category distribution of each time window based on the first interaction data in each time window under the initial value of the time window; determine the similarity of the risk category distribution of every two time windows based on the difference and sensitivity of the proportion of each risk category in the risk category distribution of every two time windows; if the similarities both meet the first threshold, determine the initial value as the time window value.

[0098] In one embodiment, the window determination module 11 is further used to: if there is a similarity that does not meet the first threshold, adjust the initial value, and determine the similarity of the risk category distribution of every two time windows under the adjusted initial value until the similarities all meet the first threshold.

[0099] In one embodiment, the similarity determination module 12 is also used to: determine the proportion of unclassified risk categories in the second risk category distribution, the proportion of unclassified risk categories being the proportion of risk categories in the second risk category distribution that do not belong to the first risk category distribution; determine the target similarity based on sensitivity, the difference between the proportions of risk categories in the first risk category distribution and the second risk category distribution, and the proportion of unclassified risk categories.

[0100] In one embodiment, the anomaly determination module 13 is also used to: determine the historical similarity between the risk category distribution of the target interactive object in the historical time window and the first risk category distribution; detect the abnormal similarity in the similarity sequence composed of the target similarity and the historical similarity based on the anomaly detection algorithm, and determine whether the second interaction data in the time window corresponding to the abnormal similarity exists anomaly.

[0101] In one embodiment, the abnormality determination module 13 is also used to: generate an alarm message if there is an abnormality in the second interaction data, the alarm message including the abnormal reason of the second interaction data; if there is an abnormality in the second interaction data, suspend the interaction behavior between the target interaction object and the target environment.

[0102] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0103] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0104] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0105] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0106] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a data processing method. For example, in some embodiments, a data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of a data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform a data processing method in any other appropriate manner (e.g., by means of firmware).

[0107] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0109] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0112] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0114] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0115] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A data processing method, comprising: Determine, based on first interaction data between the target environment and all interaction objects, a first risk category distribution of the target environment and a sensitivity of each risk category to abnormal interaction behavior; Based on the sensitivity, determining a time window value that satisfies a target condition, wherein the target condition indicates that the similarity of the risk category distribution of the first interaction data in every two time windows satisfies a first threshold; Determining a second risk category distribution of the target interactive object based on second interaction data between the target interactive object and the target environment in a recent time window; Determine a target similarity between the first risk category distribution and the second risk category distribution based on the sensitivity and the difference between the proportions of the risk category in the first risk category distribution and the second risk category distribution; If the target similarity satisfies the second threshold, it is determined that the second interaction data within the most recent time window is abnormal.

2. The method according to claim 1, wherein determining the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior comprises: Inputting the first interaction data into a risk classification model, and outputting a first risk category distribution of the target environment and a sensitivity of each risk category through the risk classification model, wherein the risk classification model is trained based on the training interaction data and the risk categories corresponding to the training interaction data; The risk category corresponding to the training interaction data is obtained by clustering the training interaction data based on the weight value of the feature in the training interaction data and the difference between the feature values ​​of every two pieces of training interaction data.

3. The method according to claim 1, wherein determining the time window value that meets the target condition based on the sensitivity comprises: Under the initial value of the time window, based on the first interaction data in each time window, determine the risk category distribution of each time window; Determine the similarity of the risk category distributions between each two time windows based on the difference in the proportion of each risk category in the risk category distributions between each two time windows and the sensitivity; If the similarities all satisfy the first threshold, the initial value is determined as the time window value.

4. The method according to claim 3, further comprising: If there is a similarity that does not meet the first threshold, the initial value is adjusted, and under the adjusted initial value, the similarity of the risk category distribution of every two time windows is determined until the similarities all meet the first threshold.

5. The method according to claim 1, wherein determining the target similarity between the first risk category distribution and the second risk category distribution comprises: Determine the proportion of unclassified risk categories in the second risk category distribution, where the proportion of unclassified risk categories is the proportion of risk categories in the second risk category distribution that do not belong to the first risk category distribution; The target similarity is determined based on the sensitivity, the difference between the proportions of the risk category in the first risk category distribution and the second risk category distribution, and the proportion of unclassified risk categories.

6. The method according to any one of claims 1 to 5, further comprising: Determine the historical similarity between the risk category distribution of the target interactive object in the historical time window and the first risk category distribution; Based on an anomaly detection algorithm, an abnormal similarity in a similarity sequence consisting of the target similarity and the historical similarity is detected, and it is determined that an abnormality exists in the second interaction data within a time window corresponding to the abnormal similarity.

7. The method according to any one of claims 1 to 5, further comprising at least one of the following: If the second interaction data is abnormal, generating an alarm message, wherein the alarm message includes a reason for the abnormality of the second interaction data; If there is an abnormality in the second interaction data, the interaction between the target interaction object and the target environment is suspended.

8. A data processing device, comprising: A category determination module, configured to determine, based on first interaction data between the target environment and all interaction objects, a first risk category distribution of the target environment and a sensitivity of each risk category to abnormal interaction behavior; A window determination module, configured to determine, based on the sensitivity, a time window value that satisfies a target condition, wherein the target condition indicates that the similarity of the risk category distribution of the first interaction data in every two time windows satisfies a first threshold; The category determination module is further used to determine a second risk category distribution of the target interactive object based on second interaction data between the target interactive object and the target environment in a recent time window; A similarity determination module, configured to determine a target similarity between the first risk category distribution and the second risk category distribution based on the sensitivity and a difference between the proportions of the risk category in the first risk category distribution and the second risk category distribution; The anomaly determination module is used to determine that an anomaly exists in the second interaction data within the most recent time window if the target similarity satisfies a second threshold.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method according to any one of claims 1 to 7.

Images