A data processing method, device, apparatus, and storage medium

Abstract

The present disclosure provides a data processing method, device and equipment and storage medium, and relates to the technical field of computer. The data processing method comprises: determining a first risk category distribution of a target environment and a sensitivity of each risk category to abnormal interaction behavior based on first interaction data of the target environment and all interaction objects; determining a time window value meeting a target condition based on the sensitivity; determining a second risk category distribution of a target interaction object based on second interaction data of the target interaction object with the target environment in a recent time window; determining a target similarity of the first risk category distribution and the second risk category distribution based on the sensitivity and a difference between proportions of risk categories in the first risk category distribution and the second risk category distribution; and determining that the second interaction data in the recent time window is abnormal if the target similarity meets a second threshold.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a data processing method and device, equipment and storage medium. BACKGROUND

[0002] At present, there is no clear industry standard for risk detection of data abnormal transfer, network traffic anomaly and other similar network risk behaviors. The commonly used solution is to set a fixed threshold and determine whether the related data meets the fixed threshold. However, the above solution cannot dynamically determine a reasonable fixed threshold according to the enterprise situation, lacks flexibility, and cannot determine a relatively stable time window for risk detection, resulting in low accuracy of the detection result. In addition, the fixed threshold may cause an alarm storm, thereby covering up the real abnormal risk. SUMMARY

[0003] The present disclosure provides a data processing method, device, equipment and storage medium to at least solve the above technical problems in the prior art.

[0004] According to a first aspect of the present disclosure, a data processing method is provided, comprising: determining a first risk category distribution of a target environment and a sensitivity of each risk category to abnormal interaction behavior based on first interaction data of the target environment and all interaction objects; determining a time window value meeting a target condition based on the sensitivity, the target condition representing that the similarity of the risk category distribution of the first interaction data in each two time windows meets a first threshold; determining a second risk category distribution of a target interaction object based on second interaction data of the target interaction object with the target environment in a latest time window; determining a target similarity of the first risk category distribution and the second risk category distribution based on the sensitivity and a difference between the proportions of the risk categories in the first risk category distribution and the second risk category distribution; and determining that the second interaction data in the latest time window is abnormal if the target similarity meets a second threshold.

[0005] In an implementable manner, the determination of the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior comprises: inputting the first interaction data into a risk classification model, and outputting the first risk category distribution of the target environment and the sensitivity of each risk category through the risk classification model, the risk classification model being trained based on training interaction data and risk categories corresponding to the training interaction data; wherein the risk categories corresponding to the training interaction data are obtained by clustering the training interaction data based on the weight values of the features in the training interaction data and the difference between the feature values of each two training interaction data.

[0006] In an implementation, the determining the time window value satisfying the target condition based on the sensitivity comprises: determining, based on the first interaction data in each time window, a risk category distribution of each time window at an initial value of the time window; determining, based on the sensitivity and a difference between proportions of each risk category in the risk category distributions of each two time windows, a similarity of the risk category distributions of each two time windows; and determining the initial value as the time window value if the similarities all satisfy the first threshold.

[0007] In an implementation, the data processing method further comprises: if there is a similarity that does not satisfy the first threshold, adjusting the initial value, and determining, at the adjusted initial value, the similarity of the risk category distributions of each two time windows until the similarities all satisfy the first threshold.

[0008] In an implementation, the determining the target similarity of the first risk category distribution and the second risk category distribution comprises: determining a proportion of unclassified risk categories in the second risk category distribution, the proportion of unclassified risk categories being a proportion of risk categories in the second risk category distribution that do not belong to the first risk category distribution; and determining, based on the sensitivity, a difference between proportions of the risk categories in the first risk category distribution and the second risk category distribution, and the proportion of unclassified risk categories, the target similarity.

[0009] In an implementation, the data processing method further comprises: determining a historical similarity of the risk category distribution of the target interaction object in a historical time window and the first risk category distribution; detecting, based on an anomaly detection algorithm, an abnormal similarity in a similarity sequence composed of the target similarity and the historical similarity, and determining that the second interaction data in a time window corresponding to the abnormal similarity is abnormal.

[0010] In an implementation, the data processing method further comprises at least one of the following: if the second interaction data is abnormal, generating an alarm information, the alarm information including an abnormal reason of the second interaction data; and if the second interaction data is abnormal, suspending an interaction behavior between the target interaction object and the target environment.

[0011] According to a second aspect of the present disclosure, a data processing apparatus is provided, comprising: a category determining module configured to determine a first risk category distribution of a target environment and a sensitivity of each risk category to abnormal interaction behavior based on first interaction data of the target environment and all interaction objects; a window determining module configured to determine a time window value satisfying a target condition based on the sensitivity, the target condition representing that a similarity of risk category distributions of the first interaction data in each two time windows satisfies a first threshold; the category determining module is further configured to determine a second risk category distribution of a target interaction object based on second interaction data of the target environment in a latest time window by the target interaction object; a similarity determining module configured to determine a target similarity of the first risk category distribution and the second risk category distribution based on the sensitivity and a difference of proportions of the risk categories in the first risk category distribution and the second risk category distribution; and an abnormality determining module configured to determine that the second interaction data in the latest time window is abnormal if the target similarity satisfies a second threshold.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0013] at least one processor; and

[0014] a memory connected with the at least one processor; wherein,

[0015] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.

[0016] According to a fourth aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, the computer instructions being used to cause the computer to perform the method of the present disclosure.

[0017] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0019] In the drawings, identical or corresponding reference signs indicate identical or corresponding parts.

[0020] Figure 1A flowchart of a data processing method is shown Figure 1 ;

[0021] Figure 2 A scenario of a data processing method is shown Figure 1 ; wherein,

[0022] Figure 2 A risk category distribution diagram of a data processing method is shown

[0023] Figure 2 A sensitivity distribution diagram of a data processing method is shown

[0024] Figure 3 A flowchart of a data processing method is shown Figure 2 ;

[0025] Figure 4 A scenario of a data processing method is shown Figure 2 ;

[0026] Figure 5 A flowchart of a data processing method is shown Figure 3 ;

[0027] Figure 6 A scenario of a data processing method is shown Figure 3 ;

[0028] Figure 7 A structural diagram of a data processing device is shown

[0029] Figure 8 A structural diagram of an electronic device is shown DETAILED DESCRIPTION

[0030] In order to make the objectives, characteristics and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present disclosure.

[0031] Figure 1 A flowchart of a data processing method is shown Figure 1 As shown in Figure 1 , a data processing method includes:

[0032] Step S101, based on the first interaction data of the target environment and all interaction objects, determine the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior.

[0033] In this embodiment, the target environment is a work environment that may have network risk behavior, for example, post, system, platform, and work type, etc., and the interaction object is the object interacting with the target environment, for example, the employee on the post, the personnel operating the system, and the user interacting with the platform, etc. Based on the first interaction data of the target environment and all its interaction objects, the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior are determined, wherein the first risk category distribution includes the risk categories that may exist in the target environment and the proportion of each risk category, and the lower the sensitivity, the less the abnormal interaction behavior, and the lower the corresponding risk tolerance, that is, unable to tolerate a large amount of data or traffic transfer. In an example, the first interaction data can be clustered to obtain the risk categories that may exist in the target environment and the proportion of each risk category, and the median of each risk category is determined as the sensitivity of the risk category.

[0034] Figure 2 A shows a risk category distribution diagram of a data processing method according to an embodiment of the present disclosure, Figure 1 Figure 2 A shows a risk category distribution diagram of a data processing method according to an embodiment of the present disclosure, Figure 2 B shows a sensitivity distribution diagram of a data processing method according to an embodiment of the present disclosure, as Figure 2 A shows, the first risk category distribution of a certain post can include risk categories C1, C2, C3, C4, C5 and C6, and the corresponding proportions are 13.0%, 6.0%, 36.0%, 29.0%, 9.0% and 7.0% respectively; as Figure 2 B shows, as Figure 2 A, the sensitivities corresponding to the risk categories C1, C2, C3, C4, C5 and C6 are S1, S2, S3, S4, S5 and S6 respectively, and the proportions corresponding to each sensitivity are 31.0%, 24.0%, 19.0%, 15.0%, 7.0% and 4.0% respectively.

[0035] Step S102, based on the sensitivity, determine the time window value that meets the target condition.

[0036] ​In the embodiment, it is also required to determine the time window value for detecting the interaction data, which needs to meet a target condition, the target condition representing that the similarity of the risk category distribution of the first interaction data in each two time windows meets a first threshold, that is, the risk category distribution of the first interaction data in each two time windows is relatively stable. In an example, the time window value can be determined from 1, that is, the first interaction data generated each day is determined as the data in the same time window, so as to divide the first interaction data into multiple time windows. Based on the sensitivity of each risk category calculated in step S101, the similarity of the risk category distribution of the first interaction data in each two time windows is determined. If the similarity meets the first threshold, the time window value is determined as 1.

[0037] In step S103, the second risk category distribution of the target interaction object is determined based on the second interaction data of the target interaction object with the target environment in the latest time window.

[0038] In the embodiment, the target interaction object is an interaction object for which it is required to determine whether there is interaction data anomaly. The second interaction data of the target interaction object with the target environment in the latest time window can be selected, and the second risk category distribution of the target interaction object is determined based on the second interaction data. The process of determining the second risk category distribution is similar to that of determining the first risk category distribution, which will not be described herein.

[0039] In step S104, the target similarity of the first risk category distribution and the second risk category distribution is determined based on the difference between the sensitivity and the proportion of the risk category in the first risk category distribution and the second risk category distribution.

[0040] In the embodiment, the first risk category distribution of the target environment is the average risk category distribution of the target environment in a period of time, and the second risk category distribution is the risk category distribution of the target interaction object in the latest time window. If the second risk category distribution deviates from the first risk category distribution far, it is proved that the data of the target interaction object interacting with the target environment in the latest time window is abnormal. Therefore, it is required to determine the target similarity of the first risk category distribution and the second risk category distribution, which is used to represent the degree of deviation of the second risk category distribution from the first risk category.

[0041] In the embodiment, the target similarity of the first risk category distribution and the second risk category distribution can be determined based on the difference between the sensitivity and the proportion of the risk category in the first risk category distribution and the second risk category distribution calculated in step S101. The target similarity can be determined based on the following formula one:

[0042] Similarity = 1 -∑(Si x |Pi -Oi|) Formula one

[0043] wherein, Similarity is the target similarity, Si is the sensitivity of each risk category i, Pi is the proportion of risk category i in the first risk category distribution, and Oi is the proportion of risk category i in the second risk category distribution.

[0044] In step S105, if the target similarity meets the second threshold, it is determined that the second interaction data in the recent time window is abnormal.

[0045] In this embodiment, if the target similarity meets the second threshold, that is, the target similarity is small, which proves that the second risk category distribution deviates far from the first risk category distribution, it is determined that the second interaction data of the target interaction object in the recent time window is abnormal, which may have data abnormal transfer, network traffic anomaly and other similar network risk behaviors.

[0046] In this disclosure, first, the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior are determined, then the second risk category distribution of the target interaction object in the recent time window is determined, based on the sensitivity and the difference between the proportion of the risk category in the first risk category distribution and the second risk category distribution, the target similarity of the first risk category distribution and the second risk category distribution is determined, if the target similarity is small, it is determined that the second interaction data in the recent time window is abnormal. Therefore, the reasonable value of the time window can be automatically determined, the accuracy of the time window is guaranteed, and there is no need to set a fixed threshold, nor too much prior knowledge, dynamic anomaly recognition can be performed according to the change of the interaction data, so as to avoid false alarm and alarm storm, and further improve the accuracy of the abnormal detection result.

[0047] In another embodiment, the step S101 of "determining the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior" comprises:

[0048] The first interaction data is input into the risk classification model, and the first risk category distribution of the target environment and the sensitivity of each risk category are output by the risk classification model, the risk classification model is trained based on the training interaction data and the risk category corresponding to the training interaction data; wherein, the risk category corresponding to the training interaction data is obtained by clustering the training interaction data based on the weight value of the feature in the training interaction data and the difference between the feature values of each two training interaction data.

[0049] In the embodiment, the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behaviors can be determined based on a risk classification model, which is trained based on training interaction data and risk categories corresponding to the training interaction data. The first interaction data is input into the risk classification model, and the risk classification model outputs the first risk category distribution of the target environment and the sensitivity of each risk category.

[0050] In an implementation, the training process of the risk classification model is as follows: obtaining training interaction data, which can be interaction data corresponding to a plurality of target environments and interaction objects; performing feature selection on the training interaction data, such as selecting "file / traffic transmission size per unit time", "file / traffic transmission quantity per unit time", "data security level", "interaction object job level", etc. as features, and performing normalization processing on the selected features; then using a set-based decision tree model to evaluate the importance of each feature and convert it into a weight value of each feature, and the sum of the weight values of all features is 1; then the training interaction data after feature selection can be clustered based on the HDBSCAN (Hierachical Density-Based Spatial Clustering of Applications with Noise) algorithm. In an example, the distance between the training interaction data can be calculated based on the following formula two:

[0051] metric(X,Y)=∑(Wi×|Xi-Yi|) Formula two

[0052] Wherein, metric(X,Y) is the distance between the training interaction data X and Y, Wi is the weight of feature i, Xi is the value of feature i in the training interaction data X, and Yi is the value of feature i in the training interaction data Y. Using the above formula two to calculate the distance between the training interaction data can better reflect the importance of each feature in risk classification, thereby improving the accuracy of the clustering result.

[0053] After obtaining the plurality of risk categories of the training interaction data based on the HDBSCAN algorithm, the training interaction data and the risk categories corresponding to the training interaction data can be input into a predetermined initial model for training to obtain a risk classification model. After classifying the interaction data, the risk classification model can determine the proportion of each risk category and determine the median of each risk category as the sensitivity of the risk category. In an example, the value corresponding to the training interaction data can be determined based on the weight Wi of the feature i and the value of the feature i in the training interaction data, and the median of the values corresponding to all training interaction data in each risk category is determined as the sensitivity of the risk category. It should be emphasized that the features and the weight values of the features used by the risk classification model can also be iteratively updated based on the warning of the abnormal interaction data.

[0054] Figure 3 A flowchart of a data processing method according to an embodiment of the present disclosure is shown Figure 2 As shown in Figure 2 A data processing method includes:

[0055] Step S201, based on the first interaction data of the target environment and all interaction objects, determining the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior.

[0056] The specific implementation details of step S201 are similar to those of step S101, which will not be described here.

[0057] Step S202, based on the first interaction data in each time window, determining the risk category distribution of each time window at the initial value of the time window.

[0058] In this embodiment, in order to determine the time window value for detecting the interaction data, the initial value of the time window can be determined first, for example, the initial value can be 1, that is, the first interaction data generated each day is determined as the data in the same time window, thereby dividing the first interaction data into multiple time windows, and then determining the risk category distribution of the first interaction data in each time window.

[0059] Step S203, based on the difference between the proportion of each risk category in the risk category distribution of each two time windows and the sensitivity, determining the similarity of the risk category distribution of each two time windows.

[0060] In this embodiment, in order to ensure that the risk category distribution of the first interaction data in all time windows is relatively stable, the similarity of the risk category distribution of each two time windows also needs to be determined based on the difference between the proportion of each risk category in the risk category distribution of each two time windows and the sensitivity. In an example, the similarity of the risk category distribution of each two time windows can be determined based on the following formula three:

[0061] Similarity (M,N) =1-∑(Si×|Mi-Ni|) Formula 3

[0062] Among them, Similarity (M,N) Let S be the similarity between time windows M and N, Si be the sensitivity of each risk category i, Mi be the proportion of risk category i in the risk category distribution of time window M, and Ni be the proportion of risk category i in the risk category distribution of time window N.

[0063] Figure 4 This illustration shows a scenario illustrating a data processing method according to an embodiment of the present disclosure. Figure 2 ,like Figure 4 As shown, the first interaction data of n days are taken to form a time window sequence (T1, T2, T3, ..., T...). n In this context, T1 represents the first time window, T2 the second time window, and so on. If the initial value of the time window is 3, then the similarity between two symmetrical window subsequences Seq can be calculated using the midpoint of the sequence as the axis of symmetry. For example, to calculate the similarity between the time window subsequences Seq... 1-X Risk category distribution and time window subsequence Seq for (T1, T2, T3) 1-y (T n-2 T n-1 T n Similarity1 is used to calculate the similarity of risk category distributions within a time window (Seq). 2-X Risk category distribution and time window subsequence Seq for (T2, T3, T4) 2-y (T n-3 T n-2 T n-1 The similarity of the risk category distribution is Similarity2, and so on, until the similarity sequence (Similarity1, Similarity2, Similarity3, ...) is obtained.

[0064] If the similarity scores all meet the first threshold, then step S204 is executed to determine the initial values ​​as the time window values.

[0065] If there are similarities that do not meet the first threshold, then proceed to step S205 to adjust the initial values. Under the adjusted initial values, determine the similarity of the risk category distributions for every two time windows until the similarities all meet the first threshold.

[0066] In the embodiment, if the similarities in the similarity sequence all satisfy the first threshold value, it is proved that the risk category distribution of the first interaction data in all time windows is relatively stable, and the initial value can be determined as the time window value; if there is a similarity that does not satisfy the first threshold value, the initial value is adjusted, for example, the initial value is increased by 1, and then the similarity of the risk category distribution of each two time windows is determined again at the adjusted initial value, until all similarities satisfy the first threshold value, and the initial value at this time is determined as the time window value after adjustment. Therefore, the value of the time window that guarantees the stability of the risk category distribution of all time windows can be automatically determined, the accuracy of the time window is guaranteed, and the accuracy of the subsequent abnormal detection result of the second interaction data in the time window is further improved.

[0067] In step S206, the second risk category distribution of the target interaction object is determined based on the second interaction data of the target interaction object with the target environment in the latest time window.

[0068] In step S207, the target similarity of the first risk category distribution and the second risk category distribution is determined based on the difference between the sensitivity and the proportion of the risk category in the first risk category distribution and the second risk category distribution.

[0069] In step S208, if the target similarity satisfies the second threshold value, it is determined that the second interaction data in the latest time window is abnormal.

[0070] The specific implementation details of steps S206-S208 are similar to steps S103-S105, and will not be described here.

[0071] Figure 5 A flowchart of a data processing method according to an embodiment of the present disclosure is shown Figure 3 As shown in Figure 5 A data processing method includes:

[0072] In step S301, the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior are determined based on the first interaction data of the target environment and all interaction objects.

[0073] In step S302, the time window value satisfying the target condition is determined based on the sensitivity.

[0074] In step S303, the second risk category distribution of the target interaction object is determined based on the second interaction data of the target interaction object with the target environment in the latest time window.

[0075] The specific implementation details of steps S301-S303 are similar to steps S101-S103, and will not be described here.

[0076] Step S304, determine the proportion of unclassified risk categories in the second risk category distribution.

[0077] In this embodiment, the proportion of unclassified risk categories in the second risk category distribution is the proportion of risk categories in the second risk category distribution that do not belong to the first risk category distribution. For example, Figure 2 For example, if the first risk category distribution of a certain post includes risk categories C1, C2, C3, C4, C5 and C6, and the corresponding proportions are 13.0%, 6.0%, 36.0%, 29.0%, 9.0% and 7.0% respectively; the second risk category distribution of an employee in this post includes risk categories C1, C2, C3 and C7, and the corresponding proportions are 15.0%, 6.0%, 38.0% and 41.0% respectively, since risk category C7 does not belong to the risk categories in the first risk category distribution, the proportion of risk category C7, 41.0%, is taken as the proportion of unclassified risk categories.

[0078] Step S305, determine the target similarity based on the sensitivity, the difference between the proportions of risk categories in the first risk category distribution and the second risk category distribution, and the proportion of unclassified risk categories.

[0079] In this embodiment, the first risk category distribution of the target environment is the average risk category distribution of the target environment over a period of time, and the second risk category distribution is the risk category distribution of the target interactive object in the recent time window. If the second risk category distribution deviates far from the first risk category distribution, it proves that the data of the target interactive object interacting with the target environment in the recent time window is abnormal, so it is necessary to determine the target similarity of the first risk category distribution and the second risk category distribution, which is used to represent the degree of deviation of the second risk category distribution from the first risk category.

[0080] In this embodiment, the target similarity of the first risk category distribution and the second risk category distribution can be determined based on the sensitivity calculated in step S101, the difference between the proportions of risk categories in the first risk category distribution and the second risk category distribution, and the proportion of unclassified risk categories. The target similarity can be determined based on the following formula four:

[0081] Similarity=(1-∑(Si×|Pi-Oi|))×(1-D) Formula four

[0082] Wherein, Similarity is the target similarity, Si is the sensitivity of each risk category i, Pi is the proportion of risk category i in the first risk category distribution, Oi is the proportion of risk category i in the second risk category distribution, and D is the proportion of unclassified risk categories in the second risk category distribution.

[0083] It should be emphasized that in order to ensure the stability of the algorithm, when calculating the target similarity using formula four, the sum of the sensitivity and the proportion of all risk categories involved in the first risk category distribution needs to be converted to 1. For example, if the first risk category distribution of a certain post includes risk categories C1, C2, C3, C4, C5 and C6, and the corresponding proportions are 13.0%, 6.0%, 36.0%, 29.0%, 9.0% and 7.0% respectively; the second risk category distribution of an employee in this post includes risk categories C1, C2, C3 and C7, and the corresponding proportions are 15.0%, 6.0%, 38.0% and 41.0% respectively, when calculating the target similarity, all risk categories involved in the first risk category distribution including C1, C2 and C3, therefore, the sum of the proportions of C1, C2 and C3 in the first risk category distribution needs to be converted to 1, i.e. transforming 13.0%, 6.0% and 36.0% so that the sum of the three proportions after transformation is 1, and correspondingly, the sum of the sensitivities of C1, C2 and C3 also needs to be converted to 1.

[0084] Step S306, if the target similarity meets the second threshold, it is determined that the second interaction data in the recent time window is abnormal.

[0085] The specific implementation details of step S306 are similar to those of step S105, which will not be described here.

[0086] In the present disclosure, when determining the target similarity of the first risk category distribution and the second risk category distribution, the proportion of the unclassified risk category in the second risk category distribution is introduced, which can more comprehensively evaluate the similarity between the behavior of the target interaction object and the characteristics of the corresponding target environment. For example, the target interaction object accesses some new files or traffic, which may not be classified, but as long as the proportion of the unclassified risk category is within a certain threshold, it is considered normal behavior. If the proportion of the unclassified risk category is too high, it is considered abnormal behavior, which further improves the accuracy of the abnormal detection result.

[0087] In another embodiment, a data processing method further comprises:

[0088] determining a historical similarity of the risk category distribution of the target interaction object in the historical time window and the first risk category distribution; detecting an abnormal similarity in a similarity sequence composed of the target similarity and the historical similarity based on an anomaly detection algorithm, and determining that the second interaction data in the time window corresponding to the abnormal similarity is abnormal.

[0089] In this embodiment, after determining the time window value satisfying the target condition, the second interaction data of the target interaction object with the target environment in a plurality of historical time windows can be determined based on the time window value, the historical time windows are a plurality of time windows before the latest time window, the risk category distribution of the second interaction data in the plurality of historical time windows is determined, and then the historical similarity of the risk category distribution of the second interaction data in the plurality of historical time windows and the first risk category distribution is respectively determined, the plurality of historical similarities and the target similarity corresponding to the latest time window form a similarity sequence, the abnormal similarity in the similarity sequence can be detected based on an anomaly detection algorithm, and it is determined that the second interaction data in the time window corresponding to the abnormal similarity is abnormal, wherein the anomaly detection algorithm can be an isolation forest algorithm and a K-nearest neighbor algorithm. Thus, the abnormal similarity is determined from the similarity sequence of the risk category distribution of the target interaction object in a plurality of different time windows and the first risk category distribution, and it is determined that the second interaction data corresponding to the abnormal similarity is abnormal, which can avoid false alarm problems in a single time window detection process. For example, if only a single time window is detected, the time window can be determined to be abnormal, but a slight fluctuation in the time window can not be determined to be abnormal when viewed from a plurality of time windows.

[0090] Figure 6 A scene diagram of a data processing method according to an embodiment of the present disclosure is shown Figure 3 As shown in Figure 6 If the anomaly detection algorithm detects that the similarities at points a and b are abnormal, it can be determined that the second interaction data in the time window corresponding to points a and b is abnormal.

[0091] In another embodiment, a data processing method further includes at least one of the following:

[0092] If the second interaction data is abnormal, an alarm information is generated, and the alarm information includes the abnormal reason of the second interaction data; if the second interaction data is abnormal, the interaction behavior between the target interaction object and the target environment is suspended.

[0093] In this embodiment, if it is determined that the second interaction data of the target interaction object in a certain time window is abnormal, an alarm information can be generated to prompt that the interaction behavior of the target interaction object is abnormal and the abnormal reason, so that the target interaction object records or processes the abnormality in a timely manner based on the alarm information; the interaction behavior between the target interaction object and the target environment can also be automatically suspended, for example, if the target interaction object transmits a file in the target environment, if the file or traffic is too large during transmission, the transmission of the file is automatically suspended to avoid the expansion of abnormal interaction behavior.

[0094] Figure 7 A structural schematic diagram of a data processing apparatus is shown in FIG. 1, which is an embodiment of the present disclosure. Figure 7 As shown in FIG. 1, a data processing apparatus includes:

[0095] The category determining module 10 is configured to determine a first risk category distribution of the target environment and a sensitivity of each risk category to abnormal interaction behavior based on the first interaction data of the target environment and all interaction objects; the window determining module 11 is configured to determine a time window value that satisfies a target condition based on the sensitivity, the target condition representing that the similarity of the risk category distribution of the first interaction data in each two time windows satisfies a first threshold; the category determining module 10 is further configured to determine a second risk category distribution of the target interaction object based on the second interaction data of the target interaction object in the latest time window and the target environment; the similarity determining module 12 is configured to determine a target similarity of the first risk category distribution and the second risk category distribution based on the sensitivity and the difference of the proportion of the risk category in the first risk category distribution and the second risk category distribution; and the abnormality determining module 13 is configured to determine that the second interaction data in the latest time window is abnormal if the target similarity satisfies a second threshold.

[0096] In an implementation, the category determining module 10 is further configured to input the first interaction data into a risk classification model, and output the first risk category distribution of the target environment and the sensitivity of each risk category through the risk classification model, the risk classification model being trained based on training interaction data and a risk category corresponding to the training interaction data; wherein the risk category corresponding to the training interaction data is obtained by clustering the training interaction data based on the weight value of a feature in the training interaction data and the difference of the feature values of each two training interaction data.

[0097] In an implementation, the window determining module 11 is further configured to determine, under an initial value of the time window, a risk category distribution of each time window based on the first interaction data in each time window; determine the similarity of the risk category distribution of each two time windows based on the difference of the proportion of each risk category in the risk category distribution of each two time windows and the sensitivity; and determine the initial value as the time window value if the similarity satisfies the first threshold.

[0098] In an implementation, the window determining module 11 is further configured to adjust the initial value if there is a similarity that does not satisfy the first threshold, and determine the similarity of the risk category distribution of each two time windows under the adjusted initial value until the similarity satisfies the first threshold.

[0099] In an implementation, the similarity determination module 12 is further configured to determine a proportion of unclassified risk categories in the second risk category distribution, the proportion of unclassified risk categories being a proportion of risk categories in the second risk category distribution that do not belong to the first risk category distribution; and determine the target similarity based on the sensitivity, a difference between proportions of the risk categories in the first risk category distribution and the second risk category distribution, and the proportion of unclassified risk categories.

[0100] In an implementation, the anomaly determination module 13 is further configured to determine a historical similarity between the risk category distribution of the target interaction object in the historical time window and the first risk category distribution; detect an abnormal similarity in a similarity sequence composed of the target similarity and the historical similarity based on the anomaly detection algorithm, and determine that the second interaction data in a time window corresponding to the abnormal similarity is abnormal.

[0101] In an implementation, the anomaly determination module 13 is further configured to generate an alarm information if the second interaction data is abnormal, the alarm information including an abnormal reason of the second interaction data; and suspend the interaction behavior between the target interaction object and the target environment if the second interaction data is abnormal.

[0102] According to embodiments of the present disclosure, the present disclosure further provides an electronic device and a readable storage medium.

[0103] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0104] As shown in Figure 8 The device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0105] A plurality of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0106] The computing unit 801 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs various methods and processes described above, such as a data processing method. For example, in some embodiments, a data processing method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded onto the RAM 803 and executed by the computing unit 801, one or more steps of a data processing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform a data processing method by any other appropriate means, such as by means of firmware.

[0107] Various implementations of the systems and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip system (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0108] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0109] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0110] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0111] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0112] The computer system can include clients and servers. The clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server can arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0113] It should be understood that the steps as shown above can be reordered, added, or deleted, using various forms of flow. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions of the present disclosure can be achieved, and the present disclosure is not limited herein.

[0114] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0115] The above description is merely a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present disclosure, which should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A data processing method, comprising: Based on the first interaction data between the target environment and all interacting objects, determine the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior; Based on the sensitivity, the value of the time window that satisfies the target condition is determined. The target condition represents that the similarity of the risk category distribution of the first interactive data in every two time windows satisfies the first threshold. Based on the second interaction data between the target interaction object and the target environment in the most recent time window, determine the second risk category distribution of the target interaction object; Based on the sensitivity and the difference between the proportion of the risk category in the first risk category distribution and the second risk category distribution, the target similarity between the first risk category distribution and the second risk category distribution is determined; If the target similarity meets the second threshold, then it is determined that the second interaction data within the most recent time window is abnormal.

2. The method according to claim 1, wherein determining the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interactive behavior comprises: The first interactive data is input into the risk classification model, and the risk classification model outputs the first risk category distribution of the target environment and the sensitivity of each risk category. The risk classification model is trained based on the training interactive data and the risk categories corresponding to the training interactive data. The risk category corresponding to the training interaction data is obtained by clustering the training interaction data based on the weight values ​​of the features in the training interaction data and the difference between the feature values ​​of each pair of training interaction data.

3. The method according to claim 1, wherein determining the time window value for satisfying the target condition based on the sensitivity includes: Given the initial values ​​of the time windows, the risk category distribution for each time window is determined based on the first interaction data within each time window. The similarity of the risk category distributions between two time windows is determined based on the difference in the proportion of each risk category in the risk category distributions between two time windows and the sensitivity mentioned above. If the similarity scores all meet the first threshold, then the initial value is determined as the time window value.

4. The method according to claim 3, further comprising: If there is a similarity that does not meet the first threshold, the initial value is adjusted. Under the adjusted initial value, the similarity of the risk category distribution between every two time windows is determined until the similarity meets the first threshold.

5. The method according to claim 1, wherein determining the target similarity between the first risk category distribution and the second risk category distribution comprises: Determine the proportion of unclassified risk categories in the second risk category distribution, wherein the proportion of unclassified risk categories in the second risk category distribution is the proportion of risk categories that do not belong to the first risk category distribution; The target similarity is determined based on the sensitivity, the difference between the proportion of the risk category in the first risk category distribution and the second risk category distribution, and the proportion of the unclassified risk category.

6. The method according to any one of claims 1-5, further comprising: Determine the historical similarity between the risk category distribution of the target interaction object within a historical time window and the first risk category distribution; Anomaly detection algorithms are used to detect abnormal similarities in the similarity sequence composed of the target similarity and the historical similarity, and to determine that the second interaction data within the time window corresponding to the abnormal similarity is abnormal.

7. The method according to any one of claims 1-5, further comprising at least one of the following: If the second interactive data is abnormal, an alarm message is generated, which includes the reason for the abnormality of the second interactive data. If the second interaction data is abnormal, the interaction between the target interaction object and the target environment is paused.

8. A data processing apparatus, comprising: The category determination module is used to determine the first risk category distribution of the target environment and the sensitivity of each risk category to abnormal interaction behavior based on the first interaction data between the target environment and all interacting objects. The window determination module is used to determine the value of the time window that meets the target condition based on the sensitivity, wherein the target condition represents that the similarity of the risk category distribution of the first interactive data in every two time windows meets a first threshold. The category determination module is also used to determine the second risk category distribution of the target interaction object based on the second interaction data between the target interaction object and the target environment in the most recent time window; The similarity determination module is used to determine the target similarity between the first risk category distribution and the second risk category distribution based on the sensitivity and the difference between the proportion of the risk category in the first risk category distribution and the second risk category distribution; An anomaly determination module is used to determine that the second interaction data within the most recent time window is abnormal if the target similarity meets a second threshold.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method according to any one of claims 1-7.

Images