Behavior anomaly detection method and device, equipment, medium and program product

Abstract

The invention provides a behavior anomaly detection method and device, equipment, a medium and a program product, and relates to the field of artificial intelligence, and the method is characterized in that the method comprises the steps: collecting safety audit data of a user, and carrying out the feature processing of the safety audit data, and obtaining a behavior feature vector set; an initial candidate cluster number and a hard affiliation variable are obtained, a center vector and an initial mixing proportion of each initial candidate cluster are determined, and the hard affiliation variable represents an affiliation relation between the behavior feature vector and the cluster; and iteratively updating the hard affiliation variable, the center vector and the mixing ratio based on an unsupervised k-means objective function, dynamically eliminating clusters of which the mixing ratio is smaller than a preset threshold value until a convergence condition is met, and constructing a user normal behavior baseline, the clustering objective function comprises at least one of a distance cost item, two types of entropy penalty items and an entropy-driven competitive adjustment item; and judging whether the behavior of the user corresponding to the to-be-detected data is abnormal or not based on the normal behavior baseline of the user, and outputting a judgment result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, device, medium, and program product for detecting abnormal behavior. Background Technology

[0002] User and Entity Behavior Analytics (UEBA) is a well-known concept in the security field. It refers to establishing a baseline model of normal behavior by analyzing historical behavioral data of users and entities (such as servers and applications), and then detecting abnormal activities that deviate from this baseline to discover potential threats. In the traditional field of UEBA, the construction of behavioral baselines relies on manually preset cluster numbers and weak cluster interference, resulting in low baseline accuracy. Summary of the Invention

[0003] This application provides a method and apparatus for detecting abnormal behavior, which can solve the problem of low accuracy of abnormal behavior detection results.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a method for detecting abnormal behavior, the method comprising:

[0006] Collect user security audit data, and after performing feature processing on the security audit data, obtain a set of behavioral feature vectors;

[0007] Obtain the initial number of candidate clusters and hard attribution variables, determine the center vector and initial mixing ratio of each initial candidate cluster, and the hard attribution variables represent the relationship between the behavioral feature vector and the cluster attribution;

[0008] Based on the clustering objective function, the hard attribution variable, center vector and mixing ratio are iteratively updated, and clusters with mixing ratios less than a preset threshold are dynamically eliminated until the convergence condition is met, and a baseline of normal user behavior is constructed. The clustering objective function includes at least one of the following: distance cost term, two types of entropy penalty term and entropy-driven competitive adjustment term.

[0009] Based on the baseline of normal user behavior, determine whether the behavior of the user corresponding to the test data is abnormal, and output the determination result.

[0010] Secondly, embodiments of this application provide a behavioral anomaly detection device, the device comprising:

[0011] The acquisition module is used to collect the user's security audit data, and after performing feature processing on the security audit data, obtain a set of behavioral feature vectors.

[0012] The determination module is used to obtain the initial number of candidate clusters and hard attribution variables, determine the center vector and initial mixing ratio of each initial candidate cluster, and the hard attribution variables represent the relationship between the behavioral feature vector and the cluster attribution;

[0013] The update module is used to iteratively update the hard attribution variable, center vector and mixing ratio based on the clustering objective function, dynamically eliminate clusters with a mixing ratio less than a preset threshold until the convergence condition is met, and construct a baseline for normal user behavior. The clustering objective function includes at least one of a distance cost term, two types of entropy penalty terms and an entropy-driven competitive adjustment term.

[0014] The judgment module is used to determine whether the behavior of the user corresponding to the test data is abnormal based on the user's normal behavior baseline, and output the judgment result.

[0015] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and the program or instructions, when executed by the processor, implement the steps of the behavior anomaly detection method as described in the first aspect.

[0016] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the behavior anomaly detection method as described in the first aspect.

[0017] In this embodiment, user security audit data is collected, and after feature processing of the security audit data, a set of behavioral feature vectors is obtained. An initial number of candidate clusters and a hard attribution variable are acquired, and the center vector and initial mixing ratio of each initial candidate cluster are determined. The hard attribution variable represents the attribution relationship between the behavioral feature vector and the cluster. Based on a clustering objective function, the hard attribution variable, center vector, and mixing ratio are iteratively updated, and clusters with a mixing ratio less than a preset threshold are dynamically eliminated until the convergence condition is met, thus constructing a baseline for normal user behavior. The clustering objective function includes at least one of a distance cost term, two types of entropy penalty terms, and an entropy-driven competitive adjustment term. Based on the baseline for normal user behavior, it is determined whether the behavior of the user corresponding to the test data is abnormal, and the determination result is output. In this way, by fusing distance cost and entropy penalty terms in the clustering objective function, combining feature processing with an improved clustering framework, implementing an iterative update mechanism for dynamically eliminating redundant clusters with low mixing ratios, competitive cost determination based on behavioral baselines, and adaptive clustering logic that does not require manual parameter tuning, we have achieved fully automatic cluster number determination without the need for preset cluster numbers and parameter ranges, effective processing of high-dimensional multi-source security audit data, and reduced computational complexity of massive data processing. This has improved the accuracy of behavioral anomaly detection results while enhancing robustness to different data distributions. Attached Figure Description

[0018] Figure 1 This is one of the flowcharts illustrating the behavior anomaly detection method provided in the embodiments of this application;

[0019] Figure 2 A flowchart illustrating the unsupervised k-means algorithm provided in this application embodiment;

[0020] Figure 3 This is a schematic diagram of streaming incremental update and window management provided in an embodiment of this application;

[0021] Figure 4 The second schematic flowchart of the behavior anomaly detection method provided in the embodiments of this application;

[0022] Figure 5 This is a schematic diagram illustrating the anomaly detection and interpretable output provided in the embodiments of this application;

[0023] Figure 6 This is a schematic diagram of the structure of the behavior anomaly detection device provided in the embodiments of this application;

[0024] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The following description, in conjunction with the accompanying drawings, further illustrates the abnormal behavior detection method, apparatus, device, and storage medium proposed in the embodiments of the application.

[0027] Please see Figure 1 , Figure 1 A flowchart illustrating a behavioral anomaly detection method provided in this application is shown in the figure. The method includes:

[0028] Step 110: Collect user security audit data, and after performing feature processing on the security audit data, obtain a set of behavioral feature vectors;

[0029] In this step, the aforementioned security audit data refers to various behavioral records related to user operations within the information system. These can include user login / logout records, session duration data, geographic location information, device fingerprint data, command sequences, SQL operation characteristics, file or table access frequency and time distribution data, network traffic statistics, and API call path data. The aforementioned feature processing involves organizing and transforming the collected raw security audit data. This process converts unstructured or semi-structured data into structured numerical forms. The behavioral feature vector set is the final product of this processing, which can be a set of vectors composed of multiple dimensions of numerical values. Each vector uniquely corresponds to the feature representation of a user behavior record. For example, for login behavior data, after feature processing, feature vectors containing dimensions such as login time, login location, device identifier, and login duration can be obtained; for database operation data, feature vectors containing dimensions such as operation type, accessed table name, and operation frequency can be obtained.

[0030] For example, user-related security audit events and characteristics can be collected, such as login / logout, session duration, geolocation and device fingerprint, command sequence or SQL characteristics, file / table access frequency and time distribution, network traffic statistics, API call paths, and risk tags. Furthermore, time alignment, cleaning, and feature engineering (normalization, standardization, binning, embedding / bag-of-words, sequence statistics, matrix factorization, etc.) can be performed to obtain a d-dimensional feature vector set. .

[0031] In some alternative implementations, the initial cluster number can be... Initial cluster center (Or through sampling / parallel seeding), initial mixing ratio ;

[0032] Define hard attribution variables ;

[0033] Learning rate .

[0034] Step 120: Obtain the initial number of candidate clusters and hard attribution variables, and determine the center vector and initial mixing ratio of each initial candidate cluster. The hard attribution variables represent the relationship between the behavioral feature vector and the cluster attribution.

[0035] In this step, the initial number of candidate clusters can be understood as the number of initial clusters set when the algorithm starts. Its value can be flexibly set according to the data scale; for example, it can be set to 200 or 300, ensuring that there are enough clusters participating in the subsequent competitive elimination process in the initial state. The hard attribution variable is used to characterize the attribution relationship between the behavioral feature vector and the cluster. Its value can be 0 or 1. When the variable is 1, it indicates that the corresponding behavioral feature vector belongs to that cluster; when it is 0, it indicates that it does not belong to that cluster. The center vector is the core representative vector of each cluster. Its dimension is consistent with the dimension of the behavioral feature vector. The initial center vector can be determined by randomly selecting a portion of vectors from the behavioral feature vector set as the initial center, or by extracting representative vectors from different data subsets through stratified sampling. The aforementioned initial mixing ratio refers to the proportion of each cluster in the initial state. It can be set in an equal distribution manner. For example, when the initial number of candidate clusters is 200, the initial mixing ratio of each cluster can be set to 1 / 200. Alternatively, different ratio values ​​can be preset according to the potential importance of the initial clusters. For example, a higher initial mixing ratio can be assigned to the initial clusters corresponding to the important behaviors initially identified in the data.

[0036] Step 130: Iteratively update the hard attribution variable, center vector and mixing ratio based on the clustering objective function, dynamically eliminate clusters with a mixing ratio less than a preset threshold until the convergence condition is met, and construct a baseline for normal user behavior. The clustering objective function includes at least one of a distance cost term, two types of entropy penalty terms and an entropy-driven competitive adjustment term.

[0037] In this step, the clustering objective function mentioned above is an optimization function built based on the k-means algorithm framework. It does not rely on manually labeled categories and can guide the cluster structure towards a more rational approach through its own optimization logic. The composition of this objective function can be flexibly chosen. It can include only a distance cost term to measure the compactness of the data within a cluster; it can also include both a distance cost term and two types of entropy penalty terms, introducing an inter-cluster competition mechanism through the entropy penalty term; or it can include a distance cost term, two types of entropy penalty terms, and an entropy-driven competitive adjustment term to further enhance the dynamic optimization capability of the clusters. Iterative updates involve repeatedly calculating and adjusting the values ​​of the hard attribution variables, center vectors, and mixing ratios to gradually optimize the objective function value. In each iteration, the hard attribution variables can be updated based on the calculation results of the objective function, redetermining the cluster to which each behavioral feature vector belongs; based on the updated attribution relationships, the center vectors of each cluster are recalculated to better represent the characteristics of the data within the cluster; simultaneously, the mixing ratios of each cluster are updated so that the mixing ratios reflect the actual representativeness of the clusters. During the dynamic elimination process, the preset threshold can be set according to the total amount of data, for example, it can be set to 1 / n (where n is the total number of behavioral feature vectors), or it can be set to other reasonable values. When the mixing ratio of a certain cluster is lower than this threshold, it indicates that the representativeness of the cluster is weak and it belongs to a redundant cluster, so it is removed from the cluster set. The above convergence condition is the judgment criterion for stopping the algorithm iteration. It can be that the number of iterations reaches a preset upper limit, or that the maximum change of the cluster center vector is less than the preset convergence threshold. For example, the preset convergence threshold can be set to 0.001 or 0.0001. When the convergence condition is met, it indicates that the cluster structure has become stable. At this time, a user normal behavior baseline is constructed based on the final cluster structure. The user normal behavior baseline is the benchmark that represents the user's normal behavior pattern. It can be composed only of the final cluster center vector, or it can be composed of the cluster center vector and intra-cluster statistical information. Intra-cluster statistical information can include cluster radius, cluster density, intra-cluster feature time distribution, etc.

[0038] For example, an unsupervised objective function can be defined (without presetting k):

[0039]

[0040] Iterative update rules

[0041] Taking the partial derivative of the objective function, we obtain the following iterative update rule:

[0042] Attribution Update:

[0043] Center Update:

[0044] Ratio update:

[0045] And on Normalization .

[0046] Adaptive parameters: ;

[0047] in , Based on stability inequalities to ensure ;

[0048] Cluster elimination and reunification: if If an invalid ratio is found, the cluster will be removed and a new cluster will be set. ; for the remainder Return to oneness;

[0049] Convergence and acceleration: If And if c remains stable, then set Accelerate convergence; when Stop when the time comes.

[0050] Furthermore, the final cluster center Intra-cluster statistics (radius / density / temporal distribution, etc.) can form a baseline for the behavior of users or groups.

[0051] Step 140: Based on the baseline of normal user behavior, determine whether the behavior of the user corresponding to the test data is abnormal, and output the determination result.

[0052] In this step, the data to be tested refers to the user behavior data that needs to be anomaly detected. Its data type is consistent with the collected security audit data, such as real-time user login data or database operation data. When determining whether a behavior is abnormal, the data to be tested is first subjected to feature processing consistent with the previous steps to obtain a feature vector to be detected. Then, the degree of matching between this vector and each cluster in the user's normal behavior baseline is calculated. If the degree of matching is lower than a preset standard, it is determined to be abnormal behavior; if the degree of matching meets the preset standard, it is determined to be normal behavior. When outputting the judgment result, only the judgment conclusion of abnormality or normality can be output, or the judgment basis can be output simultaneously, such as the degree of deviation of the detected feature vector from the closest baseline cluster, the main deviation feature dimensions, etc. The output format can be a text report or a visual interface display, convenient for relevant personnel to view and subsequently process.

[0053] In the behavior anomaly detection method of this application embodiment, by fusing distance cost and entropy penalty terms with an unsupervised k-means objective function, combining feature processing with an improved clustering framework, an iterative update mechanism for dynamically eliminating redundant clusters with low mixing ratios, competitive cost determination based on behavioral baselines, and adaptive clustering logic that does not require manual parameter tuning, the method achieves fully automatic cluster number determination without preset cluster number and parameter range, effective processing of high-dimensional multi-source security audit data, and reduction of computational complexity in massive data processing, while also enhancing robustness to different data distributions.

[0054] Optionally, the feature processing includes preprocessing and feature engineering; the preprocessing includes time alignment and data cleaning; and the feature engineering includes at least one of normalization, standardization, embedding, sequence statistics, and matrix factorization.

[0055] In this implementation, the preprocessing described above is a fundamental step in feature processing. It can initially organize the raw security audit data, laying the foundation for subsequent feature engineering. Specifically, it can include time alignment processing and data cleaning processing. Time alignment processing adjusts security audit data from different sources and with different time formats to a consistent time dimension through a unified time benchmark, ensuring that the data is comparable and correlated in the time dimension. For example, for behavior records from terminal operation logs and behavior records from network traffic logs, their timestamp formats may be in milliseconds and seconds, respectively. Time alignment processing can round down the millisecond timestamps to the second level, or precisely extend the second timestamps to the millisecond level, keeping the time stamps of the two types of data consistent. Furthermore, for security audit data generated across time zones, time alignment processing can convert the timestamps of all data to the standard time of the same time zone, avoiding deviations in behavior timing analysis caused by time zone differences. Data cleaning processing identifies and processes invalid information, outliers, and duplicate data in the security audit data, ensuring the accuracy and integrity of the data. For example, for null fields in login logs, data cleaning can either fill the field with common values ​​or directly remove the entire record containing key null values. For access frequency data in database access logs that clearly exceeds a reasonable range, data cleaning can identify them as outliers and replace them with the mean or median of the field. Alternatively, it can set thresholds based on business scenarios and mark data outside the thresholds as pending verification before targeted processing. For repeatedly collected records of the same operation, data cleaning can either retain the earliest record or merge the valid information from duplicate records to form a complete record.

[0056] The feature engineering process described above, based on preprocessing, further transforms and optimizes the normalized data. Through extraction and transformation operations, the data is converted into a feature form more suitable for algorithmic models, improving the model's clustering performance and anomaly detection accuracy. Feature engineering processes can be flexibly selected according to data type and business needs, including at least one of the following: normalization, standardization, embedding, sequence statistics, and matrix factorization.

[0057] The normalization process described above uses specific mathematical transformations to map data feature values ​​to a fixed numerical range, eliminating dimensional differences between different feature dimensions. For example, normalization can employ min-max normalization to map feature values ​​to the range of 0 to 1, or decimal scaling normalization to map feature values ​​to the range of -1 to 1 by shifting the decimal point. For features with different dimensions, such as user session duration and file access frequency, normalization ensures that all features have equal weight in model training, avoiding model bias caused by dimensional differences.

[0058] The standardization process described above, based on the statistical characteristics of the data, transforms the data into a standard normal distribution with a mean of 0 and a variance of 1, making the data characteristics more consistent with the assumptions of the algorithm model. For example, standardization can employ the Z-score standardization method, which calculates the difference between the feature value and the mean and then divides it by the standard deviation; alternatively, it can use the median standardization method, which divides the difference between the feature value and the median by the interquartile range. For feature data that follows a non-normal distribution, such as network traffic statistics and API call counts, standardization can effectively reduce the impact of data distribution skewness on the model.

[0059] The embedding process described above transforms high-dimensional discrete or sequential features into low-dimensional dense vectors using mapping functions, while preserving the inherent relationships between features. For example, for user command sequences or SQL statement sequences, the embedding process can use the Word2Vec model to map each command or statement into a low-dimensional vector, or it can use the GloVe model to construct embedding vectors based on global word frequency statistics. For discrete categorical features such as device fingerprints and geographic locations, the embedding process can transform them into continuous low-dimensional vectors, making it easier for the model to capture the potential relationships between features of different categories.

[0060] The aforementioned sequence statistical processing involves statistically analyzing security audit data with time-series characteristics to extract statistical features that characterize behavioral patterns. For example, for a user's login behavior sequence over a period of time, sequence statistical processing can calculate the number of logins per unit time, the mean or variance of login intervals, and can also statistically analyze the usage frequency and proportion of different login devices. For SQL statement sequences of database operations, sequence statistical processing can statistically analyze the frequency of occurrence of various operation statements, the distribution characteristics of statement lengths, and can also analyze the dependencies between operation sequences and extract relevant statistical indicators.

[0061] The matrix factorization process described above decomposes a high-dimensional original data matrix into a product of multiple low-dimensional matrices through matrix operations, achieving the dual goals of data dimensionality reduction and feature extraction. For example, matrix factorization can employ principal component analysis (PCA) to decompose the high-dimensional data matrix into eigenvector matrices and eigenvalue matrices, selecting principal components to form low-dimensional features. Alternatively, singular value decomposition (SVD) can be used to decompose the original data matrix into left singular matrices, singular value matrices, and right singular matrices, selecting low-dimensional features based on the contribution of the singular values. For high-dimensional feature matrices that integrate multi-source log data, matrix factorization can effectively remove redundant information, retain the core features of the data, and mitigate the impact of high-dimensional data on algorithm performance.

[0062] In practical applications, the above feature engineering processing can flexibly select a single processing method or combine multiple processing methods according to the specific type, feature dimension and business scenario requirements of security audit data, to ensure that the final behavioral feature vector set can accurately characterize user behavior patterns and provide reliable support for subsequent clustering analysis and anomaly detection.

[0063] In the behavior anomaly detection method of this application embodiment, feature engineering (such as embedding and matrix factorization) is used to reduce the dimensionality and extract features from the original high-dimensional log data, transforming it into a more representative low-dimensional feature vector. This provides an effective input for distance-based clustering algorithms, indirectly alleviating the high-dimensionality problem. Furthermore, the algorithm's final output is the cluster centers representing the behavioral baseline, which still have clear mathematical and business significance in the high-dimensional feature space.

[0064] Optionally, the two types of entropy penalty terms include a first entropy penalty term and a second entropy penalty term. The first entropy penalty term is constructed based on the distribution characteristics of the cluster mixing ratio, and the second entropy penalty term is constructed based on the association characteristics between the behavioral feature vector and the cluster mixing ratio.

[0065] And / or,

[0066] The distance cost term characterizes the compactness of the data within the cluster with respect to the cluster center.

[0067] In this implementation, the two types of entropy penalty terms are key components of the objective function for achieving adaptive competition among clusters. Their core function is to guide the dynamic evolution of clusters through mathematical constraints, avoiding manual intervention in cluster number settings and allowing the cluster structure to naturally tend towards the optimal. The first entropy penalty term is constructed based on the distribution characteristics of cluster mixing ratios, which represent the proportion of each cluster in the entire cluster set. The distribution characteristics can be understood as the uniformity or concentration of the mixing ratios of each cluster. The core function of the first entropy penalty term is to penalize unreasonable mixing ratio distributions, driving the mixing ratios of weak clusters to gradually decrease and the mixing ratios of strong clusters to continuously strengthen, thereby achieving the natural elimination of redundant clusters. For example, when the mixing ratio of a certain cluster is much lower than that of other clusters, the first entropy penalty term will increase accordingly, causing that cluster to be further weakened in subsequent iterations until it is eliminated. Alternatively, when the mixing ratios of multiple clusters are too close, resulting in a dispersed distribution, the first entropy penalty term will adjust to gradually increase the proportion of some clusters and continuously decrease the proportion of others, forming a clear cluster hierarchy.

[0068] The second entropy penalty term is constructed based on the association features between the cluster affiliation of behavioral feature vectors and the cluster mixing ratio. The affiliation relationship refers to the correspondence between each behavioral feature vector and a specific cluster, and the association feature can be understood as the degree of matching or consistency between this affiliation relationship and the mixing ratio of each cluster. The role of the second entropy penalty term is to match the affiliation decision of behavioral feature vectors with the actual competitiveness of the clusters, strengthen healthy competition among clusters, and prevent weak clusters from occupying too many data resources. For example, when the mixing ratio of a cluster is high and remains stable during iteration, it indicates that the cluster has strong representativeness. The second entropy penalty term will guide more behavioral feature vectors to affiliate to this cluster, further consolidating its competitive advantage. Alternatively, when the mixing ratio of a cluster continues to decline, it indicates that its representativeness is gradually weakening. The second entropy penalty term will reduce the attractiveness of this cluster to behavioral feature vectors, prompting these vectors to affiliate to other clusters with more reasonable mixing ratios and stronger representativeness, accelerating the elimination process of weak clusters.

[0069] The distance cost term is a fundamental component of the objective function to ensure clustering quality. Its core function is to characterize the compactness of the data within a cluster with respect to the cluster center. Compactness can be understood as the degree of aggregation between all behavioral feature vectors within a cluster and the cluster center vector. The smaller the value of the distance cost term, the more concentrated and similar the data within the cluster, and the more reasonable the clustering result. The design purpose of this part is to ensure that each cluster can accurately characterize a type of normal user behavior pattern, providing a reliable baseline for subsequent anomaly detection. The implementation method of the distance cost term can be flexibly chosen. For example, it can be constructed by calculating the sum of the squares of the Euclidean distances between each behavioral feature vector within the cluster and the cluster center vector. This method can effectively amplify the impact of outliers on the cost, prompting the cluster center to move closer to the data-dense region. Alternatively, it can be constructed by calculating the sum of the cosine distances. This method is more suitable for high-dimensional data scenarios and can effectively measure the directional consistency between feature vectors, ensuring the similarity of data within the cluster in semantic or feature dimensions.

[0070] In some alternative implementations, the expression for the objective function J is:

[0071]

[0072] Among them, by adjusting hyperparameters and It controls the balance between cluster compactness and cluster diversity / competition, driving the algorithm to automatically eliminate redundant clusters and merge similar clusters from the initial excessive number of candidate clusters, and finally converge to the optimal number of clusters.

[0073] Two types of entropy penalty terms (mixed ratio) can be introduced into the classic k-means objective function. Shannon entropy and Attribution-Proportional Cross-Entropy ), and construct a new objective function J that does not require a preset cluster number k or its range.

[0074] In practical applications, the two types of entropy penalty terms can be applied to the objective function individually or simultaneously. The distance cost term works in conjunction with the entropy penalty term as a basic component. They can be flexibly combined according to the characteristics of data distribution and business needs, ensuring the compactness of data within clusters while achieving adaptive competition between clusters, ultimately outputting the optimal clustering results and user behavior baseline.

[0075] In the behavior anomaly detection method of this application embodiment, a mixing ratio-based method is introduced into the objective function. The entropy penalty term constructs an inherent competition and elimination mechanism. The algorithm starts by initializing an excessive number of candidate clusters, and during iteration, updates the ratio according to the given rules to adjust the mixing ratio of redundant or weak clusters. Automatic decay until it is eliminated (e.g.) Strong clusters continue to grow and eventually stabilize at the optimal number of clusters. This process is entirely data-driven and requires no prior knowledge about the number of clusters.

[0076] Optionally, the clustering objective function based on a distance cost term and two types of entropy penalty terms, combined with an entropy-driven competitive adjustment term, iteratively updates the hard attribution variable, center vector, and mixing ratio, dynamically eliminating clusters with a mixing ratio less than a preset threshold, until the convergence condition is met, including:

[0077] Based on the distance between the behavioral feature vector and the center vector corresponding to each candidate cluster, and the correlation between the mixing ratio of each candidate cluster, the cluster to which each behavioral feature vector uniquely belongs is determined, and the hard attribution variable is updated.

[0078] Update the center vector of the corresponding cluster based on the mean of all behavioral feature vectors belonging to each cluster;

[0079] By combining the proportion of behavioral feature vectors belonging to each cluster to the total number, and the competition adjustment term in the clustering objective function, a new mixing ratio for each cluster is obtained, and the new mixing ratio is normalized.

[0080] Clusters whose normalized mixing ratio is less than a preset threshold or are invalid are removed. The current cluster count is updated based on the number of removed clusters. Normalization is then performed on the mixing ratio and hard attribution variable of the remaining clusters.

[0081] In this implementation, determining the unique cluster to which each behavioral feature vector belongs, based on the distance between the behavioral feature vector and the center vector of each candidate cluster, and the correlation between the mixing ratio of each candidate cluster, is a crucial step in achieving healthy competition among clusters. This correlation can be understood as the priority of a behavioral feature vector's cluster affiliation being determined by both distance and mixing ratio. A smaller distance indicates a higher similarity between the vector and the cluster center, while a higher mixing ratio indicates stronger cluster competitiveness. The two work together to form a comprehensive basis for affiliation determination. The distance calculation method can be flexibly chosen; for example, Euclidean distance can be used to measure absolute differences in the vector space, or cosine distance can be used to measure similarity along the vector direction. For the correlation weight of the mixing ratio, either a linear method can be used to add the logarithmic terms of the distance and the mixing ratio as the affiliation determination index, or a non-linear method can be used to weight the mixing ratio and combine it with the distance. The update of the aforementioned hard affiliation variable, based on the comprehensive determination result, assigns a unique cluster identifier to each behavioral feature vector, ensuring that each vector belongs to only one cluster, making the cluster structure clear and unambiguous.

[0082] The core step of updating the cluster center vector based on the mean of all behavioral feature vectors belonging to each cluster is to ensure that the cluster center continuously aligns with the data characteristics within the cluster. The mean calculation here can be a simple arithmetic mean, i.e., summing the values ​​of each dimension of all behavioral feature vectors belonging to the cluster and dividing by the number of vectors; or it can be a weighted mean, for example, assigning weights based on the collection time of the behavioral feature vectors, giving higher weights to recent data to adapt to dynamic changes in behavioral patterns, or assigning different weights based on the confidence level of the vectors (such as data integrity and the reliability of the collection equipment) to improve the representativeness of the center vector. By continuously updating the center vector, the core features of each cluster can be continuously optimized, more accurately depicting a type of user behavior pattern.

[0083] By combining the proportion of behavioral feature vectors belonging to each cluster to the total number of clusters, and the competition adjustment term in the clustering objective function, a new mixing ratio for each cluster can be obtained, serving as a key step in strengthening inter-cluster competition. The proportion of behavioral feature vectors belonging to each cluster to the total number of clusters represents the sample coverage rate of that cluster, reflecting its basic size. The competition adjustment term is a dynamic adjustment factor designed based on the entropy penalty term in the objective function. It can amplify the advantages of strong clusters and weaken the competitiveness of weak clusters. For example, when the mixing ratio of a cluster continuously increases, the competition adjustment term can further enhance its growth rate; when the mixing ratio of a cluster continuously decreases, the competition adjustment term can accelerate its decay. The calculation of the new mixing ratio can employ a linear fusion method, combining the sample coverage rate and the competition adjustment term with fixed weights, or a dynamic fusion method, adjusting the weight ratio of the two according to the number of iterations, emphasizing sample coverage rate in the early stages of iteration and emphasizing the competition adjustment term in the later stages. Normalizing the new mixing ratios ensures that the sum of the mixing ratios of all clusters is 1, guaranteeing the rationality and comparability of the ratios. The normalization method can be L1 normalization, which is to divide the mixing ratio of each cluster by the sum of the mixing ratios of all clusters, or L2 normalization, which is adjusted according to the sum of squares of the mixing ratios.

[0084] Removing clusters whose normalized mixing ratio is less than a preset threshold or whose values ​​are invalid is the core step in achieving automatic cluster number optimization. The preset threshold can be dynamically adjusted based on the data scale. For example, it can be set to the reciprocal of the total number of samples to ensure that only redundant clusters with extremely low sample coverage are eliminated. Alternatively, it can be set to a fixed threshold, such as 0.001, suitable for scenarios with relatively stable data scales. Invalid values ​​are those that do not conform to the ratio definition, such as negative numbers, values ​​greater than 1, or null values. These values ​​usually arise from calculation errors or the influence of abnormal data; removing such clusters avoids their interference with the overall clustering results. After updating the current cluster number based on the number of clusters removed, normalization processing needs to be performed on the mixing ratio and hard attribution variable of the remaining clusters to ensure data consistency. The mixing ratio of the remaining clusters can be recalculated so that the sum of the proportions of each cluster is 1. The hard attribution variable can be relabeled based on the cluster index after removal. For example, if the original cluster index was 1 to 100, after removing 5 redundant clusters, the remaining cluster index can be reorganized to 1 to 95. Alternatively, the original index can be retained while only the labels corresponding to redundant clusters are deleted to ensure accurate variable mapping in subsequent iterations.

[0085] In some alternative implementations, the mixing ratio The update formula can be:

[0086]

[0087] After the update, normalization is required, based on a preset threshold (e.g., ...). Dynamically eliminate redundant clusters to achieve automatic reduction of the number of clusters.

[0088] Furthermore, in determining the attribution of data points to clusters, the aforementioned attribution variables... The assignment rules can be:

[0089]

[0090] This rule allows the size (proportion) of a cluster to be used as a reflection of its competitiveness, directly participating in the competition for data point ownership.

[0091] By employing a single-iteration process of "initial overload - competitive elimination," although the initial number of clusters c is relatively large, redundant clusters can be quickly eliminated in the early stages of the iteration through an efficient competition mechanism (ownership update rule) and a dynamic elimination mechanism (proportional update rule), causing the effective number of clusters c to decrease rapidly, thereby reducing the complexity of the single iteration. It is controlled at a low level. Overall, it avoids the outer loop search required by X-means, thus reducing the total computational overhead.

[0092] For single iteration complexity Driven by competitive elimination, c decreases rapidly, outperforming schemes that require outer model selection. z can be computed in partitioned parallelism with local statistics, and then globally updated after central merging. With parameters; it can provide REST / message interface to connect to SIEM / UEBA platforms.

[0093] For example, the UK-means-SA algorithm process can be referred to Figure 2 The process begins with an input feature vector set, obtained from security audit data after feature processing. Each vector corresponds to a structured feature representation of a user behavior. During the initialization phase, the hard attribution variable is set to an initial state, with a value of 1, used to initially label the cluster affiliation of the feature vectors.

[0094] After entering the attribution update phase, for each feature vector in the set, its association cost with each cluster center is calculated. This cost comprehensively considers the squared distance between the vector and the cluster center and the logarithm of the cluster mixing ratio. When the cost corresponding to a certain cluster meets the minimum condition, the hard attribution variable corresponding to that vector is set to 1, indicating that the vector belongs to that cluster; otherwise, it is set to 0 to ensure that each feature vector uniquely belongs to only one cluster.

[0095] The process then proceeds to the ratio and parameter update step. Based on the allocation results of the hard attribution variables, a new mixing ratio α for each cluster is calculated. This calculation process integrates the attribution vector count and the competition adjustment term. After obtaining the new mixing ratio, it needs to be normalized to ensure that the sum of the mixing ratios of all clusters is 1. At the same time, relevant parameters can be dynamically adjusted according to the current cluster structure and changes in the mixing ratio, so that the parameters adapt to the cluster competition state in the iteration process.

[0096] The next step is the cluster elimination process. It checks whether the mixing ratio of each cluster is less than a preset threshold (e.g., the reciprocal of the total sample size, 1 / n) or if there are invalid values. If either condition is met, the cluster is removed, and the current number of valid clusters, c, is updated based on the number of removed clusters. After cluster elimination, the mixing ratio α and hard attribution variable z of the remaining clusters need to be re-normalized to ensure data consistency and reasonableness.

[0097] After cluster elimination is completed, the algorithm enters the convergence judgment phase. The core of this phase is to detect the changes in the cluster centers during the iteration process, calculate the degree of difference of the same cluster center in two adjacent iterations, take the maximum value of the changes in all cluster centers, and determine whether this maximum value is less than the preset convergence threshold ε. If the maximum value is less than ε, it means that the cluster centers have stabilized, and the algorithm stops iterating; otherwise, it returns to the affiliation update phase to continue the loop iteration.

[0098] Once the algorithm meets the convergence condition, the final output includes the set of cluster centers corresponding to the user behavior baseline, the set of mixing ratios for each cluster, and intra-cluster statistics. These outputs together constitute the benchmark for normal user behavior and can provide core evidence for subsequent anomaly detection.

[0099] The entire iterative update and cluster elimination process will be executed cyclically until the convergence condition is met. Each cycle will make the cluster structure more reasonable, the representativeness of strong clusters will be continuously enhanced, and weak clusters will be gradually eliminated, eventually forming a cluster set that highly matches the user's normal behavior pattern, providing high-quality basic data for the subsequent construction of behavioral baselines.

[0100] In the behavior anomaly detection method of this application embodiment, the objective function J is derived with respect to the mixing ratio. The partial derivative yields a unique mixing ratio. The rules are updated dynamically. This rule relies not only on the hard attribution of data points. It also introduces a competing adjustment term driven by the current mixing ratio entropy, which makes the ratio of small-sized or poorly descriptive clusters... The decay occurs continuously during iteration. Furthermore, the traditional nearest neighbor assignment rule is modified by incorporating the logarithm of the distance from a point to the cluster center into the cluster proportion. Combining these factors, a competitive attribution criterion can be formed. This rule makes data points more likely to be assigned to clusters with a larger proportion and stronger performance, even if they are slightly farther away, thereby accelerating the elimination process of weak clusters.

[0101] Optionally, the convergence condition is that the maximum value of the change in the cluster centers during the iteration process is less than a preset convergence threshold;

[0102] The baseline of normal user behavior includes the cluster center and intra-cluster statistics after iterative convergence. The intra-cluster statistics include at least one of cluster radius, cluster density, and intra-cluster characteristic temporal distribution.

[0103] In this implementation, the convergence condition is the core criterion for determining whether clustering iteration has stopped. Specifically, it is defined as the maximum value of the change in cluster centers during the iteration process being less than a preset convergence threshold. The aforementioned change in cluster centers refers to the degree of difference between the same cluster center vector in two adjacent iterations. This difference can be measured by the changes in the values ​​of each dimension of the vector. The maximum value of the change in cluster centers is the largest among all cluster center changes, which directly reflects the overall stability of the cluster structure. For example, when calculating the change in cluster centers, Euclidean distance can be used to measure the overall difference in vectors, or Manhattan distance can be used to focus on the sum of the absolute changes in the values ​​of each dimension. Regarding the change in cluster centers during the iteration process, if the changes in the centers of all clusters after a certain iteration are 0.002, 0.0015, and 0.0008 respectively, then the maximum value of the change in cluster centers is 0.002. The preset convergence threshold is a very small value set in advance to determine whether the cluster centers have reached a stable state. Its value can be flexibly adjusted according to the accuracy requirements of the data. For example, it can be set to 0.001, which is suitable for scenarios with moderate clustering accuracy requirements; or it can be set to 0.0001, which is suitable for scenarios with high clustering accuracy requirements. When the maximum value of the change in each cluster center is lower than this threshold, it indicates that the cluster centers have basically stabilized, and continuing to iterate will not significantly improve the clustering quality. At this time, the algorithm stops iterating.

[0104] The aforementioned baseline of normal user behavior is the core basis for characterizing users' regular behavioral patterns. It is composed of cluster centers after iterative convergence and intra-cluster statistics. The cluster centers after iterative convergence are the core representative vectors of each cluster, accurately reflecting the core characteristics of the behavioral pattern corresponding to that cluster, and are a fundamental component of the behavioral baseline. Intra-cluster statistics further supplement and describe the characteristics of the data within the cluster, making the behavioral baseline more comprehensive and accurate. They may include at least one of cluster radius, cluster density, and intra-cluster feature temporal distribution.

[0105] The cluster radius, an indicator of the dispersion of data within a cluster, can be calculated flexibly. For example, it can be the average distance from all behavioral feature vectors within the cluster to the cluster center, reflecting the overall concentration of data within the cluster; or it can be the maximum distance from all behavioral feature vectors within the cluster to the cluster center, clearly defining the cluster's coverage boundary. Cluster density, an indicator of the density of data within a cluster, can be the ratio of the number of behavioral feature vectors within the cluster to the volume of space covered by the cluster, suitable for measuring the density of cluster distribution in space; or it can be the number of behavioral feature vectors within the cluster within a unit distance, more intuitively reflecting the local density of data within the cluster. The temporal distribution of cluster features refers to the distribution pattern of behavioral features within the cluster over time. For example, it can be the frequency distribution of a specific behavioral feature at different times of the day, reflecting the user's temporal behavior habits; or it can be the distribution of the changing trend of behavioral features over time, reflecting the temporal evolution of user behavior.

[0106] In practical applications, the selection of intra-cluster statistical information can be flexibly combined according to business needs. For example, only cluster radius and cluster density can be selected to build a simpler behavioral baseline, or cluster radius, cluster density and intra-cluster feature time distribution can be included to form a more comprehensive behavioral baseline, ensuring that the user's normal behavior pattern can be accurately captured and providing reliable support for subsequent anomaly detection.

[0107] The behavior anomaly detection method of this application ensures the stability and reliability of the cluster structure by clearly defining the convergence conditions, and provides an accurate comparison benchmark for anomaly detection by clearly defining the baseline components, thus ensuring the scientific nature and effectiveness of the entire anomaly detection process.

[0108] Optionally, the entropy penalty term includes an adjustment parameter, wherein the adjustment parameter of the first entropy penalty term is dynamically adjusted according to the current number of clusters; and the adjustment parameter of the second entropy penalty term is dynamically adjusted according to the change in the mixing ratio during iteration.

[0109] In this embodiment, the aforementioned adjustment parameter can be understood as a parameter that varies with time (iteration number t) and the current cluster number. Dynamically adjust hyperparameters and A strategy, rather than a fixed value. For example, This causes it to increase as the number of clusters decreases, gradually strengthening its dependence on the cluster ratio; It then adaptively adjusts based on the stability of changes in the mixing ratio.

[0110] The adjustment parameter of the first entropy penalty term is dynamically adjusted according to the current number of clusters, which is the number of effective clusters currently existing during the clustering iteration process. This number gradually decreases and stabilizes as redundant clusters are eliminated. The core logic of this dynamic adjustment is to adapt to different stages of inter-cluster competition. In the early stages of iteration, the current number of clusters is large, with many candidate clusters participating in the competition. At this time, the adjustment parameter can be set relatively small to weaken the constraint of the entropy penalty term, providing sufficient exploration space for inter-cluster competition and avoiding premature elimination of potentially valuable clusters. In the later stages of iteration, the current number of clusters gradually decreases, and the cluster structure begins to stabilize. At this time, the adjustment parameter can be increased accordingly to strengthen the constraint of the entropy penalty term, accelerate the elimination of redundant clusters, and consolidate the optimal cluster structure. The specific adjustment method of the adjustment parameter can be flexibly chosen. For example, an exponential function can be used, where the adjustment parameter increases exponentially as the current number of clusters decreases, allowing the penalty intensity to increase rapidly; or a linear function can be used, where the adjustment parameter increases by a preset fixed amount for every fixed decrease in the current number of clusters, allowing the penalty intensity to increase steadily.

[0111] The adjustment parameter of the second entropy penalty term is dynamically adjusted based on the change in the mixing ratio during iteration. The change in the mixing ratio during iteration represents the difference in the mixing ratio of the same cluster between two adjacent iterations, which directly reflects the change in the cluster's competitiveness. A large change indicates drastic fluctuations in the cluster's competitiveness, while a small change indicates that the cluster's competitiveness is stabilizing. This adjustment logic balances the stability and competitiveness of the cluster structure. If the change in the mixing ratio is large, it indicates that the cluster's competitiveness is not yet stable. In this case, the adjustment parameter can be appropriately reduced to decrease the strength of the entropy penalty term and avoid excessive penalty leading to cluster structure imbalance. If the change in the mixing ratio is small, it indicates that the cluster's competitiveness has stabilized. In this case, the adjustment parameter can be increased to strengthen the constraint of the entropy penalty term and further optimize the cluster structure. The calculation method for the change amplitude can be flexibly chosen. For example, the absolute value of the difference in the mixing ratio between two adjacent iterations can be calculated, and this absolute value can be used directly as a measure of the change amplitude; alternatively, the average amplitude of the change in the mixing ratio over multiple iterations can be calculated to reflect the overall fluctuation of the cluster's competitiveness. The adjustment of the adjustment parameter can be based on the threshold of the change range. For example, when the change range is greater than the preset threshold, the adjustment parameter is lowered to a reasonable range; when the change range is less than the preset threshold, the adjustment parameter is raised to the corresponding range. Alternatively, a mapping relationship between the change range and the adjustment parameter can be established through a dynamic function, allowing the adjustment parameter to be adaptively adjusted in real time according to the change range.

[0112] In some alternative implementations, the method for adaptively adjusting hyperparameters during unsupervised clustering iterations may include at least one of the following:

[0113] (1) Penalty coefficient Based on the current cluster number Make dynamic adjustments: .

[0114] (2) Penalty coefficient Based on the mixing ratio between successive iterations The change range is dynamically adjusted, and the strategy is as follows: ,in This is the attenuation factor.

[0115] In the behavior anomaly detection method of this application embodiment, hyperparameters are used. and An adaptive adjustment strategy. With cluster number The decrease is dynamically increased. Then according to the mixing ratio The algorithm dynamically adjusts the update stability. This adaptive mechanism encourages competition and exploration in the early stages (when the number of clusters is large) and promotes convergence in the later stages (when the number of clusters is stable), enhancing the algorithm's adaptability to different data sizes and distributions and reducing its dependence on the initial values ​​of hyperparameters.

[0116] Optionally, after establishing the user's normal behavioral baseline, the method further includes:

[0117] Dynamically optimizing the baseline of normal user behavior specifically includes:

[0118] The process of dynamically optimizing the normal behavioral baseline includes:

[0119] A sliding window is used to continuously maintain the relevant statistics of the clusters. The statistics include the count of behavioral feature vectors belonging to each cluster, the cumulative sum of behavioral feature vectors, and the cumulative sum of behavioral feature vectors outside the cluster.

[0120] The continuously generated new security audit data is divided into several micro-batch data according to preset rules, and the micro-batch data is subjected to feature processing to obtain micro-batch behavior feature vector data.

[0121] Based on the micro-batch behavior feature vector data and the maintained cluster statistics, iterative calibration is performed on the cluster centers and mixing ratios, and the cluster structure is updated.

[0122] The normal behavior baseline is optimized based on the calibrated cluster centers, mixing ratios, and updated cluster structure.

[0123] In this implementation, the core of the aforementioned dynamic optimization of the normal behavior baseline is to continuously integrate the features of new data into the existing baseline through a streaming processing mechanism, without retraining the entire model, thus balancing efficiency and adaptability. The specific process is as follows:

[0124] Continuously maintaining cluster-related statistics using a sliding window is fundamental to incremental learning. A sliding window can be understood as a fixed-range data container that retains only recently valid data, automatically discarding outdated data to ensure that statistics reflect the latest behavioral characteristics. The window can be flexibly configured; for example, it can be set by time dimension, with a window length of 12 hours, scrolling forward once per hour to maintain only the statistics for the most recent 12 hours; or it can be set by data volume dimension, with the window containing a fixed 50,000 behavioral feature vectors, scrolling once every 5,000 new data points to discard the statistics corresponding to the oldest 5,000 data points. Cluster-related statistics include the count of behavioral feature vectors belonging to each cluster, the cumulative sum of behavioral feature vectors, and the cumulative sum of behavioral feature vectors outside the cluster. The vector count is the total number of behavioral feature vectors belonging to each cluster; the cumulative sum of vectors is the sum of the values ​​of each dimension of all behavioral feature vectors within each cluster; and the cumulative sum of vectors outside the cluster is the sum of the products of all behavioral feature vectors within each cluster and their transposes. These statistics comprehensively preserve the core information of the cluster, providing data support for subsequent calibration. When maintaining statistics, you can recalculate the statistical values ​​after each window scroll, or receive new data in real time and dynamically update the statistical values ​​to ensure the timeliness of statistical information.

[0125] The continuously generated new security audit data is divided into several micro-batches according to preset rules to achieve batch processing of data, balancing processing efficiency and model update frequency. New security audit data refers to user behavior-related data continuously collected subsequently, with its data source consistent with the initially collected security audit data, ensuring the continuity of data features. Preset rules are the basis for dividing the data into micro-batches. For example, it can be divided by time interval, with each batch of new data generated within a 10-minute interval forming a micro-batch, or by data volume, with each batch consisting of 3,000 new data entries. The feature processing method for the micro-batch data is consistent with that of the initial data. Through operations such as time alignment, cleaning, normalization, and standardization, the original micro-batch data is transformed into structured micro-batch behavioral feature vector data, ensuring that the feature format of the micro-batch data is consistent with historical data and can be directly used for model calibration.

[0126] The core step of dynamic optimization is to perform iterative calibration of cluster centers and mixing ratios based on micro-batch behavioral feature vector data and maintained cluster statistics, and to update the cluster structure. Iterative calibration can be understood as using the new features of micro-batch data to make small adjustments to the existing cluster centers and mixing ratios, gradually adapting them to the latest behavioral patterns, rather than completely reconstructing them. For example, when calibrating cluster centers, the statistics of micro-batch behavioral feature vectors can be merged with historical statistics within the window, and the cluster centers can be recalculated based on the merged statistics, allowing the cluster centers to slowly shift towards the new behavioral features. Alternatively, temporary cluster centers can be calculated separately using micro-batch data, and then the temporary cluster centers can be merged with historical cluster centers according to a certain weight to obtain the calibrated cluster centers. The weights can be set according to the ratio of the amount of micro-batch data to the amount of historical data. Updating the cluster structure allows the baseline to adapt to new behavioral patterns or eliminate outdated patterns. For example, when a certain behavioral feature appears continuously in micro-batch data and cannot be effectively covered by existing clusters, a new candidate cluster is added through a competition mechanism, and a stable cluster is formed after multiple iterations of calibration. When the statistics of an existing cluster continue to decay and the proportion of vector counts is lower than a preset ratio, it is judged as an outdated cluster and eliminated. When the center distance of two clusters is too close after calibration, indicating that the behavioral patterns are highly similar, they are merged into one cluster to ensure the rationality of the cluster structure.

[0127] The ultimate goal of dynamic optimization is to optimize the baseline of normal user behavior based on the calibrated cluster centers, mixing ratios, and updated cluster structure. The optimized baseline retains the core characteristics of historical behavior patterns while incorporating the latest behavioral features reflected in new data, enabling a more accurate depiction of the boundaries of current normal user behavior. For example, the calibrated cluster centers better reflect recent user habits, and the updated cluster structure eliminates outdated behavior patterns that no longer appear and adds stable new behavior patterns. The normal behavior baseline reconstructed based on this information has improved accuracy and adaptability, effectively detecting abnormal behaviors that might have been missed by the old baseline, while reducing false alarms caused by changes in behavior patterns.

[0128] For example, you can refer to Figure 3 Cluster statistics are maintained using a sliding window W and a step size S. After the micro-batch arrives, a small number of iterative calibrations are performed; when new patterns continue to emerge or old clusters decay significantly, clusters are adaptively added / merged / eliminated through a competition mechanism.

[0129] In this implementation, the adaptive clustering algorithm is combined with a sliding window model, utilizing maintained cluster statistics (such as... It enables incremental learning and model updates for streaming data and can handle concept drift.

[0130] In some alternative implementations, user behavior baseline updates in a streaming environment can be based on receiving micro-batch data using a sliding window W and a step size S, and incrementally updating the cluster center using historical cluster statistics. and mixing ratio And when a new pattern is detected to continue to emerge or an old cluster is significantly decayed, the competition mechanism will adaptively add or remove clusters.

[0131] In some alternative implementations, in the anomaly detection method for the adaptive clustering results, new samples can be calculated. Competitive costs to each behavioral baseline cluster , and its minimum value Compare with the historical cost distribution of the corresponding cluster (such as the 95th percentile), and if it exceeds the value, it is judged as an anomaly.

[0132] In the behavior anomaly detection method of this application embodiment, sufficient statistics of the cluster are maintained ( When new batches of data arrive, the algorithm can perform a few iterative calibrations based on the existing model to quickly update the cluster centers and mixing ratios. Meanwhile, the core competition and elimination mechanism remains effective: if new patterns continue to emerge, new candidate clusters can stand out through competition; if old patterns disappear, the proportion of the corresponding cluster will decay and it will be eliminated, thus achieving dynamic evolution of the baseline.

[0133] Optionally, determining whether the behavior of the user corresponding to the test data is abnormal based on the user's normal behavior baseline includes:

[0134] Perform feature processing on the data to be tested to obtain the feature vector of the behavior to be detected;

[0135] The cost of obtaining the feature vector of the behavior to be detected into each baseline cluster in the user's normal behavior baseline;

[0136] If the minimum cost exceeds the historical cost distribution threshold of the cluster corresponding to the minimum cost, the user's behavior is determined to be abnormal.

[0137] If the minimum cost does not exceed the historical cost distribution threshold of the cluster corresponding to the minimum cost, then the user's behavior is determined to be normal.

[0138] In this implementation, feature processing is performed on the data to be tested to obtain the feature vector of the behavior to be detected, which can serve as a basic prerequisite for anomaly determination. The data to be tested refers to user operation-related data that needs to be verified for behavioral security. It can be real-time user login operation data, or various data related to security auditing such as database access and network traffic. The data source is consistent with the security audit data collected when constructing the behavioral baseline. The feature processing method is exactly the same as the feature processing method used when constructing the user's normal behavior baseline. The purpose is to ensure that the feature vector of the behavior to be detected has the same format and dimension as the feature vector of the cluster in the baseline, thus ensuring the effectiveness of subsequent cost calculation. For example, the login data to be tested can be processed using the same normalization and sequence statistical processing as the initial data to extract feature vectors of dimensions such as login time, device identification, and geographical location. The database operation data to be tested can be processed using standardization and embedding to transform the SQL statement sequence into a low-dimensional dense feature vector, so that the vector to be detected can be directly compared with the baseline cluster.

[0139] Obtaining the cost from the feature vector of the behavior to be detected to each baseline cluster in the user's normal behavior baseline is the core step in measuring the degree of matching between the detected behavior and the normal behavior pattern. This cost can be understood as the degree of difference between the feature vector of the behavior to be detected and each baseline cluster. The smaller the cost, the more similar the detected behavior is to the normal behavior pattern corresponding to that baseline cluster; the larger the cost, the more significant the difference. The method of calculating the cost can be flexibly chosen. For example, it can be calculated by combining the distance from the feature vector of the behavior to be detected to the center of the baseline cluster with the logarithm of the mixing ratio of that cluster, considering both the absolute difference in the vector space and the competitiveness weight of the baseline cluster; alternatively, it can be calculated based on the fit between the detected vector and the statistical information within the baseline cluster, incorporating information such as cluster radius and cluster density, so that the cost better reflects the true matching situation. In actual calculation, all baseline clusters in the user's normal behavior baseline can be traversed, and the cost between the detected vector and each cluster can be calculated separately to ensure that no possible normal behavior pattern matching items are missed.

[0140] If the minimum cost exceeds the historical cost distribution threshold of the cluster corresponding to the minimum cost, the user's behavior can be determined to be abnormal. The minimum cost is the smallest cost among all baseline clusters of the vector to be detected. The corresponding cluster is the normal behavior pattern cluster most similar to the behavior to be detected. The historical cost distribution threshold of this cluster is a critical value obtained based on the cost statistics of all past normal behavior feature vectors of this cluster, used to define the cost boundary of normal behavior. The setting of the historical cost distribution threshold can be adjusted according to the detection requirements. For example, the 95th percentile of the historical cost of this cluster can be selected as the threshold, which is suitable for scenarios with moderate requirements for anomaly detection sensitivity; or the 90th percentile can be selected as the threshold, which is suitable for scenarios that require strict control of the false alarm rate. For example, if the minimum cost of the vector to be detected and a certain baseline cluster is 0.75, and the historical cost distribution threshold of this cluster is 0.6, then the minimum cost exceeds the threshold, indicating that the behavior to be detected differs too much from the most similar normal pattern, and is judged as abnormal; if the minimum cost is 0.5, which does not exceed the threshold, then the behavior to be detected conforms to the normal behavior pattern, and is judged as normal. This method, which compares the minimum cost with historical thresholds, can accurately filter out abnormal behaviors that deviate from the baseline of normal behavior, while minimizing misjudgments caused by single-pattern matching bias.

[0141] In this implementation, the output is a defined cluster of behavioral baselines with statistical characteristics (center, radius, etc.). During the detection phase, the competitive cost of new samples to each baseline is calculated. Once identified as an anomaly, its nearest normal baseline cluster can be immediately located. (corresponding clusters), and by comparing the differences between outliers and the baseline center in each feature dimension, the main deviation features are identified, providing security analysts with intuitive and actionable diagnostic information.

[0142] In some alternative implementations, a complete system integrating data acquisition, feature processing, adaptive clustering engine, baseline management, and anomaly detection services can be constructed. This system may include:

[0143] (1) Data acquisition and feature processing module, used to acquire and preprocess multi-source security audit log data;

[0144] (2) An adaptive clustering engine module, used to load and run the objective function J and the update rules, and output the user behavior baseline cluster;

[0145] (3) Streaming incremental processing module, used to implement the streaming incremental learning method;

[0146] (4) Anomaly detection and service module, used to implement the anomaly detection method and provide services to the outside world through API interface.

[0147] In the behavior anomaly detection method of this application embodiment, the accuracy and consistency of anomaly detection results can be improved by standardized feature processing, more accurate cost calculation and reasonable threshold comparison, providing a more reliable basis for judgment for security audit.

[0148] In some alternative implementations, the dispersion of the cluster distribution can be measured or used as a penalty term using a mathematical form different from Shannon entropy, for example:

[0149] Using Gini impurity: Replace the penalty term with .

[0150] Using Tsallis entropy: using a more generalized form of entropy. , where q is a real parameter.

[0151] Using L1 regularization: Introducing a feature on the mixing ratio L1 norm penalty By inducing sparsity, the proportion of weak clusters is forced to zero.

[0152] In some alternative implementations, the overall approach to constructing the objective function can also be changed:

[0153] The framework of "distance cost + entropy penalty" can be abandoned, and methods for automatically determining the number of clusters under other theoretical systems can be adopted. For example, a Bayesian nonparametric model (such as the Dirichlet process hybrid model) can be used: this model introduces a Dirichlet process as a prior, theoretically allowing for an infinite number of clusters, but only a finite number of clusters are supported by data in the posterior inference, thus achieving automatic determination of the number of clusters. Its mathematical foundation is based on probabilistic generative models and Bayesian inference, which is completely different from the idea of ​​optimizing the objective function in this invention.

[0154] Alternatively, the minimum description length principle can be used: construct an optimization objective that minimizes both the description length of the model (cluster structure) and the description length of the data given the model. The number of clusters is determined by the model that minimizes the total description length.

[0155] In addition, in some alternative implementations, the following methods can be chosen as alternatives to the competition and elimination mechanism implementation path:

[0156] Two-stage iteration can be used instead of joint optimization:

[0157] By separating the "competitive elimination" process from the "cluster center update" process, an algorithm is designed to alternately execute clustering and cluster management. For example, the following steps can be executed cyclically until stable:

[0158] (a) Clustering with a fixed number of clusters: With the current number of clusters c, run the standard k-means algorithm until convergence.

[0159] (b) Evaluate and prune clusters: based on some independent criterion (such as cluster size). Clusters that do not meet the criteria (such as excessive intra-cluster variance or excessively close centroid distance to the nearest cluster) will be deleted.

[0160] (c) Optional cluster merging: Merge clusters that are too close together based on the inter-cluster distance criterion.

[0161] The continuous parameter competition process can be decomposed into a discrete clustering-decision loop.

[0162] An evolutionary computation framework can also be used:

[0163] The number and center of clusters can be encoded as chromosomes, and a global search can be performed using an evolutionary algorithm, for example:

[0164] A genetic algorithm can be used, whose fitness function considers both clustering quality (such as silhouette coefficient and Davidson-Bolding index) and model complexity (such as the penalty for the number of clusters c). The algorithm evolves the population through multiple generations via selection, crossover, and mutation operations, ultimately obtaining a solution that balances clustering quality and model simplicity. This approach completely avoids gradient descent-based iterative updates.

[0165] Furthermore, in some alternative implementations, the system architecture and module implementation can be modified as follows:

[0166] It can replace the streaming incremental update module. After obtaining the initial baseline generated based on the algorithm of this patent, it can use other incremental learning algorithms for updating, thus avoiding the use of the statistical maintenance and update rules described in the patent.

[0167] Specifically, it can interface with a standalone streaming clustering algorithm, such as StreamKM++, or use an alternative concept of drift detection and adaptation based on sampling, hashing, or decaying windows.

[0168] In some alternative implementations, the anomaly detection and judgment logic can be replaced, utilizing the clustering results (baseline) output by the patented algorithm, but with a different method for anomaly scoring, for example:

[0169] For new samples Calculate its distance to all cluster centers and directly use the shortest distance. As an outlier score, rather than a competitive cost in a patent. .

[0170] Alternatively, the clustering results can be used as features and input into a separate anomaly detection model (such as Isolation Forest or Single-Class SVM) for judgment.

[0171] In some alternative implementations, reference may be made to Figure 4 The overall system architecture can include: a data acquisition layer, a feature layer, a clustering engine, a baseline and anomaly service layer, and a visualization and API. The UK-means-SA algorithm flow can be found by referring to... Figure 3 The process includes: initialization → competitive update → cluster elimination → convergence → baseline output.

[0172] For an example of anomaly detection and interpretable output, please refer to [link / reference]. Figure 5 This can include nearest-neighbor baseline, feature contribution, and temporal evolution.

[0173] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a behavior anomaly detection device 600 provided in an embodiment of this application. As shown in the figure, the behavior anomaly detection device 600 includes:

[0174] The acquisition module 610 is used to collect the user's security audit data, and after performing feature processing on the security audit data, obtain a set of behavioral feature vectors.

[0175] The determination module 620 is used to obtain the initial number of candidate clusters and hard attribution variables, and to determine the center vector and initial mixing ratio of each initial candidate cluster. The hard attribution variables represent the relationship between the behavioral feature vector and the cluster attribution.

[0176] The update module 630 is used to iteratively update the hard attribution variable, center vector and mixing ratio based on the clustering objective function, dynamically eliminate clusters with a mixing ratio less than a preset threshold until the convergence condition is met, and construct a baseline for normal user behavior. The clustering objective function includes at least one of a distance cost term, two types of entropy penalty terms and an entropy-driven competitive adjustment term.

[0177] The judgment module 640 is used to determine whether the behavior of the user corresponding to the test data is abnormal based on the user's normal behavior baseline, and output the judgment result.

[0178] Optionally, the feature processing includes preprocessing and feature engineering; the preprocessing includes time alignment and data cleaning; and the feature engineering includes at least one of normalization, standardization, embedding, sequence statistics, and matrix factorization.

[0179] Optionally, the two types of entropy penalty terms include a first entropy penalty term and a second entropy penalty term. The first entropy penalty term is constructed based on the distribution characteristics of the cluster mixing ratio, and the second entropy penalty term is constructed based on the association characteristics between the behavioral feature vector and the cluster mixing ratio.

[0180] And / or,

[0181] The distance cost term characterizes the compactness of the data within the cluster with respect to the cluster center.

[0182] Optionally, the update module 630 can also be used for:

[0183] Based on the distance between the behavioral feature vector and the center vector corresponding to each candidate cluster, and the correlation between the mixing ratio of each candidate cluster, the cluster to which each behavioral feature vector uniquely belongs is determined, and the hard attribution variable is updated.

[0184] Update the center vector of the corresponding cluster based on the mean of all behavioral feature vectors belonging to each cluster;

[0185] By combining the proportion of behavioral feature vectors belonging to each cluster to the total number, and the competition adjustment term in the clustering objective function, a new mixing ratio for each cluster is obtained, and the new mixing ratio is normalized.

[0186] Clusters whose normalized mixing ratio is less than a preset threshold or are invalid are removed. The current cluster count is updated based on the number of removed clusters. Normalization is then performed on the mixing ratio and hard attribution variable of the remaining clusters.

[0187] Optionally, the convergence condition is that the maximum value of the change in the cluster centers during the iteration process is less than a preset convergence threshold;

[0188] The baseline of normal user behavior includes the cluster center and intra-cluster statistics after iterative convergence. The intra-cluster statistics include at least one of cluster radius, cluster density, and intra-cluster characteristic temporal distribution.

[0189] Optionally, the entropy penalty term includes an adjustment parameter, wherein the adjustment parameter of the first entropy penalty term is dynamically adjusted according to the current number of clusters; and the adjustment parameter of the second entropy penalty term is dynamically adjusted according to the change in the mixing ratio during iteration.

[0190] Optionally, the abnormal behavior detection device 600 can also be used for:

[0191] Dynamically optimizing the baseline of normal user behavior specifically includes:

[0192] The process of dynamically optimizing the normal behavioral baseline includes:

[0193] A sliding window is used to continuously maintain the relevant statistics of the clusters. The statistics include the count of behavioral feature vectors belonging to each cluster, the cumulative sum of behavioral feature vectors, and the cumulative sum of behavioral feature vectors outside the cluster.

[0194] The continuously generated new security audit data is divided into several micro-batch data according to preset rules, and the micro-batch data is subjected to feature processing to obtain micro-batch behavior feature vector data.

[0195] Based on the micro-batch behavior feature vector data and the maintained cluster statistics, iterative calibration is performed on the cluster centers and mixing ratios, and the cluster structure is updated.

[0196] The normal behavior baseline is optimized based on the calibrated cluster centers, mixing ratios, and updated cluster structure.

[0197] Optionally, the determination module 640 can also be used for:

[0198] Perform feature processing on the data to be tested to obtain the feature vector of the behavior to be detected;

[0199] The cost of obtaining the feature vector of the behavior to be detected into each baseline cluster in the user's normal behavior baseline;

[0200] If the minimum cost exceeds the historical cost distribution threshold of the cluster corresponding to the minimum cost, the user's behavior is determined to be abnormal.

[0201] If the minimum cost does not exceed the historical cost distribution threshold of the cluster corresponding to the minimum cost, then the user's behavior is determined to be normal.

[0202] For details, see Figure 7 As shown in the figure, this application embodiment also provides an electronic device, including a bus 701, a transceiver 702, an antenna 703, a bus interface 704, a processor 705, and a memory 706.

[0203] Processor 705, used for:

[0204] Collect user security audit data, and after performing feature processing on the security audit data, obtain a set of behavioral feature vectors;

[0205] Obtain the initial number of candidate clusters and hard attribution variables, determine the center vector and initial mixing ratio of each initial candidate cluster, and the hard attribution variables represent the relationship between the behavioral feature vector and the cluster attribution;

[0206] Based on the clustering objective function, the hard attribution variable, center vector and mixing ratio are iteratively updated, and clusters with mixing ratios less than a preset threshold are dynamically eliminated until the convergence condition is met, and a baseline of normal user behavior is constructed. The clustering objective function includes at least one of the following: distance cost term, two types of entropy penalty term and entropy-driven competitive adjustment term.

[0207] Based on the baseline of normal user behavior, determine whether the behavior of the user corresponding to the test data is abnormal, and output the determination result.

[0208] exist Figure 7In this document, a bus architecture (represented by bus 701) is used. Bus 701 can include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 705 and memory represented by memory 706. Bus 701 can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 706 provides an interface between bus 701 and transceiver 702. Transceiver 702 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 705 is transmitted over a wireless medium via antenna 703, which further receives data and transmits data to processor 705.

[0209] Processor 705 manages bus 701 and general processing, and also provides various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Memory 706 can be used to store data used by processor 705 during operation.

[0210] Alternatively, the processor 705 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0211] Optionally, the feature processing includes preprocessing and feature engineering; the preprocessing includes time alignment and data cleaning; and the feature engineering includes at least one of normalization, standardization, embedding, sequence statistics, and matrix factorization.

[0212] Optionally, the two types of entropy penalty terms include a first entropy penalty term and a second entropy penalty term. The first entropy penalty term is constructed based on the distribution characteristics of the cluster mixing ratio, and the second entropy penalty term is constructed based on the association characteristics between the behavioral feature vector and the cluster mixing ratio.

[0213] And / or,

[0214] The distance cost term characterizes the compactness of the data within the cluster with respect to the cluster center.

[0215] Optionally, the processor 705 can also be used for:

[0216] Based on the distance between the behavioral feature vector and the center vector corresponding to each candidate cluster, and the correlation between the mixing ratio of each candidate cluster, the cluster to which each behavioral feature vector uniquely belongs is determined, and the hard attribution variable is updated.

[0217] Update the center vector of the corresponding cluster based on the mean of all behavioral feature vectors belonging to each cluster;

[0218] By combining the proportion of behavioral feature vectors belonging to each cluster to the total number, and the competition adjustment term in the clustering objective function, a new mixing ratio for each cluster is obtained, and the new mixing ratio is normalized.

[0219] Clusters whose normalized mixing ratio is less than a preset threshold or are invalid are removed. The current cluster count is updated based on the number of removed clusters. Normalization is then performed on the mixing ratio and hard attribution variable of the remaining clusters.

[0220] Optionally, the convergence condition is that the maximum value of the change in the cluster centers during the iteration process is less than a preset convergence threshold;

[0221] The baseline of normal user behavior includes the cluster center and intra-cluster statistics after iterative convergence. The intra-cluster statistics include at least one of cluster radius, cluster density, and intra-cluster characteristic temporal distribution.

[0222] Optionally, the entropy penalty term includes an adjustment parameter, wherein the adjustment parameter of the first entropy penalty term is dynamically adjusted according to the current number of clusters; and the adjustment parameter of the second entropy penalty term is dynamically adjusted according to the change in the mixing ratio during iteration.

[0223] Optionally, the processor 705 can also be used for:

[0224] Dynamically optimizing the baseline of normal user behavior specifically includes:

[0225] The process of dynamically optimizing the normal behavioral baseline includes:

[0226] A sliding window is used to continuously maintain the relevant statistics of the clusters. The statistics include the count of behavioral feature vectors belonging to each cluster, the cumulative sum of behavioral feature vectors, and the cumulative sum of behavioral feature vectors outside the cluster.

[0227] The continuously generated new security audit data is divided into several micro-batch data according to preset rules, and the micro-batch data is subjected to feature processing to obtain micro-batch behavior feature vector data.

[0228] Based on the micro-batch behavior feature vector data and the maintained cluster statistics, iterative calibration is performed on the cluster centers and mixing ratios, and the cluster structure is updated.

[0229] The normal behavior baseline is optimized based on the calibrated cluster centers, mixing ratios, and updated cluster structure.

[0230] Optionally, the processor 705 can also be used for:

[0231] Perform feature processing on the data to be tested to obtain the feature vector of the behavior to be detected;

[0232] The cost of obtaining the feature vector of the behavior to be detected into each baseline cluster in the user's normal behavior baseline;

[0233] If the minimum cost exceeds the historical cost distribution threshold of the cluster corresponding to the minimum cost, the user's behavior is determined to be abnormal.

[0234] If the minimum cost does not exceed the historical cost distribution threshold of the cluster corresponding to the minimum cost, then the user's behavior is determined to be normal.

[0235] It should be noted that the electronic device provided in this application embodiment is a device capable of executing the above-described behavior anomaly detection method. Therefore, all implementation methods in the above-described behavior anomaly detection method embodiments are applicable to this electronic device and can achieve the same or similar beneficial effects. To avoid repetition, this embodiment will not elaborate further.

[0236] This invention also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the various processes of the above-described abnormal behavior detection method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0237] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described abnormal behavior detection method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0238] This application also provides a computer program product, including computer instructions. When executed by a processor, the computer instructions implement the various processes of the above-described abnormal behavior detection method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0239] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0240] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0241] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for detecting abnormal behavior, characterized in that, The method includes: Collect user security audit data, and after performing feature processing on the security audit data, obtain a set of behavioral feature vectors; Obtain the initial number of candidate clusters and hard attribution variables, determine the center vector and initial mixing ratio of each initial candidate cluster, and the hard attribution variables represent the relationship between the behavioral feature vector and the cluster attribution; Based on the clustering objective function, the hard attribution variable, center vector and mixing ratio are iteratively updated, and clusters with mixing ratios less than a preset threshold are dynamically eliminated until the convergence condition is met, and a baseline of normal user behavior is constructed. The clustering objective function includes at least one of the following: distance cost term, two types of entropy penalty term and entropy-driven competitive adjustment term. Based on the baseline of normal user behavior, determine whether the behavior of the user corresponding to the test data is abnormal, and output the determination result.

2. The method according to claim 1, characterized in that, The feature processing includes preprocessing and feature engineering; the preprocessing includes time alignment and data cleaning; and the feature engineering includes at least one of normalization, standardization, embedding, sequence statistics, and matrix factorization.

3. The method according to claim 2, characterized in that, The two types of entropy penalty terms include a first entropy penalty term and a second entropy penalty term. The first entropy penalty term is constructed based on the distribution characteristics of the cluster mixing ratio, and the second entropy penalty term is constructed based on the association characteristics between the behavioral feature vector and the cluster mixing ratio. And / or, The distance cost term characterizes the compactness of the data within the cluster with respect to the cluster center.

4. The method according to any one of claims 1 to 3, characterized in that, The clustering objective function, based on a distance cost term and two types of entropy penalty terms, iteratively updates the hard attribution variable, center vector, and mixing ratio using an entropy-driven competitive adjustment term, dynamically eliminating clusters with a mixing ratio less than a preset threshold until the convergence condition is met. This includes: Based on the distance between the behavioral feature vector and the center vector corresponding to each candidate cluster, and the correlation between the mixing ratio of each candidate cluster, the cluster to which each behavioral feature vector uniquely belongs is determined, and the hard attribution variable is updated. Update the center vector of the corresponding cluster based on the mean of all behavioral feature vectors belonging to each cluster; By combining the proportion of behavioral feature vectors belonging to each cluster to the total number, and the competition adjustment term in the clustering objective function, a new mixing ratio for each cluster is obtained, and the new mixing ratio is normalized. Clusters whose normalized mixing ratio is less than a preset threshold or are invalid are removed. The current cluster count is updated based on the number of removed clusters. Normalization is then performed on the mixing ratio and hard attribution variable of the remaining clusters.

5. The method according to any one of claims 1 to 3, characterized in that, The convergence condition is that the maximum value of the change in the cluster center during the iteration process is less than a preset convergence threshold. The baseline of normal user behavior includes the cluster center and intra-cluster statistics after iterative convergence. The intra-cluster statistics include at least one of cluster radius, cluster density, and intra-cluster characteristic temporal distribution.

6. The method according to claim 3, characterized in that, The entropy penalty term includes adjustment parameters. The adjustment parameters of the first entropy penalty term are dynamically adjusted according to the current number of clusters. The adjustment parameters of the second entropy penalty term are dynamically adjusted according to the change of the mixing ratio during iteration.

7. The method according to any one of claims 1 to 3, characterized in that, After establishing the baseline of the user's normal behavior, the following is also included: Dynamically optimizing the baseline of normal user behavior specifically includes: The process of dynamically optimizing the normal behavioral baseline includes: A sliding window is used to continuously maintain the relevant statistics of the clusters. The statistics include the count of behavioral feature vectors belonging to each cluster, the cumulative sum of behavioral feature vectors, and the cumulative sum of behavioral feature vectors outside the cluster. The continuously generated new security audit data is divided into several micro-batch data according to preset rules, and the micro-batch data is subjected to feature processing to obtain micro-batch behavior feature vector data. Based on the micro-batch behavior feature vector data and the maintained cluster statistics, iterative calibration is performed on the cluster centers and mixing ratios, and the cluster structure is updated. The normal behavior baseline is optimized based on the calibrated cluster centers, mixing ratios, and updated cluster structure.

8. The method according to any one of claims 1 to 3, characterized in that, The step of determining whether the behavior of the user corresponding to the test data is abnormal based on the user's normal behavior baseline includes: Perform feature processing on the data to be tested to obtain the feature vector of the behavior to be detected; The cost of obtaining the feature vector of the behavior to be detected into each baseline cluster in the user's normal behavior baseline; If the minimum cost exceeds the historical cost distribution threshold of the cluster corresponding to the minimum cost, the user's behavior is determined to be abnormal. If the minimum cost does not exceed the historical cost distribution threshold of the cluster corresponding to the minimum cost, then the user's behavior is determined to be normal.

9. A behavioral anomaly detection device, characterized in that, include: The acquisition module is used to collect the user's security audit data, and after performing feature processing on the security audit data, obtain a set of behavioral feature vectors. The determination module is used to obtain the initial number of candidate clusters and hard attribution variables, determine the center vector and initial mixing ratio of each initial candidate cluster, and the hard attribution variables represent the relationship between the behavioral feature vector and the cluster attribution; The update module is used to iteratively update the hard attribution variable, center vector and mixing ratio based on the clustering objective function, dynamically eliminate clusters with a mixing ratio less than a preset threshold until the convergence condition is met, and construct a baseline for normal user behavior. The clustering objective function includes at least one of a distance cost term, two types of entropy penalty terms and an entropy-driven competitive adjustment term. The judgment module is used to determine whether the behavior of the user corresponding to the test data is abnormal based on the user's normal behavior baseline, and output the judgment result.

10. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the behavior anomaly detection method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the behavior anomaly detection method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the behavior anomaly detection method as described in any one of claims 1 to 8.

Images