Abnormal access detection method for data and electronic device

Abstract

This specification provides a method and electronic device for detecting abnormal access to data. The method includes: in response to a user's access behavior to target data, acquiring a sample of the user's access data to the target data, and extracting the user's access behavior features to the target data from the access data sample; determining a distribution type corresponding to the feature values ​​of the access behavior features; wherein the distribution type represents the type of probability distribution of the feature values ​​of the access behavior features in the user's historical access data samples to the target data; and performing anomaly detection on the feature values ​​of the access behavior features based on a statistical analysis method adapted to the distribution type to determine whether the access behavior poses a risk.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of network security, and more particularly to a method and electronic device for detecting abnormal access to data. Background Technology

[0002] In the field of information technology, security monitoring and anomaly detection of access to and transmission of high-value data involving user privacy (such as user credit data) are core aspects of ensuring data security and meeting legal and regulatory compliance requirements.

[0003] In practical applications, such data can usually be exchanged between organizations using secure channels such as the SSH file transfer protocol (SFTP). Although such protocols provide basic encrypted transmission capabilities, they still have inherent limitations in key lifecycle management and the granularity of operational behavior auditing. Summary of the Invention

[0004] In view of the above, one or more embodiments of this specification provide the following technical solutions: According to a first aspect of one or more embodiments of this specification, a method for detecting abnormal access to data is proposed, comprising: In response to a user’s access behavior to target data, a sample of the user’s access data to the target data is obtained, and the user’s access behavior features to the target data are extracted from the access data sample. Determine the distribution type corresponding to the feature value of the access behavior feature; wherein, the distribution type is used to represent the type of probability distribution of the feature value of the access behavior feature in the historical access data sample of the user for the target data; Anomaly detection is performed on the feature values ​​of the access behavior characteristics based on a statistical analysis method adapted to the distribution type to determine whether the access behavior poses a risk.

[0005] Optionally, determining the distribution type corresponding to the feature values ​​of the access behavior characteristics includes: Obtain the user feature vector corresponding to the user; wherein, the user feature vector is used to describe the behavioral pattern of the user's historical access behavior to the target data; the user feature vector includes multiple preset dimensions of historical access behavior features, and the distribution type corresponding to the feature values ​​of each dimension of historical access behavior features respectively; The access behavior features are sequentially matched with the historical access behavior features of the multiple dimensions contained in the user feature vector; If the access behavior feature matches any dimension of the target historical access behavior feature contained in the user feature vector, the distribution type corresponding to the feature value of the target historical access behavior feature contained in the user feature vector is determined as the distribution type corresponding to the feature value of the access behavior feature.

[0006] Optionally, the user feature vector is generated in the following manner: Obtain a sample set of historical access data of the user for the target data; Multiple dimensions of historical access behavior features are extracted from the historical access data samples contained in the historical access data sample set, and statistical analysis is performed on the feature values ​​of the extracted multiple dimensions of historical access behavior features to obtain the distribution types corresponding to the feature values ​​of the multiple dimensions of historical access behavior features. The user feature vector is generated based on the distribution type corresponding to the feature values ​​of the historical access behavior features of the multiple dimensions.

[0007] Optionally, statistical analysis is performed on the feature values ​​of the extracted historical access behavior features to obtain the distribution type corresponding to the feature values ​​of the historical access behavior features, including: The feature values ​​of the extracted historical access behavior features are calculated relative to the distribution index values ​​of each of the preset multiple distribution types; where different distribution types correspond to different distribution index values. If the feature value of the historical access behavior feature reaches the distribution threshold corresponding to any target distribution type among the preset multiple distribution types, then the target distribution type is determined as the distribution type corresponding to the feature value of the historical access behavior feature.

[0008] Optionally, the user feature vector further includes statistical indicator values ​​corresponding to the distribution type; wherein, the statistical indicator values ​​include statistical indicator values ​​obtained by statistically analyzing the feature values ​​of the historical access behavior characteristics based on a statistical analysis method adapted to the distribution type; The user feature vector is generated based on the distribution type corresponding to the feature values ​​of the historical access behavior features of the multiple dimensions, including: Based on the statistical analysis method adapted to the distribution type, the feature values ​​of the historical access behavior characteristics are statistically analyzed to obtain the statistical index values ​​corresponding to the distribution type. The user feature vector is generated based on the distribution types corresponding to the feature values ​​of the historical access behavior features of the multiple dimensions, and the statistical index values ​​corresponding to the distribution types.

[0009] Optionally, anomaly detection is performed on the feature values ​​of the access behavior characteristics based on a statistical analysis method adapted to the distribution type to determine whether the access behavior poses a risk, including: Obtain the statistical index value corresponding to the distribution type contained in the user feature vector; The statistical indicator value is used as an anomaly threshold and compared with the feature value of the access behavior feature to determine whether the feature value of the access behavior feature reaches the anomaly threshold; if so, the access behavior is determined to be risky; or, the statistical indicator value and the feature value of the access behavior feature are used as calculation parameters to calculate the anomaly benchmark value corresponding to the anomaly detection method adapted to the distribution type, and it is determined whether the anomaly benchmark value reaches the anomaly threshold corresponding to the distribution type; if so, the access behavior is determined to be risky.

[0010] Optionally, the access behavior characteristics include access behavior indicators; The access behavior metrics include any one or more combinations shown below: The size of the target data accessed by the user, the access time of the user accessing the target data, the access duration of the user accessing the target data, the number of times the user accesses the target data, and the interval between user accesses the target data.

[0011] Optionally, the preset multiple distribution types include zero-inflation distribution, heavy-tailed distribution, and overly dispersed distribution; The distribution index value corresponding to the zero-inflation distribution includes the proportion of zero values; the distribution threshold corresponding to the zero-inflation distribution includes the zero-value proportion threshold. The distribution index value corresponding to the heavy-tailed distribution includes skewness; the distribution threshold corresponding to the heavy-tailed distribution includes the skewness threshold; The distribution index values ​​corresponding to the overdispersive distribution include the coefficient of variation; the ratio of variance to mean; the distribution threshold corresponding to the overdispersive distribution includes the coefficient of variation threshold; the coefficient of variation is expressed as the ratio of variance to mean.

[0012] Optionally, statistical analysis is performed on the feature values ​​of the extracted historical access behavior features to obtain the distribution type corresponding to the feature values ​​of the historical access behavior features, including: Calculate the proportion of zero values, skewness, and coefficient of variation of the extracted historical access behavior features; If the zero-value ratio reaches the zero-value ratio threshold, the distribution type corresponding to the feature value of the historical access behavior feature will be determined as a zero-inflation distribution. If the skewness reaches the skewness threshold, the distribution type corresponding to the feature value of the historical access behavior feature will be determined as a heavy-tailed distribution. If the discrete coefficient reaches the discrete coefficient threshold, the distribution type corresponding to the feature value of the historical access behavior feature is determined to be an overdispersive distribution.

[0013] Optionally, the anomaly detection method corresponding to the zero-inflated distribution includes zero-value compensation detection; the statistical index value corresponding to the zero-inflated distribution includes a specified quantile of non-zero values; the anomaly detection method corresponding to the heavy-tailed distribution includes Z-score detection based on log space; the statistical index value corresponding to the heavy-tailed distribution includes the mean and standard deviation; the anomaly detection method corresponding to the overdispersive distribution includes CDF detection based on the negative binomial distribution; the statistical index value corresponding to the overdispersive distribution includes the shape parameter of the negative binomial distribution and the success probability of the negative binomial distribution. Anomaly detection is performed on the feature values ​​of the access behavior characteristics based on a statistical analysis method adapted to the distribution type to determine whether the access behavior poses a risk, including: If the distribution type corresponding to the feature value of the access behavior feature is a zero-inflated distribution, obtain the specified quantile of the non-zero value corresponding to the zero-inflated distribution contained in the user feature vector; determine whether the feature value of the access behavior feature is non-zero; if the feature value of the access behavior feature is non-zero, further compare the feature value of the access behavior feature with the specified quantile of the non-zero value to determine whether the feature value of the access behavior feature reaches the specified quantile of the non-zero value; if yes, determine that the access behavior has a risk; otherwise, determine that the access behavior does not have a risk. If the distribution type corresponding to the feature value of the access behavior feature is a heavy-tailed distribution, obtain the mean and standard deviation corresponding to the heavy-tailed distribution contained in the user feature vector; perform a logarithmic transformation on the feature value of the access behavior feature, and use the mean, the standard deviation, and the logarithmically transformed feature value of the access behavior feature as calculation parameters to calculate the Z-score, and determine whether the Z-score reaches the anomaly threshold corresponding to the heavy-tailed distribution; if yes, determine that the access behavior has a risk; otherwise, determine that the access behavior does not have a risk. If the distribution type corresponding to the feature value of the access behavior feature is an overdispersive distribution, obtain the shape parameter and success probability of the negative binomial distribution corresponding to the overdispersive distribution contained in the user feature vector, and input the shape parameter, success probability, and feature value of the access behavior feature as calculation parameters into the CDF function to calculate the anomaly probability corresponding to the feature value of the access behavior feature; and determine whether the anomaly probability reaches the anomaly threshold corresponding to the overdispersive distribution; if yes, determine that the access behavior has a risk; otherwise, determine that the access behavior does not have a risk.

[0014] Optionally, the method further includes: If an anomaly detection is performed on the feature value of the access behavior feature based on the anomaly detection method corresponding to the distribution type, and it is determined that the access behavior does not pose a risk, then it is further detected whether the target data is target data of a specified type, and whether the user's access time for the target data is within the legal access time range corresponding to the target data of the specified type. If the target data is not of the specified type, it is determined that the access behavior does not pose a risk. If the target data is of a specified type, and the user's access time to the target data is within the legal access time range corresponding to the specified type of target data, it is determined that the access behavior does not pose a risk. If the target data is of a specified type, and the user's access time to the target data is not within the legal access time range corresponding to the specified type of target data, the access behavior is determined to be risky.

[0015] Optionally, the method further includes: If an anomaly detection is performed on the feature value of the access behavior feature based on the anomaly detection method corresponding to the distribution type, and it is determined that the access behavior does not pose a risk, then human-machine behavior recognition is further performed on the user's access behavior to the target data. If the user's access behavior to the target data fails the human-machine behavior recognition, the access behavior is determined to be risky; otherwise, the access behavior is determined to be risk-free.

[0016] Optionally, human-machine behavior recognition is performed on the user's access behavior to the target data, including: Calculate the entropy value of the historical IP addresses used by the user, and count the time interval between the user's successful login and the initiation of access behavior for the target data; If both the entropy value and the interval duration are below the threshold, it is determined that the user's access behavior to the target data has not passed the human-machine behavior recognition; otherwise, it is determined that the user's access behavior to the target data has passed the human-machine behavior recognition.

[0017] Optionally, the target data may include privacy data related to the user.

[0018] Optionally, the target data includes the user's credit data.

[0019] According to a second aspect of one or more embodiments of this specification, an electronic device is also provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor performs the executable instructions to implement the steps of the method as described in any of the first aspects above.

[0020] According to a third aspect of one or more embodiments of this specification, a computer program product is also provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any of the first aspects above.

[0021] In the above embodiments and in the above technical solutions, on the one hand, since the feature values ​​of different users' access behavior characteristics usually have different distribution types, and different distribution types reflect the differences in the behavior patterns of different users when accessing target data; therefore, through this detection method, appropriate statistical analysis methods can be adaptively selected based on different user behavior patterns to perform anomaly detection on the feature values ​​of users' access behavior characteristics, thereby improving the detection accuracy of anomaly detection. On the other hand, since statistical analysis methods can be used to detect anomalies in the feature values ​​of user access behavior, this detection method can generate clear and reliable criteria for anomaly judgment, enhance the interpretability of anomaly detection, and thus meet the requirements of security auditing. Moreover, using this lightweight statistical analysis method to detect anomalies in the feature values ​​of user access behavior can also improve the detection efficiency and reduce the latency of anomaly detection. This allows this detection method to still meet the strict performance requirements of latency and real-time performance in some detection scenarios with high requirements for detection latency and real-time performance. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the architecture of an anomaly detection system provided in an exemplary embodiment.

[0023] Figure 2 This is a flowchart of an exemplary embodiment of a method for detecting abnormal access to data.

[0024] Figure 3 This is a flowchart illustrating real-time anomaly detection of user access behavior, provided as an exemplary embodiment.

[0025] Figure 4 This is an exemplary embodiment of a flowchart for constructing the aforementioned user feature vector offline for a user.

[0026] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment.

[0027] Figure 6 This is a block diagram of an abnormal access detection device for data provided in an exemplary embodiment. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0029] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0030] In related technologies, anomaly detection for access to high-value data involving user privacy (such as user credit data) mainly relies on the following technical approaches: First, a detection method based on static rules.

[0031] This method typically sets a uniform and fixed anomaly detection threshold for all users. When a user's access behavior triggers this threshold, it is considered that the user's access to this type of data is risky.

[0032] For example, the above-mentioned anomaly detection threshold could be an access count threshold. When a user downloads this type of data more than the access count threshold in a single day, it can be determined that the user's access behavior to this type of data is risky.

[0033] However, this approach lacks personalization and cannot distinguish the inherent access behavior patterns of different users to this type of data.

[0034] For example, for active users who have a high frequency of access to this type of data, a fixed anomaly detection threshold may misjudge their normal access behavior as abnormal behavior; while for inactive users who only have a low frequency of access to this type of data, a fixed anomaly detection threshold may misjudge their truly sudden and high-risk access behavior as normal access behavior, thus causing misjudgment.

[0035] Second, detection methods based on general artificial intelligence models.

[0036] This method employs complex artificial intelligence models such as deep learning to automatically learn abnormal access patterns from massive amounts of user access behavior logs, and then uses these learned abnormal access patterns to detect anomalies in user behavior. Compared to detection methods based on static rules, this approach offers higher detection accuracy, but the decision-making process of the artificial intelligence model is opaque, making it a "black box" model that cannot provide interpretable judgment criteria that meet security audit requirements.

[0037] Moreover, in practical applications, abnormal samples are extremely rare in real-world scenarios, leading to severe sample imbalance during model training and making it easy to miss detections. In addition, the inference process of complex artificial intelligence models has high computational overhead, making it difficult to meet the stringent performance requirements of low latency (usually requiring milliseconds) in high-concurrency real-time detection scenarios.

[0038] Third, detection methods based on basic statistics.

[0039] This method calculates statistical indicators such as the mean and standard deviation from users' historical behavior data and makes anomaly detection based on the assumption of a normal distribution. However, in real-world production environments, user behavior data often deviates significantly from the normal distribution assumption, frequently exhibiting complex statistical characteristics such as "heavy tails," "zero inflation," or "overdispersion." Therefore, directly applying detection methods based on basic statistics to such data will lead to a sharp increase in the false alarm rate.

[0040] Furthermore, detection methods based on basic statistics are usually unable to effectively distinguish whether a user's access behavior is a normal human operation or the behavior of an automated script (bot).

[0041] It is evident that the technical solutions used in related technologies for anomaly detection of access to high-value data involving user privacy mainly suffer from problems such as high false positive and false negative rates due to rigid rules, difficulty in meeting compliance audit requirements due to the black-box nature of artificial intelligence models, and inability to achieve real-time response due to computational complexity.

[0042] Therefore, there is an urgent need for an anomaly access detection method that can balance detection accuracy, interpretability, and real-time detection.

[0043] Based on this, this specification proposes a method for detecting anomalies in the feature values ​​of user access behavior characteristics by adaptively selecting appropriate statistical analysis methods based on the distribution type corresponding to the feature values ​​of the user's access behavior characteristics of target data.

[0044] In this anomaly detection method, after a user initiates an access behavior to target data, the system can respond to the user's access behavior to the target data by obtaining a sample of the user's access data to the target data and extracting the user's access behavior features to the target data from the sample of the access data.

[0045] Then, the distribution type corresponding to the feature value of the access behavior feature can be determined; wherein, the distribution type is used to represent the type of probability distribution of the feature value of the access behavior feature in the historical access data sample of the user for the target data; Furthermore, anomaly detection can be performed on the feature values ​​of the above-mentioned access behavior characteristics based on statistical analysis methods adapted to the determined distribution type, in order to determine whether there is any risk in the user's access behavior.

[0046] In the above technical solutions, on the one hand, since the feature values ​​of different users' access behavior characteristics usually have different distribution types, and different distribution types reflect the differences in the behavioral patterns of different users when accessing target data; therefore, through this detection method, appropriate statistical analysis methods can be adaptively selected based on different user behavior patterns to perform anomaly detection on the feature values ​​of users' access behavior characteristics, thereby improving the detection accuracy of anomaly detection. On the other hand, since statistical analysis methods can be used to detect anomalies in the feature values ​​of user access behavior, this detection method can generate clear and reliable criteria for anomaly judgment, enhance the interpretability of anomaly detection, and thus meet the requirements of security auditing. Moreover, using this lightweight statistical analysis method to detect anomalies in the feature values ​​of user access behavior can also improve the detection efficiency and reduce the latency of anomaly detection. This allows this detection method to still meet the strict performance requirements of latency and real-time performance in some detection scenarios with high requirements for detection latency and real-time performance.

[0047] Figure 1 This is a schematic diagram of the architecture of an anomaly detection system provided in an exemplary embodiment. For example... Figure 1As shown, the system may include a server 11, a network 12, and several electronic devices, such as a PC (Personal Computer) 13, a mobile phone 14, etc.

[0048] Server 11 can be a physical server containing an independent host, or it can be a virtual server hosted in a host cluster. During operation, server 11 can run a server-side program to implement the relevant functions of the application. For example, when server 11 runs the server-side program of an anomaly detection system, it can act as the server for that anomaly detection system.

[0049] PC13 and mobile phone 14 are just some of the types of electronic devices that users can use. In reality, users can obviously also use electronic devices such as tablets, laptops, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smartwatches, etc.), etc., and one or more embodiments in this specification do not limit this. During operation, the electronic device can run a client-side program to implement the relevant functions of the application. For example, when the electronic device runs the client program of the anomaly detection system, it can be implemented as a client of the anomaly detection system.

[0050] The client application of the aforementioned anomaly detection system can be launched and run on an electronic device. This client-side program can be a native application installed on the electronic device, or it can be a mini-program, quick app, or other similar form. Alternatively, when using web technologies such as HTML5, the relevant functions can be implemented through a browser-displayed page. This browser can be a standalone browser application or a browser module embedded within some applications.

[0051] As for the network 12 that enables interaction between electronic devices such as PC13 and mobile phone 14 and server 11, communication can be achieved using either wired or wireless networks, depending on the communication methods supported by the respective electronic devices. This specification does not impose any restrictions on this. For example, PC13 can support both wired and wireless communication, so it can use either wired or wireless networks as needed. Mobile phone 14 typically only supports wireless communication, so it can use a wireless network for communication.

[0052] The technical solution of this specification will be described in detail below with reference to the accompanying drawings.

[0053] Please see Figure 2 , Figure 2 This specification illustrates a flowchart of a method for detecting abnormal data access, the method comprising the following execution steps: Step 202: In response to the user's access behavior to the target data, obtain the user's access data sample to the target data, and extract the user's access behavior features to the target data from the access data sample; The aforementioned target data can specifically include high-value digital information involving user privacy that requires security monitoring. For example, in the financial sector, the target data could be personal credit reports, loan records, etc.; in the medical sector, the target data could be electronic health records, etc.; and in a corporate environment, the target data could be the company's core intellectual property or trade secrets, etc.

[0054] The aforementioned access behaviors may include, but are not limited to, operations such as reading, downloading, uploading, modifying, and deleting.

[0055] In this specification, when a user initiates an access to target data in real time, a real-time access data sample of the target data can be obtained in response to the user's access to the target data.

[0056] The aforementioned access data sample specifically refers to structured or semi-structured data that records information related to a complete user access operation. In practical applications, the aforementioned access data sample can be collected in real time or asynchronously from application logs, network traffic logs, and database audit logs.

[0057] The specific content of the aforementioned access data samples will not be further restricted in this specification.

[0058] For example, in some embodiments, a sample of access data may contain the following fields: Unique event ID, user identifier (such as username, account ID), operation timestamp, operation type (such as "file download"), operation object (such as data identifier, file path, database table name, etc.), operation result data (such as the size of the file downloaded this time, the number of records returned by this query), source IP address, session ID, operation time, etc.

[0059] Please see Figure 3 , Figure 3 This document presents a flowchart illustrating a real-time anomaly detection method for user access behavior.

[0060] like Figure 3 As shown in this specification, after obtaining a real-time access data sample of the user for the target data, the user's real-time access behavior characteristics for the target data can be further extracted from the access data sample.

[0061] Specifically, the aforementioned access behavior characteristics can include metrics used to characterize a specific dimension of a user's behavior. In practical applications, multiple dimensions of access behavior characteristics can be extracted from a single access data sample.

[0062] In some embodiments, the above-mentioned access behavior characteristics may specifically include any or a combination of the following: Data scale characteristics; for example, the size of the data accessed by the user (in bytes, megabytes, etc.).

[0063] Temporal characteristics; for example, the access time of user access data (such as the timestamp corresponding to the time of access) and the access duration of user access data (such as the duration from the start of access to the end of access).

[0064] Frequency characteristics: For example, the number of times a user accesses data. It is important to emphasize that the above access counts can be counts at different time granularities, such as "cumulative access counts for the day", "operations per hour", etc.

[0065] Interval characteristics: For example, the time interval between two consecutive accesses to data by a user (such as the time interval between two consecutive accesses of the same type to the same data).

[0066] Of course, in practical applications, the above-mentioned access behavior characteristics may include not only data size characteristics, time sequence characteristics, frequency characteristics, interval characteristics, etc., but also other forms of characteristic types, which will not be listed one by one in this specification.

[0067] For example, in one instance, for user A's download operation targeting data, the sample data accessed might be: {User:“UserA”, Operation:“Download”, File:“report.cred”, Size: 150MB, Time:“2023-10-01 14:30:00”}.

[0068] At this point, the access behavior features that can be extracted from the access data sample can specifically include "file size" and "access time"; where the feature value of file size is 150 (unit: MB), and the feature value of "access time" is 2023-10-01 14:30:00.

[0069] Step 204: Determine the distribution type corresponding to the feature value of the access behavior feature; wherein, the distribution type is used to represent the type of probability distribution of the feature value of the access behavior feature in the historical access data sample of the user for the target data; In some embodiments, in order to perform personalized anomaly detection efficiently, a user feature vector can be pre-constructed for each user.

[0070] Specifically, the aforementioned user feature vector can be used to describe the behavioral patterns of a user's historical access to the aforementioned target data. This user feature vector can serve as a behavioral baseline for the user's access to the aforementioned target data (or it can be referred to as a digital profile of the user's historical access behavior).

[0071] In practical applications, a user feature behavior vector can be constructed by offline statistical analysis of the feature values ​​of historical access behavior characteristics contained in the user's historical access data samples.

[0072] It should be noted that, since the user's historical access behavior patterns to the aforementioned target data may change dynamically over time, the user feature vector constructed for the user can specifically be used to describe the dynamic behavior patterns of the user's historical access behavior to the aforementioned target data.

[0073] Please see Figure 4 , Figure 4 This specification illustrates a flowchart for constructing the aforementioned user feature vector offline for a user.

[0074] In some embodiments, when constructing the aforementioned user feature vector offline for each user, such as Figure 4 As shown, the first step is to obtain a sample set of historical access data for the target data mentioned above. For example, in practical applications, the sample set of historical access data can still be collected in real time or asynchronously from application logs, network traffic logs, and database audit logs.

[0075] Once a set of historical access data samples from users targeting the aforementioned data is obtained, multiple dimensions of historical access behavior features can be extracted from these samples. These extracted dimensions can be flexibly selected based on specific needs; for example, features such as data size, time series, frequency, and interval can be selected simultaneously.

[0076] Then, statistical analysis can be performed on the feature values ​​of the extracted historical access behavior features from multiple dimensions to determine the distribution type of the feature values ​​of the historical access behavior features from the multiple dimensions, so as to obtain the distribution type corresponding to the feature values ​​of the historical access behavior features from the multiple dimensions respectively.

[0077] Finally, the user feature vector can be generated for the user based on the distribution types corresponding to the feature values ​​of the historical access behavior features of the above multiple dimensions, and can serve as the user's behavior baseline. At this time, the user feature vector generated for the user can specifically include the historical access behavior features of the above multiple dimensions, as well as the distribution types corresponding to the feature values ​​of the historical access behavior features of each dimension.

[0078] The aforementioned distribution type is a statistical term, specifically referring to the type of probability distribution that all historical feature values ​​describing a particular user's browsing behavior follow. Different distribution types are typically used to characterize different user behavior patterns.

[0079] For example, in some embodiments, in addition to the common distribution type of normal distribution, the above distribution types may also include zero-inflated distribution, heavy-tailed distribution, and over-dispersed distribution. These distribution types can typically characterize different user behavior patterns.

[0080] The zero-inflation distribution is a type of distribution that can be used to characterize the low-frequency, intermittent data access behavior patterns of users. This distribution is characterized by the fact that, in a sequence of historical observations consisting of the historical observations of a certain dimension of historical behavioral features, the proportion of feature values ​​that are zero exceeds a preset threshold (e.g., >70%). This indicates that the user has not generated effective access behavior in the vast majority of historical statistical periods, and their effective access behavior is sparse and sporadic.

[0081] A heavy-tailed distribution is a type of distribution that can be used to characterize a user's stable but occasional large-scale data access behavior patterns. The characteristic of this distribution is that, in a sequence of historical observations consisting of feature values ​​of a certain dimension of historical behavior, the skewness of the feature values ​​in the sequence exceeds a preset threshold (e.g., >3), indicating a significant right skewness in the data distribution. This means that most observations are concentrated in a relatively small range (near the tail), but there are also a few extreme values ​​that are much greater than the overall trend in the data distribution – “tail” events. This indicates that the user typically performs small-scale operations but occasionally initiates batch or large-scale data accesses far exceeding their usual levels.

[0082] Overdispersive distribution is a type of distribution used to characterize user access intensity and temporal distribution patterns that are highly irregular and volatile. The characteristic of this distribution is that, in an observation sequence consisting of historical observations of feature values ​​of a certain dimension of historical behavior, the coefficient of variation (usually expressed as the ratio of variance to mean) of the feature values ​​in the observation sequence exceeds a preset threshold (e.g., >1.5). This indicates that the actual dispersion of the observations in the sequence far exceeds the expectations of uniform random models such as Poisson processes. It signifies that the user's access behavior lacks stability in frequency, scale, or interval, exhibiting either clustered bursts or long periods of silence, and cannot be predicted using simple average intensity.

[0083] The normal distribution is a type of distribution used to characterize stable and regular data access behavior patterns of users. The characteristics of this distribution are that, in a sequence of historical observations consisting of the feature values ​​of a certain dimension of historical behavior, the feature values ​​in this sequence cannot be classified as any of the aforementioned zero-inflation, heavy-tailed, or overly dispersed distributions. The observations in this sequence are always symmetrically distributed around a central value, and the degree of dispersion can be effectively measured by the mean and standard deviation. This indicates that the user's access behavior follows a relatively predictable and moderately volatile stable pattern in terms of frequency, scale, or time.

[0084] It should be noted that when performing statistical analysis on the feature values ​​of the extracted historical access behavior features from multiple dimensions to determine the distribution type of the feature values, the method of statistical analysis selected for the feature values ​​usually depends on the specific types of the preset distribution types, and the statistical analysis methods used for different distribution types will usually have some differences.

[0085] In some embodiments, for different distribution types, a distribution index value that can reflect the distribution characteristics of this distribution type can be selected as the basis for determining the distribution type.

[0086] In this context, during the statistical analysis of the feature values ​​of the extracted historical access behavior features to determine the distribution type corresponding to the feature values ​​of the historical access behavior features, the distribution index values ​​of the extracted historical access behavior features relative to each of the preset multiple distribution types can be obtained; where different distribution types correspond to different distribution index values. Then, the calculated distribution index value can be compared with the distribution threshold corresponding to each distribution type; the distribution thresholds for different distribution types can also be different. If the feature value of the above-mentioned historical access behavior feature reaches the distribution threshold corresponding to any target distribution type among the preset multiple distribution types, then the target distribution type can be determined as the distribution type corresponding to the feature value of the historical access behavior feature.

[0087] It should be noted that, for different distribution types, the selected distribution index values ​​can specifically include any type of index value that can reflect the distribution characteristics of that distribution type, and no specific restrictions are imposed in this specification.

[0088] In some embodiments, taking the aforementioned preset distribution types, including zero-inflation distribution, heavy-tailed distribution, and overly dispersed distribution, as examples, the distribution index value corresponding to the zero-inflation distribution can specifically be the proportion of zero values, and the distribution threshold corresponding to the zero-inflation distribution can specifically be the zero-value proportion threshold; the distribution index value corresponding to the heavy-tailed distribution can be the skewness, and the distribution threshold corresponding to the heavy-tailed distribution can be the skewness threshold; the distribution index value corresponding to the overly dispersed distribution can be the coefficient of variation, and the distribution threshold corresponding to the overly dispersed distribution can be the coefficient of variation threshold; wherein, the coefficient of variation can usually be represented by the ratio of variance to mean.

[0089] In this case, during the process of statistically analyzing the feature values ​​of the extracted historical access behavior features and determining the distribution type corresponding to the feature values ​​of the above-mentioned historical access behavior features, the proportion of zero values, skewness, and coefficient of dispersion of the feature values ​​of the extracted historical access behavior features can be calculated respectively. Then, the calculated zero-value proportion, skewness, and coefficient of variation are compared with the aforementioned zero-value proportion threshold, skewness threshold, and coefficient of variation threshold, respectively. On the one hand, if the proportion of zero values ​​reaches the threshold of zero value proportion, the distribution type corresponding to the feature value of the historical access behavior characteristic can be determined as a zero-inflation distribution. On the other hand, if the skewness reaches the skewness threshold, the distribution type corresponding to the feature value of the historical access behavior characteristic can be determined as a heavy-tailed distribution. Thirdly, if the aforementioned coefficient of variation reaches the aforementioned threshold, the distribution type corresponding to the feature value of the aforementioned historical access behavior characteristics can be determined as an overdispersive distribution.

[0090] Of course, if none of the above conditions are met, the distribution type corresponding to the feature values ​​of the above-mentioned historical access behavior characteristics can be determined as a normal distribution.

[0091] It should be noted that the logic for determining the distribution type corresponding to the feature values ​​of the aforementioned historical access behavior characteristics, as shown above, can also be executed in a certain priority order.

[0092] In some embodiments, when comparing the calculated zero-value ratio, skewness, and coefficient of variation with the aforementioned zero-value ratio threshold, skewness threshold, and coefficient of variation threshold, it can first be determined whether the aforementioned zero-value ratio has reached the aforementioned zero-value ratio threshold; if the aforementioned zero-value ratio has reached the aforementioned zero-value ratio threshold, the distribution type corresponding to the feature value of the historical access behavior feature can be determined as a zero-inflation distribution. If the proportion of zero values ​​does not reach the threshold of zero values, it is possible to continue to determine whether the skewness has reached the threshold of skewness. If the skewness has reached the threshold of skewness, the distribution type corresponding to the feature value of the historical access behavior characteristic can be determined as a heavy-tailed distribution. If the skewness does not reach the skewness threshold, it can be further determined whether the dispersion coefficient has reached the dispersion coefficient threshold. If the dispersion coefficient has reached the dispersion coefficient threshold, the distribution type corresponding to the feature value of the historical access behavior characteristics can be determined as an overdispersive distribution.

[0093] If none of the above conditions are met, the distribution type corresponding to the feature values ​​of the aforementioned historical access behavior characteristics can be determined as a normal distribution.

[0094] The specific values ​​of the zero-value proportion threshold, skewness threshold, and dispersion coefficient threshold can be flexibly set based on different types of distributions. For example, in one case, the zero-value proportion threshold can be set to 0.7, the skewness threshold can be set to 3, and the dispersion coefficient threshold can be set to 1.5.

[0095] For example, suppose a user's historical "daily download count" data is [0,0,0,1,0,0,0,0,2,0] (10 days in total). In this case, the proportion of zero values ​​in the feature values ​​contained in this sequence is 8 / 10 = 0.8 > 0.7 (the zero-value proportion threshold). Therefore, the distribution type of this "daily download count" feature is determined to be "zero-inflation". This distribution type describes the user's operations as highly irregular and volatile over time. During the vast majority of the observation period (70% of the time in this example), the user is completely silent (0 operations). However, at extremely random and irregular points in time, high-intensity operation bursts (such as 10, 25, or 50 times per hour) are suddenly triggered, and the intensity (peak value) of these bursts varies greatly, ranging from 10 to 50.

[0096] Suppose a user's historical "single download file size" data is [2, 1, 5, 1, 120, 3, 2, 15]. The skewness of the feature values ​​in this sequence is approximately 3.5 > 3 (skewness threshold), therefore the distribution type of this "single download file size" feature is determined to be "heavy-tailed". This distribution type describes that in the user's daily activities, the data size of their single operations is mainly concentrated at a low level (in this example, most values ​​are between 1-5MB). However, occasionally, there are a very small number of operations with unusually large sizes (120MB and 15MB in this example), these values ​​are far from the main data, forming a long "tail" in the statistical distribution.

[0097] Suppose a user's historical "number of operations per hour" data is recorded as [0, 0, 10, 0, 0, 25, 0, 0, 0, 50] (a total of 10 hours). The mean of the feature values ​​in this sequence is 8.5, the variance is approximately 250.25 (i.e., extremely volatile data), and the coefficient of variation (the ratio of variance to mean) is approximately 250.25 / 8.5 ≈ 29.44. Since the coefficient of variation 29.44 > 1.5 (the coefficient of variation threshold), the distribution type of this "number of operations per hour" feature is determined to be an "overly dispersed distribution." This distribution type describes a user who is completely silent for most of the time (0 operations), but extremely occasionally experiences bursts of high-density operations (such as 10, 25, or even 50 operations per hour), and the intensity and interval of these bursts are completely irregular, resulting in data volatility (variance) far exceeding its average activity level (mean).

[0098] In some embodiments, to support accurate calculations for subsequent real-time anomaly detection, the user feature vector may include not only the distribution type corresponding to the feature values ​​of the access behavior features in each dimension, but also the statistical index values ​​corresponding to each distribution type.

[0099] Specifically, the aforementioned statistical indicators refer to a set of core quantitative parameters derived from statistical analysis and calculation of the characteristic values ​​of users' historical access behavior based on statistical analysis methods adapted to this distribution type. These parameters are not raw data, but rather a highly abstract and mathematical generalization of users' historical behavioral patterns. Their essential function is to establish a personalized and calculable reference for normal behavior for the user in their specific behavioral dimensions.

[0100] In this case, after determining the distribution type of the feature values ​​of historical access behavior characteristics in a certain dimension, a statistical analysis method adapted to that distribution type can be immediately invoked to perform secondary analysis on all historical feature values ​​of historical access behavior characteristics in that dimension, thereby obtaining a set of corresponding statistical indicator values.

[0101] Then, based on the distribution types corresponding to the feature values ​​of the historical access behavior features of the above multiple dimensions, and the statistical index values ​​corresponding to each distribution type, the user feature vector is generated in one step.

[0102] It should be noted that, since different distribution types are adapted to different statistical analysis algorithms, the above statistical indicator values ​​corresponding to different distribution types will also have certain differences.

[0103] For example, please continue to see Figure 4 For zero-inflated distributions, anomaly detection methods adapted to zero-inflated distributions can specifically include zero-value compensation detection. The core of this method is to identify the transition from "silent" to "active" activity, and whether the scale of active events is abnormal. In this case, the statistical index value corresponding to the zero-inflated distribution can include a specified quantile of non-zero values ​​(such as the 99th percentile, P99).

[0104] During the offline generation of user feature vectors, all zero values ​​can be filtered out from historical data, and only the high quantile (e.g., P99) of the non-zero value sequence can be calculated. The P99 value serves as a personalized dynamic threshold for judging whether the non-zero value of the feature is an extreme value.

[0105] For heavy-tailed distributions, anomaly detection methods adapted to this distribution can include log-space-based Z-score detection. This method compresses the scale of extreme values ​​through logarithmic transformation, bringing the data closer to a normal distribution before standardizing and comparing differences. In this case, the statistical indicators corresponding to the heavy-tailed distribution can include the log-space mean (log_mean) and the log-space standard deviation (log_std).

[0106] In the offline generation of user feature vectors, a logarithmic transformation can be performed on the historical feature values, and then the mean and standard deviation of the logarithmically transformed historical feature values ​​can be calculated. The mean and standard deviation together define the "normal" distribution position and dispersion of the user in the logarithmic space, and are the benchmark parameters for calculating the Z-score.

[0107] For overdispersive distributions, an anomaly detection method adapted to overdispersive distributions can be based on the CDF (Cumulative Distribution Function) detection method using the negative binomial distribution. This method uses the more flexible negative binomial distribution model to fit the volatile count data and identifies anomalies through cumulative probability. In this case, the statistical indicators corresponding to the overdispersive distribution can specifically include the shape parameter (r) and success probability (p) of the negative binomial distribution.

[0108] During the offline generation of user feature vectors, methods such as maximum likelihood estimation can be used to fit the user's historical count data to a negative binomial distribution and obtain the optimal parameters r and p. These two parameters, r and p, uniquely determine the complete probability distribution model describing the fluctuation pattern of the user's behavior and are necessary inputs for calculating the probability of any observation occurring (CDF value).

[0109] Those skilled in the art will understand that the above statistical indicators are preferred embodiments. For the same distribution type, other equivalent statistical measures may be used under the same inventive concept (for example, for heavy-tailed distributions, the median may also be used as a robust statistical indicator).

[0110] In this specification, after extracting the user's access behavior characteristics for the target data from the above access data sample, the distribution type corresponding to the feature value of the access behavior characteristics can be further determined, and subsequent real-time anomaly detection can be performed based on the determined distribution type.

[0111] In some embodiments, such as Figure 3 As shown, after extracting the real-time access behavior features of the user for the target data from the above access data sample, the user feature vector generated for the user in the offline stage can be obtained, and the access behavior features can be matched with the historical access behavior features of multiple dimensions contained in the user feature vector in sequence. Then, the distribution type corresponding to the feature value of the real-time access behavior feature can be determined according to the matching result.

[0112] If the real-time access behavior feature matches any dimension of the target historical access behavior feature contained in the user feature vector, then the distribution type corresponding to the feature value of the target historical access behavior feature contained in the user feature vector can be determined as the distribution type corresponding to the feature value of the access behavior feature.

[0113] In this way, feature matching can be used to quickly determine the distribution type corresponding to the feature values ​​of the user's real-time access behavior characteristics by statistically analyzing the distribution type obtained from the user's historical access behavior characteristics in the offline stage.

[0114] Step 206: Based on a statistical analysis method adapted to the distribution type, perform anomaly detection on the feature values ​​of the access behavior characteristics to determine whether the access behavior poses a risk.

[0115] Please continue reading Figure 3Once the distribution type corresponding to the feature values ​​of the user's real-time access behavior characteristics is determined, a statistical analysis method adapted to that distribution type can be further determined. Based on the statistical analysis method adapted to that distribution type, adaptive anomaly detection can be performed on the feature values ​​of the real-time access behavior characteristics to determine whether there is any risk in the user's real-time access behavior to the target data.

[0116] In some embodiments, please continue to see Figure 3 Taking the aforementioned preset distribution types, including zero-inflation distribution, heavy-tailed distribution, and overly dispersed distribution, as an example, the statistical index value corresponding to the zero-inflation distribution can include the specified quantile of the non-zero value (such as the 99th percentile, P99), the statistical index value corresponding to the heavy-tailed distribution can include the mean (log_mean) and the standard deviation (log_std) of the log space, and the statistical index value corresponding to the overly dispersed distribution can specifically include the shape parameter (r) and the success probability (p) of the negative binomial distribution.

[0117] In this context, when performing anomaly detection on the feature values ​​of a user's real-time access behavior characteristics based on a statistical analysis method adapted to the determined distribution type, the specific type of distribution corresponding to the feature values ​​of the user's real-time access behavior characteristics can be determined first.

[0118] On the one hand, if the distribution type corresponding to the feature value of the real-time access behavior feature is a zero-inflated distribution, then the specified quantile of the non-zero value corresponding to the zero-inflated distribution contained in the user's user feature vector can be obtained.

[0119] Then, we can first determine whether the feature value of the access behavior characteristic is non-zero. If the feature value is non-zero, we can further compare it with a specified quantile of that non-zero value to determine whether the feature value reaches the specified quantile. If it does, we can determine that the user's real-time access behavior to the target data poses a risk; otherwise, we can determine that the access behavior does not pose a risk. Of course, if the feature value of the access behavior characteristic is zero, we can directly determine that the access behavior does not pose a risk.

[0120] On the other hand, if the distribution type corresponding to the feature value of the real-time access behavior feature is a heavy-tailed distribution, then the mean and standard deviation corresponding to the heavy-tailed distribution contained in the user's user feature vector can be obtained.

[0121] Then, the feature values ​​of the access behavior feature can be logarithmically transformed, and the mean, standard deviation and logarithmically transformed feature values ​​of the access behavior feature can be used as calculation parameters to calculate the Z-score score, and determine whether the Z-score score reaches the anomaly threshold corresponding to the heavy-tailed distribution; if so, it can be determined that the user's real-time access behavior to the target data is risky; otherwise, it can be determined that the access behavior is not risky.

[0122] Thirdly, if the distribution type corresponding to the feature value of the real-time access behavior feature is an overdispersive distribution, then the shape parameter of the negative binomial distribution corresponding to the overdispersive distribution and the success probability of the negative binomial distribution contained in the user feature vector can be obtained.

[0123] Then, the shape parameters of the negative binomial distribution, the success probability of the negative binomial distribution, and the feature value of the access behavior feature can be used as calculation parameters and input into the CDF function for calculation to obtain the anomaly probability corresponding to the feature value of the access behavior feature, and determine whether the anomaly probability reaches the anomaly threshold corresponding to the overdispersive distribution; if so, it can be determined that the user's real-time access behavior to the target data is risky; otherwise, it can be determined that the access behavior is not risky.

[0124] Please continue reading Figure 3 Once the anomaly detection of the feature values ​​of the user's real-time access behavior is completed, on the one hand, if it is determined that the access behavior is not risky, a risk-free audit log can be generated for security auditing; on the other hand, if it is determined that the access behavior is risky, a real-time alarm can be issued to instruct relevant personnel to intervene in such risky access behavior in a timely manner.

[0125] The following is a specific example to illustrate this.

[0126] Suppose there are three users: U1, U2, and U3. User U1 has a zero-inflation distribution, user U2 has a heavy-tailed distribution, and user U3 has an overdispersive distribution.

[0127] Example 1: User U1 Historical behavior and baseline establishment: Assume that user U1's historical "daily download count" data is: [0,0,0,1,0,0,0,0,2,0] (10 days in total).

[0128] Distribution type determination: The zero-value ratio is calculated to be 8 / 10 = 0.8. Since this zero-value ratio is greater than the zero-value ratio threshold of 0.7, it is determined that the feature value of the user's historical access behavior follows a "zero-inflated distribution".

[0129] Calculation of statistical indicator values: Extract all non-zero values ​​[1,2] and calculate their 99th percentile (P99) as the threshold. In this example, the P99 value is approximately 2.

[0130] Behavioral baseline storage: In U1's user feature vector, the following user feature vector is generated as a behavioral baseline for the "daily download count" dimension: {Distribution type: "zero inflation", Statistical indicator: {P99: 2}}.

[0131] Real-time detection scenarios: Suppose that on a certain day, the system monitors that U1 initiated 3 file download operations.

[0132] At this time, U1's real-time access behavior characteristic is "daily download count", with a characteristic value of 3.

[0133] You can query the behavioral baseline of U1 to confirm that the feature corresponds to a "zero-inflation distribution".

[0134] Anomaly detection (zero-value compensation detection): It can be determined whether the feature value is non-zero; at this time, the feature value is non-zero, and then the non-zero value 3 is compared with the statistical index value P99=2 stored in the behavioral baseline.

[0135] At this point, since the non-zero value of 3 is greater than the statistical indicator value P99=2, it can be determined that the number of real-time operations performed by U1 exceeds the 99th percentile threshold of the user's historical non-zero activity. Therefore, this access behavior is deemed risky, triggering an anomaly alarm.

[0136] This accurately captured the unusual pattern of low-frequency users suddenly exhibiting activity levels exceeding historical high frequencies.

[0137] Example 2: User U2 (heavy-tailed distribution) Historical behavior and baseline establishment: Assume user U2's historical "single download file size (MB)" data is: [2,1,5,1,120,3,2,15].

[0138] Distribution type determination: The skewness of the sequence is calculated. Since the skewness of the sequence is much greater than the skewness threshold of 3, it is determined that the feature value of the user's historical access behavior follows a "heavy-tailed distribution".

[0139] Calculation of statistical indicator values: Perform a logarithmic transformation on the feature values ​​of all historical visit behavior characteristics, and then calculate the mean (log_mean) and standard deviation (log_std) of the transformed sequence. Assume that the calculated values ​​are: log_mean ≈ 1.2, log_std ≈ 0.8.

[0140] Behavioral baseline storage: In U2's user feature vector, the following user feature vector is generated as a behavioral baseline for the "single download file size" dimension: {Distribution type: "heavy-tailed", Statistical indicators: {log_mean: 1.2, log_std: 0.8}}.

[0141] Real-time detection scenarios: Suppose that on a certain day, U2 initiates the download of a target data of 250MB.

[0142] At this point, U2's real-time access behavior is characterized by "single download file size", with a feature value of 250.

[0143] You can query the behavioral baseline of U2 to confirm that this feature corresponds to a "heavy-tailed distribution".

[0144] Anomaly detection (based on logarithmic space Z-score detection): First, the eigenvalue can be logarithmically transformed to compress the scale, and then its deviation from the individual's historical logarithmic mean can be calculated by log standard deviations.

[0145] For example, logarithmic transformations can be performed in the following way: log_size = log(250 + 1) ≈ 5.52 The Z-score is then calculated using the following method: Z-score=(5.52 - 1.2) / 0.8≈5.4 Finally, since the Z-score is 5.4 > 3 (the anomaly threshold), it indicates that the logarithm of the file size downloaded by the current U2 user in a single instance deviates from their usual value by more than 3 standard deviations, which is an extremely low probability event. Therefore, this access behavior can be determined to be high-risk. This method effectively identifies "abnormally large" operations that occur within occasional large values.

[0146] Example 3: User U3 (overly dispersed distribution) Historical behavior and baseline establishment: Assume that user U3's historical "number of operations per hour" data is: [0,0,10,0,0,25,0,0,0,50] (total 10 hours).

[0147] Distribution type determination: The ratio of variance to mean is calculated to obtain the coefficient of variation. Since this coefficient of variation is much larger than the coefficient of variation threshold of 1.5, it is determined that the feature value of the user's historical access behavior follows an "overdispersive distribution".

[0148] Calculation of statistical indicator values: We use a negative binomial distribution to fit the feature values ​​of historical access behavior to obtain the optimal shape parameter r and success probability p. Assume the fitting result is: r = 0.5, p = 0.055.

[0149] Behavioral baseline storage: In the U3 user feature vector, the following user feature vector is generated as a behavioral baseline for the "number of operations per hour" dimension: {Distribution type: "overly dispersed", Statistical indicators: {nb_r: 0.5, nb_p: 0.055}}.

[0150] Real-time detection scenarios: Suppose that within a certain hour, the system monitors that U3 initiated 30 operations.

[0151] At this point, U3's real-time access behavior characteristic is "number of operations per hour", with a characteristic value of 30.

[0152] You can query the behavioral baseline of U3 to confirm that this feature corresponds to an "overly dispersed distribution".

[0153] Anomaly detection (CDF detection based on negative binomial distribution): For "overly dispersed distribution", a negative binomial distribution model that describes the user's own fluctuation pattern can be selected to calculate the abnormal probability (right-tail probability) of "30 or more" operations.

[0154] Specifically, you can input the cumulative distribution function (CDF function) of the negative binomial distribution with k=30, r=0.5, and p=0.055 to calculate the anomaly probability: P_right=P(X>=30)≈0.008.

[0155] Since the anomaly probability P_right = 0.008 is less than the anomaly threshold of 0.01, and based on U3's own behavior model, the probability of such a high number of sudden operations is less than 1%. Therefore, it can be determined that this access behavior is risky.

[0156] This method accurately identified an abnormal outbreak with an intensity far exceeding historical experience within an otherwise irregular outbreak pattern.

[0157] In this specification, to further reduce the false negative rate, one or more layers of enhanced detection based on business rules or behavioral patterns can be superimposed after the above-mentioned adaptive detection based on distribution type.

[0158] In some embodiments, if anomaly detection is performed on the feature values ​​of the user's real-time access behavior characteristics based on the anomaly detection method corresponding to the determined distribution type, and it is determined that the access behavior does not pose a risk, it can further detect whether the target data is target data of a specified type (e.g., a credit report), and detect whether the user's access time for the target data is within the legal access time range corresponding to the specified type of target data (e.g., weekdays 9:00-17:00). If the target data is not of the specified type, it can be determined that the access behavior does not pose a risk. If the target data is of a specified type, and the user's access time to the target data is within the legal access time range corresponding to the specified type of target data, it can be determined that the access behavior does not pose a risk. If the target data is of a specified type, and the user's access time to the target data is not within the legal access time range corresponding to the specified type of target data, it can be determined that the access behavior is risky.

[0159] In some embodiments, if anomaly detection is performed on the feature values ​​of the user's real-time access behavior characteristics based on the anomaly detection method corresponding to the determined distribution type, and it is determined that the access behavior does not pose a risk, human-machine behavior recognition can be further performed on the user's access behavior for the target data.

[0160] If a user's access to the target data fails the human-machine behavior identification process, the access behavior is very likely an illegal access behavior based on some automated scripts for data crawling; it can be determined that the access behavior is risky; conversely, it can be determined that the access behavior is not risky.

[0161] The specific methods for human-machine behavior recognition of user access to the target data are not specified in this specification.

[0162] For example, in some embodiments, the entropy value of the historical IP addresses used by the user is calculated (to measure the diversity of IPs; a low entropy value indicates a fixed IP source), and the interval between the user's successful login and the initiation of access to the target data is counted. If the entropy value is lower than a threshold (e.g., 0.3) and the interval is lower than a threshold (e.g., 5 seconds), it can be determined that the user's access to the target data has not passed the human-machine behavior recognition; otherwise, it can be determined that the user's access to the target data has passed the human-machine behavior recognition.

[0163] The above technical solution abandons the approach of applying uniform, static anomaly detection rules to all users. Instead, it constructs a "behavioral baseline" for each user that reflects their unique historical behavior patterns. Based on this behavioral baseline, a personalized detection method that matches the statistical characteristics related to the user's behavior patterns is used to determine anomalies in the user's real-time access behavior. This significantly improves the accuracy of detection, reduces false positive and false negative rates, and ensures the interpretability of the detection process, thus meeting the compliance requirements of security audits.

[0164] On the one hand, since the feature values ​​of different users' access behavior characteristics usually have different distribution types, and different distribution types reflect the differences in the behavioral patterns of different users when accessing target data; therefore, through this detection method, appropriate statistical analysis methods can be adaptively selected based on different user behavior patterns to detect anomalies in the feature values ​​of users' access behavior characteristics, thereby improving the detection accuracy of anomalies. On the other hand, since statistical analysis methods can be used to detect anomalies in the feature values ​​of user access behavior, this detection method can generate clear and reliable criteria for anomaly judgment, enhance the interpretability of anomaly detection, and thus meet the requirements of security auditing. Moreover, using this lightweight statistical analysis method to detect anomalies in the feature values ​​of user access behavior can also improve the detection efficiency and reduce the latency of anomaly detection. This allows this detection method to still meet the strict performance requirements of latency and real-time performance in some detection scenarios with high requirements for detection latency and real-time performance.

[0165] Figure 5 This is a schematic structural diagram of an electronic device provided in an exemplary embodiment. For example... Figure 5 As shown, device 500 mainly consists of a communication interface 502, a user interface 504, a processor 506, and a data storage 508. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 510. The communication interface 502 enables device 500 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 502 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 502 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 502 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 502 may also include multiple physical communication interfaces, such as Wi-Fi interfaces, Bluetooth interfaces, and wide-area wireless interfaces.

[0166] User interface 504 includes receiving user input and providing output to the user. Therefore, user interface 504 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 504 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 504 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 500 may support remote access from other devices via communication interface 502 or another physical interface (not shown). User interface 504 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 504 may also be configured as a display device for rendering or displaying text fragments.

[0167] Processor 506 may contain one or more general-purpose processors and / or special-purpose processors.

[0168] Data storage 508 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 506. Data storage 508 may include removable and non-removable components.

[0169] Processor 506 is capable of executing program instructions 518 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 508 to perform the various functions described herein. Data storage 508 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 500, enable device 500 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 518 by processor 506 may result in processor 506 using data 512.

[0170] For example, program instructions 518 may include an operating system 522 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 500 and one or more applications 520 (e.g., a browser, social application, or game application). Similarly, data 512 may include operating system data 516 and application data 514. Operating system data 516 is primarily accessible to the operating system 522, while application data 514 is primarily accessible to one or more applications 520. Application data 514 may reside in a file system visible or hidden from the user of device 500.

[0171] Application 520 can communicate with operating system 522 through one or more application programming interfaces (APIs). These APIs help application 520 read and / or write application data 514, transmit or receive information via communication interface 502, receive or display information on user interface 504, etc.

[0172] In some terminology, application 520 may be simply referred to as "app". Furthermore, application 520 can be downloaded to device 500 through one or more online app stores or app markets. However, applications can also be installed on device 500 in other ways, such as through a web browser or a physical interface on device 500 (e.g., a USB port).

[0173] Please refer to Figure 6 This specification also proposes an abnormal data access detection device, which can be applied to, for example... Figure 5 The device shown is used to implement the technical solution of this specification. The device may include: The acquisition module 601, in response to the user's access behavior to the target data, acquires a sample of the user's access data to the target data, and extracts the user's access behavior features to the target data from the sample of the access data. The determining module 602 determines the distribution type corresponding to the feature value of the access behavior feature; wherein, the distribution type is used to represent the type of probability distribution of the feature value of the access behavior feature in the historical access data sample of the user for the target data; The detection module 603 performs anomaly detection on the feature values ​​of the access behavior characteristics based on a statistical analysis method adapted to the distribution type, so as to determine whether the access behavior is risky.

[0174] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0175] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0176] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0177] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0178] What those skilled in the art will understand is: In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0179] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0180] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0181] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0182] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0183] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0184] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A method for detecting abnormal data access, comprising: In response to a user’s access behavior to target data, a sample of the user’s access data to the target data is obtained, and the user’s access behavior features to the target data are extracted from the access data sample. Determine the distribution type corresponding to the feature value of the access behavior feature; wherein, the distribution type is used to represent the type of probability distribution of the feature value of the access behavior feature in the historical access data sample of the user for the target data; Anomaly detection is performed on the feature values ​​of the access behavior characteristics based on a statistical analysis method adapted to the distribution type to determine whether the access behavior poses a risk.

2. The method as described in claim 1, wherein determining the distribution type corresponding to the feature value of the access behavior feature includes: Obtain the user feature vector corresponding to the user; wherein, the user feature vector is used to describe the behavioral pattern of the user's historical access behavior to the target data; the user feature vector includes multiple preset dimensions of historical access behavior features, and the distribution type corresponding to the feature values ​​of each dimension of historical access behavior features respectively; The access behavior features are sequentially matched with the historical access behavior features of the multiple dimensions contained in the user feature vector; If the access behavior feature matches any dimension of the target historical access behavior feature contained in the user feature vector, the distribution type corresponding to the feature value of the target historical access behavior feature contained in the user feature vector is determined as the distribution type corresponding to the feature value of the access behavior feature.

3. The method as described in claim 2, wherein the user feature vector is generated in the following manner: Obtain a sample set of historical access data of the user for the target data; Multiple dimensions of historical access behavior features are extracted from the historical access data samples contained in the historical access data sample set, and statistical analysis is performed on the feature values ​​of the extracted multiple dimensions of historical access behavior features to obtain the distribution types corresponding to the feature values ​​of the multiple dimensions of historical access behavior features. The user feature vector is generated based on the distribution type corresponding to the feature values ​​of the historical access behavior features of the multiple dimensions.

4. The method as described in claim 3, wherein statistical analysis is performed on the feature values ​​of the extracted historical access behavior features to obtain the distribution type corresponding to the feature values ​​of the historical access behavior features, including: The feature values ​​of the extracted historical access behavior features are calculated relative to the distribution index values ​​of each of the preset multiple distribution types; where different distribution types correspond to different distribution index values. If the feature value of the historical access behavior feature reaches the distribution threshold corresponding to any target distribution type among the preset multiple distribution types, then the target distribution type is determined as the distribution type corresponding to the feature value of the historical access behavior feature.

5. The method as described in claim 4, wherein the user feature vector further includes statistical index values ​​corresponding to the distribution type; wherein, The statistical indicator values ​​include statistical indicator values ​​obtained by statistical analysis of the feature values ​​of the historical access behavior characteristics based on a statistical analysis method adapted to the distribution type; The user feature vector is generated based on the distribution type corresponding to the feature values ​​of the historical access behavior features of the multiple dimensions, including: Based on the statistical analysis method adapted to the distribution type, the feature values ​​of the historical access behavior characteristics are statistically analyzed to obtain the statistical index values ​​corresponding to the distribution type. The user feature vector is generated based on the distribution types corresponding to the feature values ​​of the historical access behavior features of the multiple dimensions, and the statistical index values ​​corresponding to the distribution types.

6. The method of claim 5, wherein anomaly detection is performed on the feature values ​​of the access behavior characteristics based on a statistical analysis method adapted to the distribution type to determine whether the access behavior poses a risk, comprising: Obtain the statistical index value corresponding to the distribution type contained in the user feature vector; The statistical indicator value is used as an anomaly threshold and compared with the feature value of the access behavior feature to determine whether the feature value of the access behavior feature reaches the anomaly threshold. If so, determine that the access behavior is risky; or, use the statistical index value and the feature value of the access behavior feature as calculation parameters to calculate the anomaly baseline value corresponding to the anomaly detection method adapted to the distribution type, and determine whether the anomaly baseline value reaches the anomaly threshold corresponding to the distribution type. If so, the access behavior is deemed risky.

7. The method of claim 6, wherein the access behavior characteristics include access behavior indicators; in, The access behavior metrics include any one or more combinations shown below: The size of the target data accessed by the user, the access time of the user accessing the target data, the access duration of the user accessing the target data, the number of times the user accesses the target data, and the interval between user accesses the target data.

8. The method as described in claim 6, wherein the preset multiple distribution types include a zero-inflation distribution, a heavy-tailed distribution, and an overly dispersed distribution; in, The distribution index value corresponding to the zero-inflation distribution includes the proportion of zero values; the distribution threshold corresponding to the zero-inflation distribution includes the zero-value proportion threshold. The distribution index value corresponding to the heavy-tailed distribution includes skewness; the distribution threshold corresponding to the heavy-tailed distribution includes the skewness threshold; Distribution index values ​​corresponding to an overdispersive distribution include the coefficient of variation and the ratio of variance to mean. The distribution threshold corresponding to the overdispersive distribution includes the discrete coefficient threshold; the discrete coefficient is represented by the ratio of variance to mean.

9. The method as described in claim 8, wherein statistical analysis is performed on the feature values ​​of the extracted historical access behavior features to obtain the distribution type corresponding to the feature values ​​of the historical access behavior features, including: Calculate the proportion of zero values, skewness, and coefficient of variation of the extracted historical access behavior features; If the zero-value ratio reaches the zero-value ratio threshold, the distribution type corresponding to the feature value of the historical access behavior feature will be determined as a zero-inflation distribution. If the skewness reaches the skewness threshold, the distribution type corresponding to the feature value of the historical access behavior feature will be determined as a heavy-tailed distribution. If the discrete coefficient reaches the discrete coefficient threshold, the distribution type corresponding to the feature value of the historical access behavior feature is determined to be an overdispersive distribution.

10. The method of claim 9, wherein the anomaly detection method corresponding to the zero-inflated distribution includes zero-value compensation detection; the statistical index value corresponding to the zero-inflated distribution includes a specified quantile of non-zero values; the anomaly detection method corresponding to the heavy-tailed distribution includes Z-score detection based on logarithmic space; the statistical index value corresponding to the heavy-tailed distribution includes mean and standard deviation; the anomaly detection method corresponding to the overdispersive distribution includes CDF detection based on negative binomial distribution; and the statistical index value corresponding to the overdispersive distribution includes the shape parameter of the negative binomial distribution and the success probability of the negative binomial distribution. Anomaly detection is performed on the feature values ​​of the access behavior characteristics based on a statistical analysis method adapted to the distribution type to determine whether the access behavior poses a risk, including: If the distribution type corresponding to the feature value of the access behavior feature is a zero-inflated distribution, obtain the specified quantile of the non-zero value corresponding to the zero-inflated distribution contained in the user feature vector; determine whether the feature value of the access behavior feature is non-zero; if the feature value of the access behavior feature is non-zero, further compare the feature value of the access behavior feature with the specified quantile of the non-zero value to determine whether the feature value of the access behavior feature reaches the specified quantile of the non-zero value; if yes, determine that the access behavior has a risk; otherwise, determine that the access behavior does not have a risk. If the distribution type corresponding to the feature value of the access behavior feature is a heavy-tailed distribution, obtain the mean and standard deviation corresponding to the heavy-tailed distribution contained in the user feature vector; perform a logarithmic transformation on the feature value of the access behavior feature, and use the mean, the standard deviation, and the logarithmically transformed feature value of the access behavior feature as calculation parameters to calculate the Z-score, and determine whether the Z-score reaches the anomaly threshold corresponding to the heavy-tailed distribution; if yes, determine that the access behavior has a risk; otherwise, determine that the access behavior does not have a risk. If the distribution type corresponding to the feature value of the access behavior feature is an overdispersive distribution, obtain the shape parameter and success probability of the negative binomial distribution corresponding to the overdispersive distribution contained in the user feature vector, and input the shape parameter, success probability, and feature value of the access behavior feature as calculation parameters into the CDF function to calculate the anomaly probability corresponding to the feature value of the access behavior feature; and determine whether the anomaly probability reaches the anomaly threshold corresponding to the overdispersive distribution; if yes, determine that the access behavior has a risk; otherwise, determine that the access behavior does not have a risk.

11. The method of claim 10, further comprising: If an anomaly detection is performed on the feature value of the access behavior feature based on the anomaly detection method corresponding to the distribution type, and it is determined that the access behavior does not pose a risk, then it is further detected whether the target data is target data of a specified type, and whether the user's access time for the target data is within the legal access time range corresponding to the target data of the specified type. If the target data is not of the specified type, it is determined that the access behavior does not pose a risk. If the target data is of a specified type, and the user's access time to the target data is within the legal access time range corresponding to the specified type of target data, it is determined that the access behavior does not pose a risk. If the target data is of a specified type, and the user's access time to the target data is not within the legal access time range corresponding to the specified type of target data, the access behavior is determined to be risky.

12. The method of claim 10, further comprising: If an anomaly detection is performed on the feature value of the access behavior feature based on the anomaly detection method corresponding to the distribution type, and it is determined that the access behavior does not pose a risk, then human-machine behavior recognition is further performed on the user's access behavior to the target data. If the user's access behavior to the target data fails the human-machine behavior recognition, the access behavior is determined to be risky; otherwise, the access behavior is determined to be risk-free.

13. The method of claim 12, wherein human-machine behavior recognition is performed on the user's access behavior to the target data, comprising: Calculate the entropy value of the historical IP addresses used by the user, and count the time interval between the user's successful login and the initiation of access behavior for the target data; If both the entropy value and the interval duration are below the threshold, it is determined that the user's access behavior to the target data has not passed the human-machine behavior recognition; otherwise, it is determined that the user's access behavior to the target data has passed the human-machine behavior recognition.

14. The method of claim 1, wherein the target data includes privacy data related to the user.

15. The method of claim 14, wherein the target data includes the user's credit data.

16. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1-15 by executing the executable instructions.

17. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-15.

Images