An anomaly detection method and device, electronic equipment and computer readable medium

Abstract

This application discloses anomaly detection methods, devices, electronic equipment, and computer-readable media, relating to the field of artificial intelligence recognition and classification technology. The method includes: receiving a user application request and determining the corresponding scene identifier; acquiring user scene data over a preset historical time period based on the scene identifier; invoking a clustering model to determine clusters based on the user scene data; determining anomaly screening indicators, and then determining anomalous clusters within each cluster based on the anomaly screening indicators; and intercepting the user application request in response to determining that the user scene data corresponding to the user application request matches the anomalous clusters. By identifying similar patterns of users based on scene data, no actual connection between users is required. It can identify anomalous applications with similar patterns but no actual connection, thereby enabling timely detection of new fraudulent activities and effectively preventing financial losses.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence recognition and classification technology, and in particular to an anomaly detection method, device, electronic device and computer-readable medium. Background Technology

[0002] Driven by consumption upgrades, policy support, and the development of fintech, traditional financial institutions, represented by commercial banks, are accelerating their integration with the internet, closely following internet innovation, and intensifying their internet finance strategic layout. Through self-service systems such as online banking and mobile banking, they provide customers with diversified online personal credit loan services. While a wide range of credit loan products greatly facilitates personal loans, different products target different individuals and use vastly different data, requiring lending institutions to develop different risk control strategies based on business scenarios. One challenge in credit risk control is fraud prevention. Fraud can take many forms, making it difficult to label them simply. Even with historical fraud data available for labeled supervised learning, the learned fraud patterns are often limited to those similar to the past, failing to detect new types of fraud. In various scenarios, fraud methods vary widely. Users with no historical behavior or minimal activity, such as newly registered users or inactive users, cannot be identified, leading to significant delays in the detection of new fraud.

[0003] In the process of developing this application, the inventors discovered at least the following problems in the prior art:

[0004] Fraud can take many forms in various scenarios. For users with no history of activity or few activities, such as newly registered users or inactive users, it is impossible to identify them, which leads to a significant delay in the discovery of new fraud. Summary of the Invention

[0005] In view of this, embodiments of this application provide an anomaly detection method, apparatus, electronic device, and computer-readable medium, which can solve the problem that existing methods of fraud vary greatly in multiple scenarios, and cannot identify users with no historical behavior or few behaviors, such as newly registered users or inactive users, resulting in a significant lag in the detection of new fraud.

[0006] To achieve the above objectives, according to one aspect of the embodiments of this application, an anomaly detection method is provided, comprising:

[0007] Receive user application requests and determine the corresponding scenario identifier;

[0008] Based on scene identifiers, obtain user scene data for a preset historical time period;

[0009] Invoke the clustering model to determine each cluster based on user scenario data;

[0010] Determine anomaly screening indicators, and then determine the anomalous clusters in each cluster based on the anomaly screening indicators;

[0011] In response to the determination that the user scenario data corresponding to the user's application request matches the abnormal cluster, the user's application request is intercepted.

[0012] Optionally, the anomaly detection method may further include the following before invoking the clustering model:

[0013] Obtain a training sample set, which includes user application data corresponding to the same scenario identifier, including abnormal applications with time clustering.

[0014] Feature engineering is performed on user application data, and then clustering indicators are selected based on decision trees;

[0015] Based on the clustering indicators obtained through screening, the nodes of the decision tree are updated, and then a clustering model is generated based on the updated decision tree.

[0016] Optionally, clustering metrics can be selected based on decision trees, including:

[0017] Decision trees are used to pre-classify user application data for all time periods, and then clustering metrics are determined based on the pre-classification results.

[0018] Optionally, clustering metrics are determined based on the pre-classification results, including:

[0019] Determine the proportion of outlier samples in each node of the decision tree;

[0020] The proportion of abnormal samples in each node is sorted, a preset number of nodes are selected based on the sorting, and the labels corresponding to the preset number of nodes are determined as clustering indicators.

[0021] Optionally, anomalous clusters within each cluster are determined based on anomaly screening metrics, including:

[0022] Determine the mean and standard deviation of each anomaly screening index in each cluster;

[0023] Based on the mean and standard deviation, abnormal clusters are identified in each cluster.

[0024] Optionally, based on the mean and standard deviation, outlier clusters within each cluster are identified, including:

[0025] Generate outlier reference values ​​based on the mean and standard deviation;

[0026] The clusters in which the index values ​​are greater than the abnormal reference values ​​are identified are then identified as abnormal clusters.

[0027] Optionally, the anomaly detection method also includes:

[0028] The system automatically updates user scenario data for a preset historical time period every day, and then automatically updates each cluster.

[0029] In addition, this application also provides an anomaly detection device, including:

[0030] The receiving unit is configured to receive user application requests and determine the corresponding scenario identifier;

[0031] The acquisition unit is configured to acquire user scenario data for a preset historical time period based on the scenario identifier;

[0032] The cluster determination unit is configured to invoke the clustering model to determine each cluster based on user scenario data;

[0033] The abnormal cluster determination unit is configured to determine anomaly screening indicators, and then determine the abnormal clusters in each cluster based on the anomaly screening indicators;

[0034] The anomaly detection unit is configured to intercept user application requests in response to determining whether the user scenario data corresponding to the user application request matches with anomaly clusters.

[0035] Optionally, the anomaly detection device further includes a training unit, configured to:

[0036] Obtain a training sample set, which includes user application data corresponding to the same scenario identifier, including abnormal applications with time clustering.

[0037] Feature engineering is performed on user application data, and then clustering indicators are selected based on decision trees;

[0038] Based on the clustering indicators obtained through screening, the nodes of the decision tree are updated, and then a clustering model is generated based on the updated decision tree.

[0039] Optionally, the training unit is further configured to:

[0040] Decision trees are used to pre-classify user application data for all time periods, and then clustering metrics are determined based on the pre-classification results.

[0041] Optionally, the training unit is further configured to:

[0042] Determine the proportion of outlier samples in each node of the decision tree;

[0043] The proportion of abnormal samples in each node is sorted, a preset number of nodes are selected based on the sorting, and the labels corresponding to the preset number of nodes are determined as clustering indicators.

[0044] Optionally, the abnormal cluster determination unit is further configured to:

[0045] Determine the mean and standard deviation of each anomaly screening index in each cluster;

[0046] Based on the mean and standard deviation, abnormal clusters are identified in each cluster.

[0047] Optionally, the abnormal cluster determination unit is further configured to:

[0048] Generate outlier reference values ​​based on the mean and standard deviation;

[0049] The clusters in which the index values ​​are greater than the abnormal reference values ​​are identified are then identified as abnormal clusters.

[0050] Optionally, the anomaly detection device further includes an update unit, configured to:

[0051] The system automatically updates user scenario data for a preset historical time period every day, and then automatically updates each cluster.

[0052] In addition, this application also provides an anomaly detection electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the anomaly detection method as described above.

[0053] In addition, this application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the above-described anomaly detection method.

[0054] To achieve the above objectives, according to another aspect of the embodiments of this application, a computer program product is provided.

[0055] A computer program product according to an embodiment of this application includes a computer program that, when executed by a processor, implements the anomaly detection method provided in an embodiment of this application.

[0056] One embodiment of the above invention has the following advantages or beneficial effects: This application receives a user application request, determines the corresponding scene identifier; obtains user scene data for a preset historical time period based on the scene identifier; calls a clustering model to determine each cluster based on the user scene data; determines anomaly screening indicators, and then determines abnormal clusters in each cluster based on the anomaly screening indicators; and intercepts the user application request in response to determining that the user scene data corresponding to the user application request matches the abnormal cluster. By identifying similar patterns of users based on scene data, no actual connection between users is required. It can identify abnormal applications with similar patterns but no actual connection, thereby timely detecting new fraudulent activities and effectively avoiding property losses.

[0057] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0058] The accompanying drawings are provided to better understand this application and do not constitute an undue limitation thereof. Wherein:

[0059] Figure 1 This is a schematic diagram of the main flow of the anomaly detection method according to the first embodiment of this application;

[0060] Figure 2 This is a schematic diagram of the main flow of the anomaly detection method according to the second embodiment of this application;

[0061] Figure 3 This is a schematic diagram illustrating an application scenario of the anomaly detection method according to the third embodiment of this application;

[0062] Figure 4 This is a schematic diagram of the model training process according to an embodiment of this application;

[0063] Figure 5 This is a schematic diagram of the main units of the anomaly detection device according to an embodiment of this application;

[0064] Figure 6 This is an exemplary system architecture diagram to which embodiments of this application can be applied;

[0065] Figure 7 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers in the embodiments of this application. Detailed Implementation

[0066] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with relevant national laws and regulations.

[0067] Figure 1 This is a schematic diagram of the main flow of the anomaly detection method according to the first embodiment of this application, as shown below. Figure 1 As shown, the anomaly detection methods include:

[0068] Step S101: Receive user application request and determine the corresponding scene identifier.

[0069] In this embodiment, the execution entity of the anomaly detection method (e.g., a server) can receive user application requests via wired or wireless connections. The user application request can be a loan request. Of course, it is understood that the user application request can also be an obstacle recognition application request generated by the request generation unit based on an obstacle image captured by the autonomous driving device. This embodiment does not limit the specific content of the user application request. The scene identifier can be an identifier corresponding to a business scenario or a driving road scenario; this embodiment does not limit the specific content corresponding to the scene identifier. For example, the scene identifier can be GJJ, representing a housing provident fund scenario; or SX, representing a credit granting scenario.

[0070] Specifically, scenario data refers to user data used for credit limit calculation. Credit institutions will use different data to grant credit to users based on their data possession or authorization status. For example, if a user authorizes a credit institution to query their housing provident fund data, the credit institution will grant credit based on the user's housing provident fund contribution status; in this case, the housing provident fund scenario is called the credit scenario, and the housing provident fund data is its credit data. Similarly, if a user authorizes a mobile institution to query their tax data, the credit institution will grant credit based on the user's tax payment status; in this case, the tax payment scenario is called the credit scenario, and the tax payment data is its credit data.

[0071] Step S102: Based on the scene identifier, obtain user scene data for a historical preset time period.

[0072] For example, the executing entity can obtain user scenario data for n consecutive days prior to the current time based on the scenario identifier.

[0073] Step S103: Invoke the clustering model to determine each cluster based on user scenario data.

[0074] The user scenario data acquired over a preset historical time period is input into the clustering model, which then calls the module trained by unsupervised learning in the clustering model to cluster the input user scenario data and determine each cluster.

[0075] Specifically, anomaly detection methods also include:

[0076] The system automatically updates user scenario data for a preset historical time period every day, and then automatically updates each cluster.

[0077] Specifically, before invoking the clustering model, the method also includes:

[0078] Obtain a training sample set, which includes user application data corresponding to the same scenario identifier, including abnormal applications with time clustering (e.g., a series of n days);

[0079] like Figure 4 As shown, the executing entity performs feature engineering on user application data (i.e., extracted scenario data), and then selects clustering indicators based on a decision tree for clustering. Specifically, after extracting the corresponding scenario data, the executing entity needs to perform feature engineering, including feature cleaning and feature derivation. Clustering indicators are selected based on a decision tree, where the decision tree label is the already labeled fraud label, and the variables are the features after feature engineering. Leaf nodes with a high proportion of abnormal samples are selected, and all branch paths from them to the root node are traced back to extract the features used by all nodes on the path. For example, the features corresponding to all nodes on the path, such as individual contribution base, account balance, number of contributing units, longest continuous contribution time of housing provident fund, and housing provident fund cessation time, are extracted for subsequent clustering processes. The extracted features are normalized before being used for clustering. Normalization maps features to a uniform range of values ​​within a small range. In this embodiment, feature normalization is mainly used to avoid the influence of different feature units on the weight relationship between features. Clustering is performed using a rolling daily method, taking the application samples from the previous n days for clustering each day. n can be a value between 1 and 7 depending on the amount of data and business experience. A larger `n` indicates a higher dependence on historical samples, a larger sample size, and a lag in anomaly detection; conversely, a smaller `n` has the opposite effect. An enumeration method, combined with an unsupervised component, is used to determine `n` and the number of clusters. For example... Figure 4 As shown, supervised learning uses institutional data and credit data to set early warning rules. Institutional data and credit data are shared customer data, unaffected by specific scenarios, and therefore universal. Then, feature engineering is performed on the extracted institutional data and credit data. The feature-engineered data is then binned and IV calculated to filter monitoring indicators and set early warning thresholds. Finally, anomaly identification is performed based on the clustering results obtained from unsupervised learning. Institutional data, for example, could be data from Company X or Bank X; this embodiment does not specifically limit the type of institutional data used.

[0080] Based on the clustering indicators obtained through screening, the nodes of the decision tree are updated, and then a clustering model based on unsupervised learning is generated based on the updated decision tree. For example... Figure 4 As shown, model training includes sample selection, class division using unsupervised learning, and warning rule setting using supervised learning, ultimately identifying abnormal classes.

[0081] Specifically, clustering metrics are selected based on decision trees, including:

[0082] Decision trees are used to pre-classify user application data across all time periods, and then clustering metrics are determined based on the pre-classification results. Pre-classification can be based on preset classification metrics.

[0083] Specifically, based on the pre-classification results, clustering indices are determined, including:

[0084] Determine the proportion of outliers in each node of the decision tree; sort the proportion of outliers in each node, select a preset number of nodes based on the sorting, and then determine the labels corresponding to the preset number of nodes as clustering indicators.

[0085] Specifically, the executing entity can select leaf nodes with a high proportion of outlier samples on the decision tree, trace back all branch paths from them to the root node, and extract the features used by all nodes on each path. The extracted outlier sample proportions for each node are then sorted, and the labels corresponding to the top n nodes can be selected as clustering indicators to make clustering based on these indicators more accurate.

[0086] Step S104: Determine the anomaly screening index, and then determine the abnormal clusters in each cluster based on the anomaly screening index.

[0087] The process of determining anomaly screening indicators includes: performing feature engineering (the process of transforming raw data into training data for a model, aiming to obtain better training data features) on the training sample set, linking it to user account information at the time of application, historical transaction records, historical loan information, credit information, etc. This includes feature cleaning and feature derivation. The Information Value (IV) and Population Stability Index (PSI) of the features are calculated. A higher IV indicates a stronger correlation between the feature and fraud; a higher PSI indicates less fluctuation in the variable over time. Features with high IV and relatively stable characteristics are selected. Decision tree binning is used on these features to determine individual indicators (i.e., anomaly screening indicators) for screening anomalous clusters. A threshold is set for each anomaly screening indicator (i.e., individual indicator) to define the anomalous range for that indicator.

[0088] Anomaly Cluster Identification: Combining clustering results and anomaly screening indicators, the number of samples in each cluster is counted. Small clusters (with fewer than 2% of the total number of samples) are removed. For each remaining cluster, the proportion of anomalous interval samples under each individual indicator is calculated. The mean and standard deviation of the proportion of anomalous interval samples for each individual indicator in each cluster are calculated. The clusters with indicator values ​​greater than "mean + 2 * standard deviation" are recorded. If a cluster has m or more indicator values ​​greater than "mean + 2 * standard deviation", the cluster is marked as an anomaly cluster. The specific value of m is not limited in this embodiment.

[0089] Step S105: In response to determining that the user scenario data corresponding to the user application request matches the abnormal cluster, the user application request is intercepted.

[0090] For example, when applying clustering models, combining the category classification results of unsupervised learning can determine whether the user scenario data corresponding to a user's application request falls into an abnormal cluster. Subsequently, evaluation metrics are calculated, including the proportion of abnormal cluster samples, the ratio of real abnormal samples within an abnormal cluster to all real abnormal samples throughout the day, the ratio of real abnormal samples within an abnormal cluster to the total number of samples within the abnormal cluster, the number of warning days, the warning rate, and the false alarm rate. Clustering features, clustering parameters, metrics, and thresholds are applied to model validation samples. The abnormal cluster types in the model validation samples can differ from those in the model training samples. The validation part identifies abnormal clusters in the validation samples based on the trained clustering features, clustering parameters, metrics, and thresholds, and calculates evaluation metrics. Combining the evaluation metrics from the training and validation sets, the clustering features, clustering parameters, monitoring metrics, and their thresholds are adjusted according to the set warning targets.

[0091] The model parameters, including the number of days in advance for clustering (n), the number of clusters, and the number of warning indicators (m), are adjusted based on the model's performance on training and validation samples. The determination of n primarily involves statistically analyzing the number of days with accurate warnings and the number of days with false alarms when n ranges from 1 to 7, identifying the n that corresponds to the maximum number of correct warning days and the minimum number of false alarm days. The number of clusters is determined by simultaneously enumerating different cluster numbers in daily clustering, statistically analyzing the proportion of fraudulent samples in the warning clusters, the proportion of warned fraudulent samples out of all fraudulent samples, and the degree of anomalousness of monitoring indicators in abnormal clusters, selecting the cluster number corresponding to the highest coverage of fraudulent samples, the largest proportion of fraudulent samples in abnormal clusters, and the highest degree of anomalousness in monitoring indicators for abnormal clusters. The number of warning indicators (m) is determined by statistically analyzing the number of days with accurate warnings and false alarms, selecting the m corresponding to the maximum number of days with accurate warnings and the minimum number of false alarm days.

[0092] For example, when applying a clustering model, users who applied through the same scenario in the past n days are clustered daily based on the clustering features and parameters determined during model training. Abnormal cluster numbers are then determined based on the metrics and thresholds determined during model training. If abnormal clusters are detected during daily offline training, considering that fraudulent users typically appear in a concentrated timeframe, online real-time monitoring is implemented the following day based on the results of the offline model.

[0093] On the second day, the distance between the clustering features of the new loan application and the centroids of each category on the previous day is calculated in real time to determine the category to which the loan belongs; the correlation indicator features are used in real time to determine whether it belongs to an abnormal cluster. If it is classified into an abnormal cluster, the user's application request is intercepted and sent directly to the manual approval node for manual review.

[0094] This application utilizes unsupervised learning, which automatically adjusts classifications when abnormal patterns change. Daily automatic iterative clustering quickly identifies loan applications with similar patterns that are frequently initiated within a short period. It identifies sudden surges in abnormal group applications without relying on historical application behavior of the application samples or requiring any actual correlation between abnormal samples, and is also effective for new users.

[0095] This embodiment receives user application requests and determines the corresponding scenario identifier; based on the scenario identifier, it obtains user scenario data for a preset historical time period; it calls a clustering model to determine each cluster based on the user scenario data; it determines anomaly screening indicators and then determines abnormal clusters within each cluster based on these indicators; in response to determining that the user scenario data corresponding to the user application request matches the abnormal clusters, it intercepts the user application request. By identifying similar patterns of users based on scenario data, it can identify abnormal applications with similar patterns but no actual connection between users, thereby enabling timely detection of new fraudulent activities and effectively preventing financial losses.

[0096] Figure 2 This is a schematic flowchart of the anomaly detection method according to the second embodiment of this application, as follows: Figure 2 As shown, the anomaly detection methods include:

[0097] Step S201: Receive user application request and determine the corresponding scene identifier.

[0098] Step S202: Based on the scene identifier, obtain user scene data for a historical preset time period.

[0099] Step S203: Invoke the clustering model to determine each cluster based on user scenario data.

[0100] Step S204: Determine the anomaly screening indicators and determine the mean and standard deviation of each anomaly screening indicator in each cluster.

[0101] Step S205: Based on the mean and standard deviation, identify the abnormal clusters in each cluster.

[0102] Specifically, based on the mean and standard deviation, outlier clusters within each cluster are identified, including:

[0103] Based on the mean and standard deviation, abnormal reference values ​​are generated; the clusters in which the index values ​​are greater than the abnormal reference values ​​are identified, and thus the abnormal clusters are determined.

[0104] Specifically, the class containing the index value greater than "mean + 2 * standard deviation" is recorded. If a class has m or more index values ​​greater than "mean + 2 * standard deviation", the class is marked as an abnormal cluster.

[0105] Step S206: In response to determining that the user scenario data corresponding to the user application request matches the abnormal cluster, the user application request is intercepted.

[0106] Calculate the distance from the centroid of the cluster corresponding to the user scenario data of the user application request to the centroid of each abnormal cluster, and take the abnormal cluster with the smallest distance as the classification of the user scenario data corresponding to the user application request.

[0107] This application embodiment identifies similar patterns of users based on scene data, without requiring actual connections between users, and can identify abnormal applications that have similar patterns but no actual connection.

[0108] Figure 3 This is a schematic diagram illustrating an application scenario of the anomaly detection method according to the third embodiment of this application. The anomaly detection method of this application can be applied to scenarios where users apply for loans. Figure 3 As shown, server 302 receives user application request 301 and determines the corresponding scene identifier 303. Based on scene identifier 303, server 302 obtains historical user scene data 304 for a preset historical time period. Server 302 calls clustering model 305 to determine each cluster 306 (which may include cluster 1, cluster 2, ..., cluster n) based on user scene data 304. Server 302 determines anomaly screening indicators 307, and then determines abnormal clusters 308 in each cluster 306 based on anomaly screening indicators 307. In response to determining that the user scene data corresponding to user application request 301 matches abnormal cluster 308, server 302 intercepts user application request 301. Figure 3 The process of determining each cluster 306 and the process of determining the anomaly screening index 307 can be performed simultaneously. After determining each cluster 306 and the anomaly screening index 307, the executing entity can determine the abnormal cluster 308 based on the determined clusters 306 and the anomaly screening index 307. Then, it calls the user scenario data 309 corresponding to the user application request and matches it with the abnormal cluster 308. When the match is successful, it indicates that the received user application request is similar to the pattern of the abnormal application, and the user application request is intercepted.

[0109] This application's embodiments identify similar patterns among users based on scenario data, without requiring any actual connection between users. It can identify anomalous applications with similar patterns but no real connection. The machine learning-based anomaly detection method in this application combines supervised and unsupervised learning to differentiate business scenarios. It quickly identifies and intercepts anomalous loan applications with similar patterns that are frequently initiated within a short period, and sends the customers of the anomalous applications and their triggering patterns to manual review. This application's embodiments identify anomalies by recognizing similar patterns among different users within the same time period, without relying on historical behavior. It identifies similar patterns among users based on scenario data, without requiring any actual connection between users, and can identify anomalous applications with similar patterns but no real connection.

[0110] This application establishes different clustering models for different credit granting scenarios. This has the following advantages: reduced dimensionality of clustering data (clustering data only includes data used in each specific credit granting scenario); improved computational performance (lower dimensionality leads to faster computation); reduced cluster size (most customers only have or authorize a single or limited amount of credit granting data; using data from multiple scenarios for a single cluster results in too few clusters, potentially grouping customers with missing data from one or more scenarios into a single group without further subdivision, making it unsuitable for identifying abnormal customers); and improved clustering accuracy (clustering within the same scenario allows for better identification of abnormal customers within that scenario).

[0111] In this application's embodiments, supervised learning and unsupervised learning are used: Supervised learning involves training with labeled samples to find the relationship between features and labels. Unsupervised learning involves training with unlabeled samples to reveal the inherent properties and patterns of the data. Clustering: Clustering is a type of unsupervised learning algorithm. Clustering divides samples in a dataset into several clusters, resulting in high similarity between samples within a cluster and low similarity between samples between clusters. Clustering algorithms include hierarchical clustering, k-means algorithm, and density distance, etc. Time series analysis assumes that the occurrence of anomalies depends on time.

[0112] Figure 5 This is a schematic diagram of the main units of an anomaly detection device according to an embodiment of this application. Figure 5 As shown, the anomaly detection device includes a receiving unit 501, an acquisition unit 502, a cluster determination unit 503, an anomaly cluster determination unit 504, and an anomaly detection unit 505.

[0113] The receiving unit 501 is configured to receive user application requests and determine the corresponding scene identifier.

[0114] The acquisition unit 502 is configured to acquire user scene data for a preset historical time period based on scene identifiers.

[0115] Cluster determination unit 503 is configured to invoke a clustering model to determine each cluster based on user scenario data.

[0116] The abnormal cluster determination unit 504 is configured to determine anomaly screening indicators, and then determine the abnormal clusters in each cluster based on the anomaly screening indicators.

[0117] The anomaly detection unit 505 is configured to intercept the user application request in response to determining that the user scenario data corresponding to the user application request matches the anomaly cluster.

[0118] In some embodiments, the anomaly detection device further includes Figure 5 The training unit (not shown) is configured to: acquire a training sample set, which includes user application data corresponding to the same scenario identifier, including abnormal applications with time clustering; perform feature engineering on the user application data, and then select clustering indicators based on a decision tree; update the nodes of the decision tree based on the selected clustering indicators, and then generate a clustering model based on the updated decision tree.

[0119] In some embodiments, the training unit is further configured to: pre-classify user application data for all time periods using a decision tree, and then determine clustering metrics based on the pre-classification results.

[0120] In some embodiments, the training unit is further configured to: determine the proportion of abnormal samples in each node of the decision tree; sort the proportion of abnormal samples in each node; select a preset number of nodes based on the sorting; and then determine the labels corresponding to the preset number of nodes as clustering indicators.

[0121] In some embodiments, the abnormal cluster determination unit 504 is further configured to: determine the mean and standard deviation of each anomaly screening index in each cluster; and determine the abnormal clusters in each cluster based on the mean and standard deviation.

[0122] In some embodiments, the abnormal cluster determination unit 504 is further configured to: generate abnormal reference values ​​based on the mean and standard deviation; determine the cluster in which the index value is greater than the abnormal reference value, and then determine it as an abnormal cluster.

[0123] In some embodiments, the anomaly detection device further includes Figure 5 The update unit, not shown, is configured to automatically update user scenario data for a preset historical time period every day, thereby automatically updating each cluster.

[0124] It should be noted that the anomaly detection method and anomaly detection device in this application are related in terms of specific implementation, so repeated content will not be described again.

[0125] Figure 6 An exemplary system architecture 600 is shown that can be applied to the anomaly detection method or anomaly detection device according to the embodiments of this application.

[0126] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, and 603, a network 604, and a server 605. Network 604 serves as the medium for providing communication links between terminal devices 601, 602, and 603 and server 605. Network 604 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0127] Users can use terminal devices 601, 602, and 603 to interact with server 605 via network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0128] Terminal devices 601, 602, and 603 can be various electronic devices with an anomaly detection and processing screen that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0129] Server 605 can be a server providing various services, such as a backend management server supporting user application requests submitted by users using terminal devices 601, 602, and 603 (this is just an example). The backend management server can receive user application requests and determine the corresponding scenario identifier; based on the scenario identifier, it can obtain user scenario data for a preset historical time period; it can call a clustering model to determine each cluster based on the user scenario data; it can determine anomaly screening indicators, and then determine abnormal clusters within each cluster based on these indicators; in response to determining that the user scenario data corresponding to the user application request matches the abnormal clusters, it can intercept the user application request. By identifying similar patterns of users based on scenario data, no actual connection between users is required. It can identify abnormal applications with similar patterns but no actual connection, thereby enabling timely detection of new fraudulent activities and effectively preventing financial losses.

[0130] It should be noted that the anomaly detection method provided in this application embodiment is generally executed by server 605, and correspondingly, the anomaly detection device is generally set in server 605.

[0131] It should be understood that Figure 6 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0132] The following is for reference. Figure 7 It shows a schematic diagram of the structure of a computer system 700 suitable for implementing a terminal device according to the embodiments of this application. Figure 7 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0133] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 702 or programs loaded from storage section 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the computer system 700. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0134] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0135] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the functions defined above in the system of this application.

[0136] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0138] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including a receiving unit, an acquiring unit, a cluster determination unit, an abnormal cluster determination unit, and an anomaly detection unit. The names of these units do not necessarily limit the specific unit itself.

[0139] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to receive a user application request, determine the corresponding scene identifier; obtain user scene data for a preset historical time period based on the scene identifier; invoke a clustering model to determine each cluster based on the user scene data; determine anomaly screening indicators, and then determine abnormal clusters within each cluster based on the anomaly screening indicators; and, in response to determining that the user scene data corresponding to the user application request matches an abnormal cluster, intercept the user application request.

[0140] The computer program product of this application includes a computer program that, when executed by a processor, implements the anomaly detection method in the embodiments of this application.

[0141] According to the technical solution of this application embodiment, by identifying similar patterns of users based on scene data, it is not necessary for users to have an actual relationship. It can identify abnormal applications that have no actual relationship but have similar patterns, thereby timely detecting new fraudulent behavior and effectively avoiding property loss.

[0142] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An anomaly detection method characterized by, include: The system receives an image sent by the terminal, calls the request generation unit to identify the image, generates a user application request, and determines the corresponding scene identifier. Based on the scene identifier, obtain user scene data for a historical preset time period; A clustering model is invoked to determine each cluster based on the user scenario data. Specifically, based on the user scenario data, the scenario is determined to be either a housing provident fund scenario or a credit scenario. Features such as individual contribution base, account balance, number of contributing units, longest continuous housing provident fund contribution period, and housing provident fund cessation contribution period corresponding to all nodes along the path are extracted and input into the clustering model. Leaf nodes with a high proportion of abnormal samples are selected, and all branch paths from them to the root node are traced back to obtain the path. Feature engineering is performed on the user scenario data, and the feature-engineered user scenario data is binned and IV calculated to filter monitoring indicators and set early warning thresholds. The process of determining anomaly screening indicators and then determining anomalous clusters in each cluster based on the anomaly screening indicators includes: determining the mean and standard deviation of each anomaly screening indicator in each cluster; generating anomaly reference values ​​based on the mean and standard deviation; and determining the clusters in which the indicators with values ​​greater than the anomaly reference values ​​belong as anomalous clusters. In response to determining that the user scenario data corresponding to the user application request matches the abnormal cluster, the user application request is intercepted and sent to the approval node; wherein, each cluster is automatically iterated offline daily through the clustering model, and in response to the abnormal cluster, online real-time monitoring of users applying for the corresponding scenario is initiated on the second day.

2. The method according to claim 1, characterized in that, Prior to invoking the clustering model, the method further includes: Obtain a training sample set, which includes user application data corresponding to the same scenario identifier, including abnormal applications with time clustering. The user application data is subjected to feature engineering, and then clustering indicators are selected based on decision trees. Based on the clustering indicators obtained through screening, the nodes of the decision tree are updated, and then a clustering model is generated based on the updated decision tree.

3. The method according to claim 2, characterized in that, The clustering index selection based on decision trees includes: The user application data for all time periods is pre-classified using a decision tree, and then clustering metrics are determined based on the pre-classification results.

4. The method according to claim 3, characterized in that, The step of determining clustering indices based on pre-classification results includes: Determine the proportion of outlier samples in each node of the decision tree; The abnormal sample proportions of each node are sorted, a preset number of nodes are selected based on the sorting, and the labels corresponding to the preset number of nodes are determined as clustering indicators.

5. The method according to claim 1, characterized in that, The method further includes: The system automatically updates the user scenario data for the historical preset time period every day, thereby automatically updating each cluster.

6. An anomaly detection device, characterized in that, include: The receiving unit is configured to receive images sent by the terminal, call the request generation unit to identify the images to generate user application requests, and determine the corresponding scene identifier; The acquisition unit is configured to acquire user scenario data for a historical preset time period based on the scenario identifier; The cluster determination unit is configured to invoke a clustering model to determine each cluster based on the user scenario data. Specifically, based on the user scenario data, the scenario is determined to be either a housing provident fund scenario or a credit scenario. Features such as individual contribution base, account balance, number of contributing units, longest continuous housing provident fund contribution period, and housing provident fund cessation contribution period corresponding to all nodes along the path are extracted and input into the clustering model. Leaf nodes with a high proportion of abnormal samples are selected, and all branch paths from them to the root node are traced back to obtain the path. Feature engineering is performed on the user scenario data, and the feature-engineered user scenario data is binned and IV calculated to filter monitoring indicators and set early warning thresholds. An abnormal cluster determination unit is configured to determine anomaly screening indicators, and then determine abnormal clusters in each cluster based on the anomaly screening indicators; An anomaly detection unit is configured to intercept the user application request and send it to the approval node in response to determining that the user scenario data corresponding to the user application request matches the anomaly cluster; wherein, each cluster is automatically iterated offline daily through the clustering model, and in response to the anomaly cluster, online real-time monitoring of users applying for the corresponding scenario is initiated on the second day; The abnormal cluster determination unit is further configured to: determine the mean and standard deviation of each of the abnormal screening indicators in each cluster; generate an abnormal reference value based on the mean and the standard deviation; and determine the cluster in which the indicator value is greater than the abnormal reference value belongs, and identify it as an abnormal cluster.

7. The apparatus according to claim 6, characterized in that, The anomaly detection device further includes a training unit, configured to: Obtain a training sample set, which includes user application data corresponding to the same scenario identifier, including abnormal applications with time clustering. The user application data is subjected to feature engineering, and then clustering indicators are selected based on decision trees. Based on the clustering indicators obtained through screening, the nodes of the decision tree are updated, and then a clustering model is generated based on the updated decision tree.

8. The apparatus according to claim 7, characterized in that, The training unit is further configured to: The user application data for all time periods is pre-classified using a decision tree, and then clustering metrics are determined based on the pre-classification results.

9. The apparatus according to claim 8, characterized in that, The training unit is further configured to: Determine the proportion of outlier samples in each node of the decision tree; The abnormal sample proportions of each node are sorted, a preset number of nodes are selected based on the sorting, and the labels corresponding to the preset number of nodes are determined as clustering indicators.

10. An anomaly detection electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images