Abnormality processing method and device, storage medium and electronic equipment

Abstract

The invention discloses an exception handling method and device, a storage medium and electronic equipment. The method comprises the steps of obtaining feature information of a detection object, performing anomaly detection on the feature information based on an anomaly perception model, and determining an anomaly detection object; performing clustering processing on the plurality of anomaly detection objects to obtain a plurality of anomaly clusters; and for any abnormal cluster, performing abnormal judgment on the feature information of each detection object in the abnormal cluster, determining an abnormal feature, and generating an abnormal explanation of the abnormal cluster based on the abnormal feature. By detecting the generalization detection capability of the anomaly perception model, the comprehensiveness of anomaly detection is improved, and by performing anomaly judgment on the feature information of the anomaly detection object and determining the anomaly features, the interpretability of the anomaly detection model is realized without depending on expert experience.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an exception handling method, device, storage medium and electronic device. Background Art

[0002] Anomaly detection is widely used in various business fields. Currently, anomaly detection is generally achieved in two ways: one is strategy judgment, which relies on expert experience, has strong interpretability, and can directly identify anomalies based on strategies; the other is learning model judgment, which has strong generalization ability and strong anomaly detection capability.

[0003] In the process of implementing the present invention, it is found that there are at least the following technical problems in the prior art: the strategy judgment method is heavily dependent on expert experience, and when new anomalies appear, they cannot be discovered in time, affecting the accuracy of anomaly detection. The judgment method of the learning model is poorly interpretable and cannot show the cause of anomaly detection to the user. Summary of the invention

[0004] The present invention provides an exception processing method, device, storage medium and electronic equipment to achieve accurate exception detection and exception interpretation.

[0005] According to one aspect of the present invention, there is provided an exception handling method, comprising:

[0006] Acquire feature information of a detection object, perform anomaly detection on the feature information based on an anomaly perception model, and determine an abnormal detection object;

[0007] Performing clustering processing on the multiple anomaly detection objects to obtain multiple anomaly clusters;

[0008] For any of the abnormal clusters, abnormality determination is performed on feature information of each detection object in the abnormal cluster to determine abnormal features, and an abnormal explanation of the abnormal cluster is generated based on the abnormal features.

[0009] Optionally, performing abnormality determination on feature information of each detection object in the abnormal cluster to determine abnormal features includes:

[0010] Identifying the feature contribution of the feature information in the abnormal cluster, and determining the key features corresponding to the abnormal cluster based on the feature contribution;

[0011] Based on the determination conditions of each of the key features, the feature values ​​of the key features in the abnormal cluster are determined, and the abnormal features are determined according to the determination results of each of the key features.

[0012] Optionally, the identifying the feature contribution of the feature information in the abnormal cluster includes:

[0013] Acquire feature information of a normal detection object, set a white label for the normal detection object, and set a black label for the abnormal detection object in the abnormal cluster;

[0014] A contribution recognition model is obtained based on the feature information of the abnormal detection object and the feature training of the normal detection object, and the feature information of the abnormal detection object is input into the contribution recognition model to obtain the feature contribution of each feature information.

[0015] Optionally, performing abnormality determination on feature information of each detection object in the abnormal cluster to determine abnormal features includes:

[0016] Traversing the feature information of each detection object in the abnormal cluster, and determining the data distribution of each feature information;

[0017] The feature concentration of the feature information is determined based on the data distribution of the feature information, and whether the feature information is an abnormal feature is determined based on the feature concentration of the feature information.

[0018] Optionally, performing anomaly detection on the feature information based on an anomaly perception model to determine an anomaly detection object includes:

[0019] The feature information of the detection object is converted into a feature vector through a feature processing model, and the feature vector is classified as abnormal through a feature classification module to obtain an abnormal detection result of the detection object, and whether the detection object is an abnormal detection object is determined according to the abnormal detection result.

[0020] Optionally, clustering the multiple anomaly detection objects to obtain multiple anomaly clusters includes:

[0021] Clustering is performed on the feature vectors of the multiple anomaly detection objects to obtain multiple anomaly clusters.

[0022] Optionally, after obtaining at least two abnormal clusters, the method further includes:

[0023] For any of the abnormal clusters, determine the verification index of the detection object in the abnormal cluster, and judge the verification index based on the judgment threshold to obtain a judgment result, wherein the judgment result includes abnormality and misjudgment;

[0024] Delete the abnormal cluster whose determination result is misjudgment.

[0025] Optionally, the method further includes:

[0026] An abnormality determination strategy is generated according to the abnormality feature, and the abnormality determination strategy is used to screen abnormality detection objects.

[0027] Optionally, the detection object is an electronic device, and the abnormality detection of the detection object is advertising traffic abnormality detection;

[0028] The characteristic information of the detection object includes device attribute information of the electronic device and operation information on the application.

[0029] According to another aspect of the present invention, there is provided an exception handling device, comprising:

[0030] An abnormality sensing module is used to obtain feature information of a detection object, perform abnormality detection on the feature information based on an abnormality sensing model, and determine an abnormal detection object;

[0031] A clustering module, used for clustering the plurality of abnormal detection objects to obtain a plurality of abnormal clusters;

[0032] The explanation module is used to, for any of the abnormal clusters, perform abnormality determination on the feature information of each detection object in the abnormal cluster, determine the abnormal features, and generate an abnormal explanation of the abnormal cluster based on the abnormal features.

[0033] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0034] at least one processor; and

[0035] a memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the exception handling method described in any embodiment of the present invention.

[0037] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the exception handling method described in any embodiment of the present invention when executed.

[0038] The technical solution of this embodiment performs generalized anomaly detection through an anomaly perception model, and determines anomaly detection objects from a large number of detection objects. Multiple anomaly clusters are obtained by clustering multiple anomaly detection objects, and different anomaly clusters may correspond to different anomaly causes. By performing anomaly determination on each anomaly cluster, the anomaly features corresponding to each anomaly cluster are determined to interpret the anomaly. By testing the generalized detection capability of the anomaly perception model, the comprehensiveness of anomaly detection is improved, and by performing anomaly determination on the feature information of the anomaly detection object and determining the anomaly features, the interpretability of the anomaly detection model is achieved without relying on expert experience.

[0039] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 is a flow chart of an exception handling method provided by an embodiment of the present invention;

[0042] Figure 2 Schematic diagram of a method for determining a feature contribution provided by an embodiment of the present invention;

[0043] Figure 3 is a schematic diagram of a method for determining characteristic data distribution provided by an embodiment of the present invention;

[0044] Figure 4 is a flow chart of an exception handling method provided by an embodiment of the present invention;

[0045] Figure 5 It is a schematic diagram of a feature processing flow in an advertising traffic anomaly detection scenario provided by an embodiment of the present invention;

[0046] Figure 6 is a structural schematic diagram of an exception handling device provided by an embodiment of the present invention;

[0047] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0050] Figure 1 is a flowchart of an exception handling method provided by an embodiment of the present invention. This embodiment is applicable to the case of detecting an exception and explaining the cause of the exception. The method can be executed by an exception handling device. The exception handling device can be implemented in the form of hardware and / or software. The exception handling device can be configured in electronic devices such as computers, mobile phones, servers, etc. Figure 1 As shown, the method includes:

[0051] S110. Acquire feature information of a detection object, perform anomaly detection on the feature information based on an anomaly perception model, and determine an abnormal detection object.

[0052] S120: performing clustering processing on the multiple abnormal detection objects to obtain multiple abnormal clusters.

[0053] S130: For any of the abnormal clusters, perform abnormality determination on feature information of each detection object in the abnormal cluster, determine abnormal features, and generate an abnormal explanation of the abnormal cluster based on the abnormal features.

[0054] In this embodiment, the detection objects of the anomaly detection process are different in different business scenarios, and accordingly, the feature information required for the detection object for anomaly detection is also different. Here, according to the anomaly detection requirements of the business scenario, the detection object is determined and the feature information of the detection object in the business scenario is obtained. Taking the detection of advertising traffic anomaly as an example, the detection object is an electronic device, wherein the electronic device can be a device that can install an application and operate the application, such as a terminal device such as a mobile phone, a tablet computer, a PC, etc. Correspondingly, the feature information of the electronic device includes device attribute information and operation information on the application. Taking the delivery anomaly detection of express delivery as an example, the detection object is the handheld device of the delivery person, and the handheld device can be a device with delivery node update and positioning functions. Correspondingly, the feature information of the handheld device can be positioning information in the delivery process and related information of the delivery order, etc. The above detection objects and feature information are only examples. Different detection objects and corresponding feature information can be determined according to changes in business scenarios, and this is not limited here. At the same time, the feature information of the detection object can be stored inside the detection object or in the electronic device associated with the detection object. In the case of anomaly detection, the feature information of the detection object is obtained according to the authorization of the detection object.

[0055] In this embodiment, there may be multiple detection objects, and abnormality detection can be performed on multiple detection objects through a machine learning model. The machine learning model here includes but is not limited to a neural network model, a support vector machine, a multi-layer perceptron, etc., and the type of the machine learning model is not limited here. The machine learning model can be obtained by training positive samples and negative samples in a business scenario, wherein the positive sample can be the feature information of the detection object without abnormality in the business scenario, and is set as a white label, and the negative sample can be the feature information of the abnormal detection object in the business scenario, and is set as a black label. Here, the positive and negative samples can be determined by a pre-set abnormality judgment strategy. Optionally, the machine learning model trained with the above positive and negative samples can be an abnormality perception model, which is used to predict the feature information of the detection object, and determine whether the detection object is an abnormal detection object based on the prediction result.

[0056] Optionally, the anomaly perception model may include a feature processing model and an anomaly classification model, wherein the feature processing model is used to convert feature information into a feature vector. By converting feature information into a feature vector, the feature dimension can be increased, deeper information can be mined from a higher dimension, and the classification capability, i.e., the confidence of anomaly detection, can be improved. The anomaly classification model is used to classify the feature vector to obtain anomaly detection results.

[0057] Specifically, anomaly detection is performed on the feature information based on the anomaly perception model to determine the anomaly detection object, including: converting the feature information of the detection object into a feature vector through a feature processing model, performing anomaly classification on the feature vector through a feature classification module to obtain an anomaly detection result of the detection object, and determining whether the detection object is an anomaly detection object based on the anomaly detection result.

[0058] Here, the feature vector may be an embedding vector, wherein the feature information of the detected object includes multiple information. Accordingly, converting the feature information into the feature vector may be converting each piece of information into a vector of a preset length respectively, and splicing the vectors of each piece of information into a feature vector based on the splicing order of the information.

[0059] The abnormal classification model can be a classifier, and the abnormal detection result output by it is the probability that the detection object belongs to the abnormal detection object, for example, it can be data between 0 and 1. By setting the abnormal determination threshold, the output probability is compared with the abnormal determination threshold, and whether the detection object is an abnormal detection object is determined according to the comparison result. The abnormal classification model can also be a binary classifier, and the abnormal detection result output by it includes a first identifier representing the abnormality or a second identifier representing the normality. Whether the detection object is an abnormal detection object can be determined according to the identifier in the abnormal detection result.

[0060] Taking the detection of advertising traffic anomaly as an example, the anomaly perception model here can be an advertising traffic anomaly perception model, which is trained based on the characteristic information of normal electronic devices and the characteristic information of abnormal electronic devices. The advertising traffic anomaly perception model predicts and processes the characteristic information of the electronic device to be detected, determines whether the electronic device has advertising traffic anomaly, and accordingly determines whether the electronic device is an abnormal electronic device.

[0061] Anomaly detection is performed on multiple detection objects respectively to obtain a set of anomaly detection objects. It is understandable that there may be multiple causes for the anomaly, and accordingly, the set of anomaly detection objects may include anomaly detection objects of multiple anomaly types. Taking the advertising traffic anomaly as an example, the types of anomalies that cause electronic devices include but are not limited to the electronic device being a simulator, the electronic device being a group control device, etc. In order to explain the anomaly, a large number of anomaly detection objects are clustered to facilitate the determination of the anomaly causes in sequence.

[0062] Clustering is performed on the multiple anomaly detection objects to obtain multiple anomaly clusters, and the feature information of the detection objects in each anomaly cluster may have similar rules or characteristics. Clustering can be performed by clustering the feature vectors of the multiple anomaly detection objects to obtain multiple anomaly clusters. Each feature is given a unique numerical meaning through the Embedding vector. These meanings may not be reasonably explained in human subjective cognition, but the model can better distinguish these features through continuous learning. The feature vector here can be a feature vector obtained by converting feature information during the anomaly detection process, and the repeated vector conversion process is avoided by direct calling.

[0063] Optionally, clustering is performed on the abnormal detection objects in the set of abnormal detection objects using a K-means clustering algorithm. Optionally, clustering is performed on the abnormal detection objects in the set of abnormal detection objects using a pre-trained clustering model.

[0064] It is understandable that there may be misjudgments in the anomaly detection process. Due to the difference in feature information between the misjudged detection object and the anomaly detection object, the misjudged detection object and the anomaly detection object can be divided into different clusters through the clustering process. Accordingly, after obtaining at least two anomaly clusters, each anomaly cluster is verified to determine whether there is a misjudged anomaly cluster. By verifying the anomaly cluster, the accuracy of the abnormal cluster determination is improved.

[0065] For any of the abnormal clusters, determine the verification index of the detection object in the abnormal cluster, judge the verification index based on the judgment threshold, and obtain a judgment result, wherein the judgment result includes abnormality and misjudgment; delete the abnormal cluster whose judgment result is misjudgment.

[0066] Among them, the verification index for the abnormal cluster can be determined according to the business scenario, and different business scenarios correspond to different verification indicators. The verification index can be one or more, determined according to the verification requirements of the business scenario. The verification index is judged by the set judgment threshold to determine whether the abnormal cluster is a misjudged abnormal cluster, thereby reducing the probability of misjudgment.

[0067] Taking the business scenario of abnormal advertising traffic as an example, the verification indicators include but are not limited to the secondary retention rate and the order rate. Among them, the secondary retention rate can be determined by the ratio of the number of devices that visit the abnormal cluster the next day to the number of devices in the abnormal cluster; the order rate can be determined by the ratio of the number of devices that place orders in the abnormal cluster to the number of devices in the abnormal cluster. Optionally, the judgment threshold may include a secondary retention rate threshold and an order rate threshold, and the secondary retention rate threshold is compared with the secondary retention rate corresponding to the abnormal cluster, and the order rate threshold is compared with the order rate corresponding to the abnormal cluster. When the secondary retention rate corresponding to the abnormal cluster is greater than the secondary retention rate threshold, and / or the order rate corresponding to the abnormal cluster is greater than the order rate threshold, it is determined that the judgment result of the abnormal cluster is a misjudgment. Optionally, a comprehensive verification indicator can be obtained through a preset processing method (such as weighting) through multiple verification indicators, and accordingly, the judgment threshold is a comprehensive judgment threshold, and the comprehensive verification indicators are compared to obtain a judgment result.

[0068] Misjudged anomaly clusters can be deleted to avoid erroneous prediction causes. At the same time, the detection objects in the above anomaly clusters are set as white labels, and the anomaly perception model is updated and trained to improve the accuracy of anomaly detection.

[0069] Optionally, the abnormal clusters can be sorted by verification indicators, and the attention priority of the abnormal clusters can be determined according to the sorting. Correspondingly, the subsequent processing order of the abnormal clusters can be determined according to the priority of the abnormal clusters. At the same time, the priority of the abnormal explanation obtained by each abnormal cluster can be set to provide users with a reference to the importance of the abnormal cause.

[0070] For the verified abnormal clusters, abnormality determination is performed to determine the features that cause the abnormality of the detection object, that is, the abnormal features. The abnormal explanation is obtained by determining the abnormal features, so as to explain the abnormal detection results of the model.

[0071] In some embodiments, an abnormality judgment is performed on the feature information of each detection object in the abnormal cluster to determine the abnormal features, including: identifying the feature contribution of the feature information in the abnormal cluster, and determining the key features corresponding to the abnormal cluster based on the feature contribution; based on the judgment conditions of each of the key features, the feature values ​​of the key features in the abnormal cluster are judged, and the abnormal features are determined according to the judgment results of each of the key features.

[0072] The abnormal cluster includes multiple abnormal detection objects, and the feature information of each abnormal detection object jointly determines its detection result, wherein different features have different contributions to the detection result. By identifying the feature contribution of each feature information, it is convenient to determine the key features that affect the abnormal detection object. Specifically, each feature information can be sorted from large to small based on the feature contribution, and the key features can be selected according to the sorting. The key features can be the feature information at a preset sorting position in the sorting, for example, the feature information in the top three of the sorting, or the feature information in the top five of the sorting.

[0073] In this embodiment, the feature contribution of the feature information in the abnormal cluster is identified by training the contribution recognition model. Optionally, identifying the feature contribution of the feature information in the abnormal cluster includes: obtaining the feature information of the normal detection object, setting the white label of the normal detection object, and setting the black label of the abnormal detection object in the abnormal cluster; training the contribution recognition model based on the feature information of the abnormal detection object and the feature of the normal detection object, inputting the feature information of the abnormal detection object into the contribution recognition model, and obtaining the feature contribution of each feature information.

[0074] The contribution recognition model here can be an Xgboost model. Figure 2 , Figure 2 It is a schematic diagram of a method for determining a feature contribution provided by an embodiment of the present invention. Figure 2 The middle left picture is a schematic diagram of the working principle of the Xgboost model. Figure 2 The middle right figure is a schematic diagram of contribution recognition for abnormal clusters. The Xgboost model is a classification model. The feature information of the detection object can be input into the Xgboost model to obtain the classification result of whether the detection object is abnormal. By using the feature information of the normal detection object as a positive sample and setting a white label, and using the feature information of the abnormal detection object in the abnormal cluster as a negative sample and setting a black label, the Xgboost model can be trained based on the training set formed by the above positive and negative samples to obtain a trained Xgboost model. In the process of processing feature information, the Xgboost model can output the feature contribution of each feature information, see Figure 2 W1, W2…W5 in the left figure. After the Xgboost model is trained, the feature information of each anomaly detection object in the anomaly cluster is input into the Xgboost model to obtain the feature contribution of each feature information output by the Xgboost model. Optionally, the contribution recognition model is obtained by discarding the structure after the feature contribution output layer in the trained Xgboost model.

[0075] It is understandable that a contribution recognition model can be trained for each abnormal cluster to specifically determine the feature contribution corresponding to each feature information in each abnormal cluster. The feature contribution corresponding to the same feature information in different abnormal clusters may be different.

[0076] The key features are obtained by screening according to the feature contribution of each feature information. The feature value of the key feature is determined to determine whether the key feature is an abnormal feature. The determination conditions of the key feature include but are not limited to the judgment threshold and the feature value distribution ratio threshold. The determination conditions of the key feature can be set according to the business scenario, or determined according to the corresponding feature value in the feature information of the normal detection object.

[0077] Optionally, the judgment condition of the key feature is an abnormal judgment condition, and when the characteristic value of the key feature meets the judgment condition, the key feature is determined to be an abnormal feature. It can be understood that the characteristic value of the key feature is a statistical data of multiple characteristic values ​​of the key feature in the abnormal cluster, such as the mean, minimum, maximum, mode, etc.

[0078] In some embodiments, an abnormality determination is performed on the feature information of each detection object in the abnormal cluster to determine the abnormal features, including: traversing the feature information of each detection object in the abnormal cluster to determine the data distribution of each feature information; determining the feature concentration of the feature information based on the data distribution of the feature information, and determining whether the feature information is an abnormal feature based on the feature concentration of the feature information.

[0079] As a set, the characteristic value distribution of each characteristic information in the abnormal cluster set should have the same or similar trend as the characteristic value distribution of each characteristic information in the overall set of the detection object. According to the data distribution of each characteristic information, it is determined whether the characteristic information is an abnormal feature. For example, see Figure 3 , Figure 3 Schematic diagram of a method for determining characteristic data distribution provided by an embodiment of the present invention, wherein Figure 3 This is just an example, and the value of each feature information is not limited to 2. For any feature information, the data distribution of the feature information is determined according to the multiple feature values ​​of the feature information in the abnormal cluster, and the data proportion of each feature value is determined. The data proportion can be determined by the ratio of the detection object data corresponding to the feature value in the abnormal cluster to all the detection object data in the abnormal cluster. For any feature information, the largest data proportion among the data proportions of multiple feature values ​​in the feature information can be determined as the feature aggregation degree of the feature information.

[0080] Optionally, a feature concentration threshold of each feature information is pre-set to determine the feature information by the feature concentration threshold. If the feature concentration of the feature information is greater than the feature concentration threshold, the feature information is determined to be an abnormal feature. Optionally, based on the feature information of the normal detection object, the feature concentration range of each feature information is determined, and if the feature concentration of the feature information in the abnormal cluster exceeds the corresponding feature concentration range, the feature information is determined to be an abnormal feature.

[0081] The abnormal features determined by any of the above methods can generate an abnormal explanation of the abnormal cluster, which is used to explain the abnormal state of the abnormal cluster. The abnormal explanation may include the abnormal features and abnormal feature value abnormal information of the abnormal features, such as the high feature aggregation of the abnormal features, or the feature value of the abnormal features is greater than a threshold, etc.

[0082] Based on the above embodiment, an abnormality determination strategy is generated according to the abnormal feature, and the abnormality determination strategy is used to screen abnormal detection objects. Exemplarily, the abnormality determination strategy may be that the feature value of the feature is greater than a specific value. Taking advertising traffic detection as an example, when the number of device activations is determined to be an abnormal feature, the generated abnormality determination strategy may be that the number of device activations is greater than a specific value.

[0083] At the same time, it is determined whether the anomaly determination strategy exists in the policy library. If not, the anomaly determination strategy is added to the policy library to update the policy library and improve the accuracy and comprehensiveness of policy anomaly detection. By executing the anomaly handling method of this embodiment and continuously updating the policy library, the anomaly detection process can be gradually converted into anomaly determination strategy, thereby improving the efficiency of anomaly detection. At the same time, the generation of anomaly determination strategy does not rely on expert experience, which saves manpower consumption for research and improves the timeliness of anomaly detection.

[0084] The technical solution of this embodiment performs generalized anomaly detection through an anomaly perception model, and determines anomaly detection objects from a large number of detection objects. Multiple anomaly clusters are obtained by clustering multiple anomaly detection objects, and different anomaly clusters may correspond to different anomaly causes. By performing anomaly determination on each anomaly cluster, the anomaly features corresponding to each anomaly cluster are determined to interpret the anomaly. By testing the generalized detection capability of the anomaly perception model, the comprehensiveness of anomaly detection is improved, and by performing anomaly determination on the feature information of the anomaly detection object and determining the anomaly features, the interpretability of the anomaly detection model is achieved without relying on expert experience.

[0085] Based on the above embodiments, the present invention also provides a preferred example of an exception handling method, see Figure 4 , Figure 4The flowchart of an exception handling method provided by an embodiment of the present invention is shown in FIG. 1 . The exception handling method can be executed by an exception handling device, which includes a perception model, a clustering model, a verification model, an explanation model, and a strategy engine. The exception handling method is executed sequentially by the exception handling device.

[0086] In this embodiment, the detection object is an electronic device, and the abnormality detection of the detection object is advertising traffic abnormality detection. The perception model obtains characteristic information of the electronic device, wherein the characteristic information includes device attribute information of the electronic device and operation information of the application. The above characteristic information can be extracted from the basic log of the electronic device. The characteristic information is preprocessed to obtain characteristic information in a specific format. Figure 5 , Figure 5 1 is a schematic diagram of a feature processing flow in an advertising traffic anomaly detection scenario provided by an embodiment of the present invention. Figure 5 The in-site behavior log in is the operation log of the application, which can extract the operation information of the application. At the same time, the statistical features of the alliance, channel, and device dimensions can be obtained through the basic log of the electronic device, and these features can be matched back to the electronic device through feature splicing to obtain complete features. The preprocessed feature information can include discrete features and digital features (or statistical features), or the preprocessed feature information can include hardware attributes, aggregated features, and in-site behavior features.

[0087] The embedding in the perception model (the feature processing model in the above embodiment) converts a feature information into a feature vector. The classifier (the abnormal classification model in the above embodiment) classifies the vector features. The classifier here can be a supervised classifier. The classifier here can be a NN network model. According to the marking results of the abnormal judgment strategy, the devices at the edge of the strategy classification interface and the new cheating methods are recalled.

[0088] The clustering model uses the Kmeans method based on Faiss to cluster the abnormal electronic devices identified in the perception model and obtain multiple abnormal clusters. Faiss is essentially a vector database that can find the most similar samples from the stored database. Kmeans based on Faiss can achieve clustering more efficiently. The output of the perception model is a complex large set, which includes cheating devices with various cheating methods such as simulators and group control, and misjudged electronic devices. Clustering can regularly divide the entire large set into multiple clusters, separate misjudged devices and different types of cheating devices, and analyze them separately.

[0089] The perception model only outputs relatively abnormal devices, and does not completely determine cheating, so there is a problem of misjudgment. The indicator verification in the verification model is a post-verification step, which is used to evaluate the accuracy of the abnormal clusters output by the clustering model, and is combined with more important business indicators, namely retention and orders, to assist in the analysis of abnormal clusters. Abnormal clusters with poor retention rate (retention rate = number of devices visiting the next day / total number of devices in the abnormal cluster) and order rate (order rate = number of devices placing orders / total number of devices in the abnormal cluster) will be paid special attention to, so as to reduce the situation of false filtering.

[0090] The number of features of the interpretation model is greater than or equal to the number of features of the perception model, and the processing process of the feature information is the same as that of the perception model.

[0091] The explanation model includes two implementation methods: one is to identify the feature contribution of the feature through the Xgboost model. The feature contribution can be determined by the number of times the feature is divided into attributes in all trees to determine the feature importance. The electronic devices in the abnormal cluster are used as black labels, and a batch of electronic devices with white labels are randomly selected to train the output feature importance. The feature information with the highest feature contribution score is selected, and the feature value of the feature information is determined to determine the abnormal features and perform abnormal interpretation.

[0092] The second is to determine the characteristic concentration of feature information and analyze from the distribution of data. The abnormal cluster provides a natural collection. Based on the abnormal cluster, we can count the proportion of various indicators and analyze the inside of the abnormal cluster. If it deviates greatly from the overall indicator, it can also be used as a credible explanation.

[0093] The policy engine is a link for executing filtering. It formulates policies based on the abnormal features of the output of the explanation model to achieve precise strikes against abnormal devices. It should be noted that both the Xgboost solution and the feature aggregation solution can be effective separately. The n features output by the explanation model are the angles for formulating policies, and the policies can be developed around these abnormal features. The scale of the feature library may be very large. Through the abnormal traffic perception system, the circle of abnormal features will be continuously narrowed, greatly improving the efficiency of formulating policies. Finally, the specified policy will update the black sample label, and it will be trained again to form a closed loop.

[0094] Figure 6 Schematic diagram of the structure of an exception handling device provided by an embodiment of the present invention. Figure 6 As shown, the device comprises:

[0095] The abnormality sensing module 210 is used to obtain feature information of a detection object, perform abnormality detection on the feature information based on an abnormality sensing model, and determine an abnormal detection object;

[0096] A clustering module 220, configured to perform clustering processing on the plurality of abnormal detection objects to obtain a plurality of abnormal clusters;

[0097] The explanation module 230 is used to, for any of the abnormal clusters, perform abnormality determination on the feature information of each detection object in the abnormal cluster, determine the abnormal features, and generate an abnormal explanation of the abnormal cluster based on the abnormal features.

[0098] Based on the above embodiment, optionally, the interpretation module 230 is used to:

[0099] Identifying the feature contribution of the feature information in the abnormal cluster, and determining the key features corresponding to the abnormal cluster based on the feature contribution;

[0100] Based on the determination conditions of each of the key features, the feature values ​​of the key features in the abnormal cluster are determined, and the abnormal features are determined according to the determination results of each of the key features.

[0101] Optionally, the interpretation module 230 is further configured to:

[0102] Acquire feature information of a normal detection object, set a white label for the normal detection object, and set a black label for the abnormal detection object in the abnormal cluster;

[0103] A contribution recognition model is obtained based on the feature information of the abnormal detection object and the feature training of the normal detection object, and the feature information of the abnormal detection object is input into the contribution recognition model to obtain the feature contribution of each feature information.

[0104] Based on the above embodiment, optionally, the interpretation module 230 is used to:

[0105] Traversing the feature information of each detection object in the abnormal cluster, and determining the data distribution of each feature information;

[0106] The feature concentration of the feature information is determined based on the data distribution of the feature information, and whether the feature information is an abnormal feature is determined based on the feature concentration of the feature information.

[0107] Based on the above embodiment, optionally, the abnormality sensing module 210 is used to:

[0108] The feature information of the detection object is converted into a feature vector through a feature processing model, and the feature vector is classified as abnormal through a feature classification module to obtain an abnormal detection result of the detection object, and whether the detection object is an abnormal detection object is determined according to the abnormal detection result.

[0109] Based on the above embodiment, optionally, the clustering module 220 is used to:

[0110] Clustering is performed on the feature vectors of the multiple anomaly detection objects to obtain multiple anomaly clusters.

[0111] On the basis of the above embodiment, optionally, the device also includes a verification module, which is used to determine the verification index of the detection object in any abnormal cluster after obtaining at least two abnormal clusters, and judge the verification index based on a judgment threshold to obtain a judgment result, wherein the judgment result includes anomalies and misjudgments; and delete the abnormal cluster whose judgment result is a misjudgment.

[0112] On the basis of the above embodiment, optionally, the device further includes a strategy generation module, which is used to generate an abnormality determination strategy according to the abnormality feature, and the abnormality determination strategy is used to screen abnormality detection objects.

[0113] Based on the above embodiment, optionally, the detection object is an electronic device, and the abnormality detection of the detection object is advertising traffic abnormality detection;

[0114] The characteristic information of the detection object includes device attribute information of the electronic device and operation information on the application.

[0115] The exception handling device provided in the embodiment of the present invention can execute the exception handling method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] Figure 7 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0117] like Figure 7As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0118] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as an exception handling method.

[0120] In some embodiments, the exception handling method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the exception handling method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the exception handling method in any other appropriate manner (e.g., by means of firmware).

[0121] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] The computer program for implementing the exception handling method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0123] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a processor to execute an exception handling method, the method comprising:

[0124] Acquire feature information of the detection object, perform anomaly detection on the feature information based on the anomaly perception model, and determine the abnormal detection object; perform clustering processing on multiple abnormal detection objects to obtain multiple abnormal clusters; for any abnormal cluster, perform abnormality judgment on the feature information of each detection object in the abnormal cluster, determine the abnormal features, and generate an abnormal explanation of the abnormal cluster based on the abnormal features.

[0125] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0127] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0128] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0129] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0130] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An exception handling method, characterized in that: include: Acquire feature information of a detection object, perform anomaly detection on the feature information based on an anomaly perception model, and determine an abnormal detection object; Performing clustering processing on the multiple anomaly detection objects to obtain multiple anomaly clusters; For any of the abnormal clusters, abnormality determination is performed on feature information of each detection object in the abnormal cluster to determine abnormal features, and an abnormal explanation of the abnormal cluster is generated based on the abnormal features.

2. The method according to claim 1, characterized in that The step of performing abnormality determination on the feature information of each detection object in the abnormal cluster to determine the abnormal features includes: Identifying the feature contribution of the feature information in the abnormal cluster, and determining the key features corresponding to the abnormal cluster based on the feature contribution; Based on the determination conditions of each of the key features, the feature values ​​of the key features in the abnormal cluster are determined, and the abnormal features are determined according to the determination results of each of the key features.

3. The method according to claim 2, characterized in that The identifying the feature contribution of the feature information in the abnormal cluster includes: Acquire feature information of a normal detection object, set a white label for the normal detection object, and set a black label for the abnormal detection object in the abnormal cluster; A contribution recognition model is obtained based on the feature information of the abnormal detection object and the feature training of the normal detection object, and the feature information of the abnormal detection object is input into the contribution recognition model to obtain the feature contribution of each feature information.

4. The method according to claim 1, characterized in that: The step of performing abnormality determination on the feature information of each detection object in the abnormal cluster to determine the abnormal features includes: Traversing the feature information of each detection object in the abnormal cluster, and determining the data distribution of each feature information; The feature concentration of the feature information is determined based on the data distribution of the feature information, and whether the feature information is an abnormal feature is determined based on the feature concentration of the feature information.

5. The method according to claim 1, characterized in that The performing anomaly detection on the feature information based on the anomaly perception model to determine an anomaly detection object includes: The feature information of the detection object is converted into a feature vector through a feature processing model, and the feature vector is classified as abnormal through a feature classification module to obtain an abnormal detection result of the detection object, and whether the detection object is an abnormal detection object is determined according to the abnormal detection result.

6. The method according to claim 1, characterized in that The clustering process is performed on the plurality of abnormal detection objects to obtain a plurality of abnormal clusters, including: Clustering is performed on the feature vectors of the multiple anomaly detection objects to obtain multiple anomaly clusters.

7. The method according to claim 1, characterized in that After obtaining at least two abnormal clusters, it also includes: For any of the abnormal clusters, determine the verification index of the detection object in the abnormal cluster, and judge the verification index based on the judgment threshold to obtain a judgment result, wherein the judgment result includes abnormality and misjudgment; Delete the abnormal cluster whose determination result is misjudgment.

8. The method according to claim 1, characterized in that The method further comprises: An abnormality determination strategy is generated according to the abnormality feature, and the abnormality determination strategy is used to screen abnormality detection objects.

9. The method according to claim 1, characterized in that: The detection object is an electronic device, and the abnormality detection of the detection object is advertising traffic abnormality detection; The characteristic information of the detection object includes device attribute information of the electronic device and operation information on the application.

10. An exception handling device, characterized in that: include: An abnormality sensing module is used to obtain feature information of a detection object, perform abnormality detection on the feature information based on an abnormality sensing model, and determine an abnormal detection object; A clustering module, used for clustering the plurality of abnormal detection objects to obtain a plurality of abnormal clusters; The explanation module is used to, for any of the abnormal clusters, perform abnormality determination on the feature information of each detection object in the abnormal cluster, determine the abnormal features, and generate an abnormal explanation of the abnormal cluster based on the abnormal features.

11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the exception handling method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the exception handling method according to any one of claims 1 to 9 when executed.

Images