Abnormal traffic identification method and device, storage medium, and electronic device

Abstract

The application discloses an abnormal traffic identification method and device, a storage medium and an electronic device. The method comprises the following steps: obtaining target advertisement traffic, wherein the target advertisement traffic is advertisement traffic to be identified as abnormal; preprocessing the target advertisement traffic to obtain to-be-identified data, wherein the preprocessing is used for processing the target advertisement traffic into to-be-identified data meeting preset requirements; extracting features of the to-be-identified data by using a feature extraction model to obtain data features of the to-be-identified data; and calling an abnormal traffic detection model to determine whether the target advertisement traffic is abnormal traffic by using the data features of the to-be-identified data. The application solves the technical problem of low efficiency of abnormal detection in the related art.

Description

Technical Field

[0001] The present application relates to the field of Internet, and specifically, to a method and device for identifying abnormal traffic, a storage medium, and an electronic device. Background Art

[0002] In recent years, with the continuous development and popularization of mobile internet, online advertising, which closely connects brands and products with consumers, has become increasingly popular among advertisers, and the overall scale of the digital marketing market has also expanded rapidly. However, ad fraud, which has accompanied the booming online advertising market, has become a persistent problem in marketing advertising, and preventing online ad fraud has become a major concern for advertisers. To profit illegally, advertising fraud companies use various technical maneuvers to bypass resource restrictions and increase access frequency. They use scripts, simulators, group control, domain name and app manipulation, and crowdsourcing traffic to generate large amounts of invalid traffic, resulting in increasingly rampant ad fraud.

[0003] Currently, most abnormal traffic detection methods use rule-based approaches. Experts classify common abnormal traffic and summarize each anomaly category to derive rules for identifying each anomaly category. These rules are then used to identify and filter the abnormal traffic within the rules within a large volume of traffic. Faced with the continuous iteration of fraudulent activities, this approach lacks generalizability and requires manual intervention (e.g., updating the rules after each iteration to ensure they take effect). The selection of rule thresholds significantly affects the accuracy of abnormal traffic detection.

[0004] Currently, no effective solution has been proposed to address the above-mentioned problem of low efficiency caused by manual participation. Summary of the Invention

[0005] The embodiments of the present application provide a method and device for identifying abnormal traffic, a storage medium, and an electronic device to at least solve the technical problem of low efficiency in abnormality detection in related technologies.

[0006] According to one aspect of an embodiment of the present application, a method for identifying abnormal traffic is provided, including: obtaining target advertising traffic, the target advertising traffic being advertising traffic to be identified as abnormal; preprocessing the target advertising traffic to obtain data to be identified, wherein the preprocessing is used to process the target advertising traffic into data to be identified that meets preset requirements; using a feature extraction model to extract features from the data to be identified to obtain data features of the data to be identified; calling an abnormal traffic detection model, and using the data features of the data to be identified to determine whether the target advertising traffic is abnormal traffic.

[0007] Optionally, an abnormal traffic detection model is called to use the data features of the data to be identified to determine whether the target advertising traffic is abnormal traffic, including: obtaining an abnormal traffic detection model, wherein the abnormal traffic detection model is trained based on a semi-supervised network of a graph convolutional neural network and a class prototype vector; calling the abnormal traffic detection model to use the saved feature mapping layer parameters and the class prototype vector to identify the data features of the data to be identified to determine whether the target advertising traffic is abnormal traffic.

[0008] Optionally, obtaining an abnormal traffic detection model includes: obtaining original advertising traffic, wherein the original advertising traffic is advertising traffic used for training; preprocessing the original advertising traffic to process the original advertising traffic into advertising traffic samples that meet preset requirements; performing feature extraction and feature combination statistics on the advertising traffic samples to obtain a feature extraction model; constructing a semi-supervised network model based on a graph convolutional neural network and a class prototype vector, and using the data features of the advertising traffic samples to train the semi-supervised network model to obtain an abnormal traffic detection model.

[0009] Optionally, a semi-supervised network model based on a graph convolutional neural network and a class prototype vector is constructed, including: constructing a graph convolutional neural network GCN: taking each advertising traffic sample as a node in the graph convolutional neural network GCN, if the user identifiers of the advertising traffic samples of two nodes are the same, then an edge between the two nodes is established and the association weight between the two nodes is set to 1; if the user identifiers of the advertising traffic samples of the two nodes are different, then no edge between the two nodes is established and the association weight between the two nodes is set to 0; constructing a semi-supervised network model based on the graph convolutional neural network GCN and a class prototype vector.

[0010] Optionally, a semi-supervised network model based on a graph convolutional neural network (GCN) and a class prototype vector is constructed, including constructing a semi-supervised network model including the following network: an input layer, the input of the input layer includes a data set of data features of an advertising traffic sample; a graph convolution layer, the graph convolution layer is used to convolve the data set to obtain a pseudo-label probability matrix; a feature mapping layer, the feature mapping layer is used to map data features to aggregate projection vectors of data features of the same category and alienate projection vectors of data features of different categories; a class contribution layer, the class contribution layer is used to determine the contribution of the sample to the constructed class prototype vector, wherein the closer the projection vector of the sample in the feature space is to the class prototype vector, the greater the contribution; a prototype vector layer, the prototype vector layer is used to determine the class prototype vector; a classifier layer, the classifier layer is used to determine the category to which the sample belongs and the probability of belonging to the category.

[0011] Optionally, the advertisement traffic sample includes a marked traffic sample and an unmarked traffic sample.

[0012] Optionally, the original advertisement traffic is preprocessed, including at least one of the following: data alignment on the original advertisement traffic; data type conversion on the original advertisement traffic; data encoding on the original advertisement traffic; missing value filling on the original advertisement traffic; duplicate removal processing on the original advertisement traffic; data bucketing processing on the original advertisement traffic; and feature standardization processing on the original advertisement traffic.

[0013] According to another aspect of the embodiments of the present application, an abnormal traffic identification device is further provided, including: an acquisition unit configured to acquire target advertisement traffic, wherein the target advertisement traffic is advertisement traffic to be identified for abnormality; a preprocessing unit configured to preprocess the target advertisement traffic to obtain to-be-identified data, wherein the preprocessing is configured to process the target advertisement traffic into to-be-identified data meeting preset requirements; a feature extraction unit configured to extract features of the to-be-identified data by using a feature extraction model; and an identification unit configured to call an abnormal traffic detection model and determine whether the target advertisement traffic is abnormal traffic by using the features of the to-be-identified data.

[0014] According to another aspect of the embodiments of the present application, a storage medium is further provided, which includes a stored program, and the program performs the above method when running.

[0015] According to another aspect of the embodiments of the present application, an electronic device is further provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor performs the above method by using the computer program.

[0016] According to an aspect of the present application, a computer program product or a computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device perform the steps of any of the above embodiments.

[0017] The present application can be used in EIP products to support data capabilities (such as data mining). In the embodiments of the present application, target advertisement traffic is acquired, the target advertisement traffic is advertisement traffic to be identified for abnormality; the target advertisement traffic is preprocessed to obtain to-be-identified data, the preprocessing is configured to process the target advertisement traffic into to-be-identified data meeting preset requirements; features of the to-be-identified data are extracted by using a feature extraction model; and an abnormal traffic detection model is called to determine whether the target advertisement traffic is abnormal traffic by using the features of the to-be-identified data. The artificial intelligence technology can be used to realize rapid detection of advertisement traffic abnormality, and the semi-supervised model can greatly reduce the requirement for samples, thereby solving the technical problem of low efficiency of abnormality detection in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a schematic diagram of a hardware environment for a method for identifying abnormal traffic according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of an optional method for identifying abnormal traffic according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an optional abnormal traffic identification solution according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of an optional abnormal flow identification device according to an embodiment of the present application; and

[0023] Figure 5 This is a structural block diagram of a terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] According to one aspect of the embodiments of the present application, a method embodiment of a method for identifying abnormal traffic is provided.

[0027] Optionally, in this embodiment, the above abnormal traffic identification method can be applied to Figure 1 In the hardware environment composed of the terminal 101 and the server 103 shown in FIG. Figure 1 As shown, the server 103 is connected to the terminal 101 via a network, and can be used to provide services (such as game services, application services, etc.) to the terminal or the client installed on the terminal. A database 105 can be set on the server or independently of the server to provide data storage services for the server 103. The above-mentioned network includes but is not limited to: a wide area network, a metropolitan area network or a local area network, and the terminal 101 is not limited to a PC, a mobile phone, a tablet computer, etc. The method for identifying abnormal traffic in the embodiment of the present application can be executed by the server 103, or by the terminal 101, or by the server 103 and the terminal 101. Among them, the terminal 101 executes the method for identifying abnormal traffic in the embodiment of the present application or it can be executed by the client installed thereon.

[0028] Figure 2 This is a flow chart of an optional method for identifying abnormal traffic according to an embodiment of the present application, such as Figure 2 As shown, the method may include the following steps:

[0029] Step S202: construct and train an abnormal traffic detection model.

[0030] Step S2022: Obtain original advertising traffic. The original advertising traffic is the advertising traffic used for training. The advertising traffic here can be the process generated by normal access or the traffic generated by the advertising black market using some abnormal means to access the advertisement.

[0031] Step S2024: pre-process the original advertisement traffic to convert the original advertisement traffic into an advertisement traffic sample that meets preset requirements.

[0032] It should be noted that if the original advertising traffic has been labeled (marked as abnormal traffic), the preprocessing here is to retain the label. If there is no label, part of the traffic can be labeled during preprocessing. This is equivalent to the obtained advertising traffic samples including labeled traffic samples and unlabeled traffic samples.

[0033] Optionally, the original advertising traffic is preprocessed, including at least one of the following: data alignment of the original advertising traffic to ensure that the input data length is consistent; data type conversion of the original advertising traffic; data encoding of the original advertising traffic; missing value filling of the original advertising traffic; deduplication of the original advertising traffic; data bucketing of the original advertising traffic; and feature standardization of the original advertising traffic.

[0034] Step S2026: perform feature extraction and feature combination statistics on the advertising traffic samples to obtain a feature extraction model.

[0035] When constructing a feature extraction model for advertising traffic data, since the monitoring system collects a large number of traffic data variables and there is redundancy between the variables, it is necessary to extract features from the traffic data variables, mainly including statistical feature combinations and automatic feature extraction based on deep learning models.

[0036] Step S2028: construct a semi-supervised network model based on graph convolutional neural network and class prototype vector, and use the data features of advertising traffic samples to train the semi-supervised network model to obtain an abnormal traffic detection model.

[0037] Alternatively, a semi-supervised network model based on graph convolutional neural networks and class prototype vectors can be constructed as follows:

[0038] 1) Constructing a graph convolutional neural network (GCN): Each ad traffic sample is used as a node in the GCN. If the user IDs of the ad traffic samples of two nodes are the same, an edge is established between the two nodes and the association weight between the two nodes is set to 1. If the user IDs of the ad traffic samples of two nodes are different, no edge is established between the two nodes and the association weight between the two nodes is set to 0.

[0039] 2) Build a semi-supervised network model based on graph convolutional neural network (GCN) and class prototype vectors. This includes the following semi-supervised network models:

[0040] Input layer: the input of the input layer includes a data set of data features of advertising traffic samples; graph convolution layer: the graph convolution layer is used to convolve the data set to obtain a pseudo-label probability matrix; feature mapping layer: the feature mapping layer is used to map data features to aggregate the projection vectors of data features of the same category and alienate the projection vectors of data features of different categories; class contribution layer: the class contribution layer is used to determine the contribution of the sample to the constructed class prototype vector, where the closer the projection vector of the sample in the feature space is to the class prototype vector, the greater the contribution; prototype vector layer: the prototype vector layer is used to determine the class prototype vector; classifier layer: the classifier layer is used to determine the category to which the sample belongs and the probability of belonging to the category.

[0041] After completing the model construction and training, you can use the model to monitor abnormal traffic by following the steps below:

[0042] Step S204: acquiring target advertising traffic, where the target advertising traffic is the advertising traffic to be identified as abnormal.

[0043] In step S206, the target advertising traffic is preprocessed to obtain the to-be-recognized data. The preprocessing is used to process the target advertising traffic into to-be-recognized data meeting preset requirements. The preprocessing is specifically described above.

[0044] In step S208, a feature extraction model is used to extract features of the to-be-recognized data, to obtain data features of the to-be-recognized data. The extraction mainly includes extraction of general features and statistical features.

[0045] In step S210, an abnormal traffic detection model is called to determine whether the target advertising traffic is abnormal traffic by using the data features of the to-be-recognized data. The abnormal traffic detection model is trained by using a semi-supervised learning method. The semi-supervised learning method greatly reduces the requirement for data samples and greatly enhances the practicability.

[0046] In order to prevent the endless forms of advertising fraud, artificial intelligence technology can be applied to the advertising traffic anomaly detection in the advertising marketing industry. The main process of the advertising traffic anomaly detection based on artificial intelligence technology is data collection, feature engineering, and prediction based on a machine learning algorithm. For example, a supervised machine learning model is used to convert the abnormal traffic detection problem into a classification problem, and then the relationship between the input data and the traffic sample label is mined to realize advertising traffic anti-fraud. However, in order to obtain high-precision detection performance, a supervised learning model often needs a sufficient number of data sets with class labels for training, which is difficult to meet in the actual advertising industry. In actual advertising traffic monitoring, although a large amount of advertising traffic data is generated every day, the data samples with class labels are very limited. Instead, most of the traffic data is unclassified, and the labeling of the traffic class label also consumes a lot of manpower and material resources, and the labeling quality is difficult to judge. Therefore, in the actual business scenario, it is difficult to realize the abnormal detection of advertising traffic only by relying on a supervised model.

[0047] Through the technical solution of the present application, the target advertising traffic is obtained, the target advertising traffic is the advertising traffic to be recognized as abnormal; the target advertising traffic is preprocessed to obtain the to-be-recognized data, the preprocessing is used to process the target advertising traffic into to-be-recognized data meeting preset requirements; the feature extraction model is used to extract features of the to-be-recognized data, to obtain data features of the to-be-recognized data; the abnormal traffic detection model is called to determine whether the target advertising traffic is abnormal traffic by using the data features of the to-be-recognized data. The artificial intelligence technology can realize the rapid detection of advertising traffic anomalies, and the semi-supervised model can greatly reduce the requirement for samples, and can solve the technical problem of low efficiency of abnormal detection in the related art.

[0048] In the technical solution of the present application, a method for detecting abnormal advertising traffic based on a semi-supervised graph convolutional prototype network is proposed. This method constructs a graph structure model between advertising traffic data, uses a graph convolutional network to learn and mine the proximity relationship between advertising traffic data, makes full use of unlabeled data information, and considers the impact of unlabeled data on the contribution of the classifier. The prototype vector classification method is used to train the abnormal traffic detection classifier.

[0049] As an optional embodiment, the following Figure 3 The specific implementation method further details the technical solution of this application:

[0050] Step 1: Preprocess the advertising traffic data.

[0051] Preprocessing of advertising traffic data. Due to the limitations of the monitoring system, the actual advertising traffic data collected often has the characteristics of missing, noisy, multivariate, and heterogeneous. Therefore, before analysis, it must be preprocessed, mainly including data alignment, data type conversion, data encoding, missing value filling, deduplication, data bucketing, feature standardization, etc.

[0052] Step 2: Build a feature extraction model for advertising traffic data.

[0053] A feature extraction model is constructed for advertising traffic data. Since the monitoring system collects a large number of traffic data variables and there is redundancy between the variables, it is necessary to extract features from the traffic data variables, mainly including statistical feature combination and automatic feature extraction based on deep learning models.

[0054] Statistical feature combination uses statistical ideas to perform statistical calculations on the original data, assuming The final advertising traffic dataset contains labeled data With unlabeled datasets The first l samples of are labeled data, and the remaining u samples are unlabeled data.

[0055] Automatic feature extraction based on deep learning models uses autoencoders built by deep learning models, such as convolutional autoencoders and sparse autoencoders, to perform feature mapping on standardized data in the latent space, and train the autoencoders through unsupervised learning methods to extract general abstract feature datasets after dimensionality reduction.

[0056] The graph-based semi-supervised learning (GBSSL) algorithm can propagate the labels of labeled data samples to unlabeled samples based on the adjacency relationship within the graph model. It has a solid mathematical foundation, high accuracy, fast computing speed, avoids waste of data sources, and requires low labor costs. The semi-supervised learning algorithm is suitable for industrial scenarios.

[0057] Semi-supervised graph convolutional networks (GCNs) build graph-structured models for weakly labeled datasets, then perform graph convolution on the graph-structured data to process neighborhood information within the graph. This information is then propagated through layered propagation rules, pseudo-labeling the samples within the dataset (labeled, unlabeled, and test data) to obtain a label probability matrix. During network model training, high-precision classification is achieved with a small amount of labeled data by minimizing the cross-entropy loss function for labeled data nodes.

[0058] Step 3: Build and train an abnormal traffic detection model based on a semi-supervised graph convolutional prototype network.

[0059] Build and train an abnormal traffic detection model based on a semi-supervised graph convolutional prototype network. The specific steps are as follows:

[0060] Step 3.1: Construct the graph model G, taking the i-th flow data sample point X i is the node V of the graph model G i ; Take sample point X i With X j Construct edge E of graph model G based on whether the user IDs are the same. ij , if they are the same, then X i With X j There is an edge between them, with weight a ij =1, otherwise no edge is connected, a ij =0; construct the adjacency matrix A∈R of G in the order of sample numbers N*N , N is the number of all samples in the data set, and the Laplace approximation matrix is ​​calculated based on equations (2-1) to (2-4)

[0061]

[0062]

[0063]

[0064]

[0065] I is the identity matrix, is an element in the self-circularity matrix The elements in is the self-loop adjacency matrix, i and j are the indices of elements in the graph model G.

[0066] Step 3.2, a semi-supervised learning model based on GCN and class prototype vector is constructed, which mainly includes an input layer, a graph convolution layer, a feature mapping layer, a class contribution degree layer, a prototype vector layer and a classifier layer.

[0067] (1) The input of the input layer is the abstract feature data set and the Laplace approximation matrix

[0068] (2) The graph convolution layer performs graph convolution on based on formula (2-5) and (2-6) to obtain the pseudo label probability matrix Z:

[0069]

[0070]

[0071] where H 1 is the output of the first layer graph convolution layer, W 1 and W 2 are the weights of the first layer graph convolution layer and the second layer graph convolution layer respectively.

[0072] (3) The feature mapping layer performs feature mapping on the input data based on formula (2-7), and the projection vectors of the same category data are aggregated, and the projection vectors of different category data are distant:

[0073] φ(x) = ReLU(xW φ +b φ ); (2-7)

[0074] where W φ and b φ are the weight and bias of the feature mapping layer respectively.

[0075] (4) The class contribution degree layer is based on the principle that the closer the sample is to the class prototype vector in the feature space, the greater the contribution degree, and the greater the variance of the pseudo label probability value, the higher the reliability. Assuming that m ij is the sample x i , the contribution degree of the prototype vector of the sample class c k , var(z i ) is the variance of the pseudo label probability vector of the sample x i generated by the graph convolution layer, w z is the weight of var(z i ), s ik represents the distance between the sample and the prototype in the high-dimensional space, w k and b k are the weight and bias of sik Weights and biases of high-dimensional mappings, w s For s ik The weight, b m is the bias of the class contribution layer, σ is the sigmoid activation function, and the sample x is calculated based on formulas (2-8) and (2-9) i For constructing sample category c k Contribution m of the prototype vector ik :

[0076] m ik =σ(var(z i )w z +σ(s ik w k +b k )w s +b m ), (2-8)

[0077]

[0078] (5) The prototype vector layer calculates the prototype vector of each frequent main fault category in the feature map φ space based on formula (2-9)

[0079]

[0080] A prototype is a representative point in the sample space. By projecting samples from the sample space into a suitable feature space based on the nearest neighbor class mean rule, the feature vectors of similar samples are closer together, while those of heterogeneous samples are farther apart. By calculating the mean of the feature vectors of similar samples, the center of the class to which the group of samples belongs, i.e., the prototype vector, is obtained. During classification tasks, the distance between the sample data's feature vector in the feature space and each class prototype vector and a Sigmoid calculation are used to determine the probability and class of the sample. During classification training, the class prototype vectors are automatically updated as the feature representation changes, making the classifier robust to changes in the feature representation. Consequently, classification algorithms based on class prototype vectors have attracted attention and are being applied in scenarios with a small number of samples, such as small-sample learning and incremental learning.

[0081] (6): The classifier layer is based on formula (2-11), taking the class prototype in the feature space and the labeled data feature vector as input, and calculating the category c to which the sample in the labeled data set belongs k Probability of:

[0082]

[0083] During training, the loss function L of the network is expressed as (2-12):

[0084]

[0085] By minimizing L, the model is optimized, and finally the optimized feature mapping layer parameters and class prototype vector are saved for anomaly detection of new traffic.

[0086] Step 4: Perform anomaly detection on new traffic data based on the class prototype vector.

[0087] Anomaly detection of new advertising traffic data. The main process of this part is: preprocessing new traffic data; extracting general abstract features of the data; and identifying newly collected traffic based on the saved feature mapping layer parameters and class prototype vectors.

[0088] In the technical solution of this application, a semi-supervised graph convolutional prototype network is constructed, and a method for detecting abnormal advertising traffic based on the semi-supervised graph convolutional prototype network is proposed. The propagation of data labels is realized through the similarity between data, and the class prototype vectors of normal and abnormal advertising traffic are calculated at the same time, thereby realizing the abnormal detection and reuse of advertising traffic.

[0089] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0091] According to another aspect of the embodiments of the present application, an abnormal traffic identification device for implementing the above-mentioned abnormal traffic identification method is also provided. Figure 4 is a schematic diagram of an optional abnormal flow identification device according to an embodiment of the present application, such as Figure 4 As shown, the device may include:

[0092] An acquisition unit 41 is used to acquire target advertising traffic, wherein the target advertising traffic is advertising traffic to be identified as abnormal; a preprocessing unit 43 is used to preprocess the target advertising traffic to obtain data to be identified, wherein the preprocessing is used to process the target advertising traffic into the data to be identified that meets preset requirements; a feature extraction unit 45 is used to use a feature extraction model to extract features from the data to be identified to obtain data features of the data to be identified; an identification unit 47 is used to call an abnormal traffic detection model and use the data features of the data to be identified to determine whether the target advertising traffic is abnormal traffic.

[0093] Through the above module, the target advertising traffic is obtained, and the target advertising traffic is the advertising traffic to be identified as abnormal; the target advertising traffic is preprocessed to obtain the data to be identified, and the preprocessing is used to process the target advertising traffic into the data to be identified that meets the preset requirements; the feature extraction model is used to extract features from the data to be identified to obtain the data features of the data to be identified; the abnormal traffic detection model is called, and the data features of the data to be identified are used to determine whether the target advertising traffic is abnormal traffic. The use of artificial intelligence technology can realize the rapid detection of advertising traffic anomalies, and the semi-supervised model can greatly reduce the requirements for samples, which can solve the technical problem of low efficiency of anomaly detection in related technologies.

[0094] Optionally, the device of the present application may also include: a pre-training unit for obtaining an abnormal traffic detection model, wherein the abnormal traffic detection model is trained based on a semi-supervised network of a graph convolutional neural network and a class prototype vector; an identification unit, further for calling the abnormal traffic detection model, and using the saved feature mapping layer parameters and the class prototype vector to identify the data features of the data to be identified, so as to determine whether the target advertising traffic is abnormal traffic.

[0095] Optionally, the pre-training unit is also used to: obtain original advertising traffic, wherein the original advertising traffic is the advertising traffic used for training; pre-process the original advertising traffic to process the original advertising traffic into an advertising traffic sample that meets preset requirements; perform feature extraction and feature combination statistics on the advertising traffic sample to obtain the feature extraction model; construct a semi-supervised network model based on a graph convolutional neural network and a class prototype vector, and use the data features of the advertising traffic sample to train the semi-supervised network model to obtain the abnormal traffic detection model.

[0096] Optionally, the pre-training unit is also used to construct a graph convolutional neural network GCN: taking each of the advertising traffic samples as a node in the graph convolutional neural network GCN, if the user identifiers of the advertising traffic samples of two nodes are the same, then an edge between the two nodes is established and the association weight between the two nodes is set to 1; if the user identifiers of the advertising traffic samples of the two nodes are different, then no edge between the two nodes is established and the association weight between the two nodes is set to 0; construct the semi-supervised network model based on the graph convolutional neural network GCN and the class prototype vector.

[0097] Optionally, the pre-training unit is also used to construct the semi-supervised network model including the following network: an input layer, the input of the input layer includes a data set of data features of the advertising traffic sample; a graph convolution layer, the graph convolution layer is used to convolve the data set to obtain a pseudo-label probability matrix; a feature mapping layer, the feature mapping layer is used to map data features to aggregate the projection vectors of data features of the same category and alienate the projection vectors of data features of different categories; a class contribution layer, the class contribution layer is used to determine the contribution of the sample to the constructed class prototype vector, wherein the closer the projection vector of the sample in the feature space is to the class prototype vector, the greater the contribution; a prototype vector layer, the prototype vector layer is used to determine the class prototype vector; a classifier layer, the classifier layer is used to determine the category to which the sample belongs and the probability of belonging to the category.

[0098] Optionally, the advertisement traffic samples include marked traffic samples and unmarked traffic samples.

[0099] Optionally, the preprocessing unit is also used to: perform data alignment on the original advertising traffic; perform data type conversion on the original advertising traffic; perform data encoding on the original advertising traffic; perform missing value filling on the original advertising traffic; perform deduplication processing on the original advertising traffic; perform data bucketing processing on the original advertising traffic; and perform feature standardization processing on the original advertising traffic.

[0100] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. Figure 1 The hardware environment shown can be implemented through software or hardware, wherein the hardware environment includes a network environment.

[0101] According to another aspect of the embodiments of the present application, a server or terminal for implementing the above-mentioned abnormal traffic identification method is also provided.

[0102] Figure 5is a structural block diagram of a terminal according to an embodiment of the present application, such as Figure 5 As shown, the terminal may include: one or more (only one is shown in the figure) processors 501, a memory 503, and a transmission device 505, as shown in FIG. Figure 5 As shown, the terminal may further include input and output devices 507 .

[0103] Among them, the memory 503 can be used to store software programs and modules, such as the program instructions / modules corresponding to the abnormal traffic identification method and device in the embodiment of the present application. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 503, that is, realizing the above-mentioned abnormal traffic identification method. The memory 503 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 503 may further include a memory remotely arranged relative to the processor 501, and these remote memories can be connected to the terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, corporate intranet, local area network, mobile communication network and combinations thereof.

[0104] The above-mentioned transmission device 505 is used to receive or send data via a network, and can also be used for data transmission between a processor and a memory. Specific examples of the above-mentioned network may include wired networks and wireless networks. In one embodiment, the transmission device 505 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one embodiment, the transmission device 505 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0105] Specifically, the memory 503 is used to store application programs.

[0106] The processor 501 may call the application stored in the memory 503 through the transmission device 505 to perform the following steps:

[0107] Obtain target advertising traffic, wherein the target advertising traffic is advertising traffic to be identified as abnormal; preprocess the target advertising traffic to obtain data to be identified, wherein the preprocessing is used to process the target advertising traffic into the data to be identified that meets preset requirements; use a feature extraction model to extract features from the data to be identified to obtain data features of the data to be identified; call an abnormal traffic detection model, and use the data features of the data to be identified to determine whether the target advertising traffic is abnormal traffic.

[0108] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0109] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only, and the terminal may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 5 It does not limit the structure of the above electronic device. For example, the terminal may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 5 Different configurations shown.

[0110] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0111] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to execute the program code of the abnormal traffic identification method.

[0112] Optionally, in this embodiment, the above-mentioned storage medium may be located on at least one network device among the multiple network devices in the network shown in the above-mentioned embodiment.

[0113] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps:

[0114] Obtain target advertising traffic, wherein the target advertising traffic is advertising traffic to be identified as abnormal; preprocess the target advertising traffic to obtain data to be identified, wherein the preprocessing is used to process the target advertising traffic into the data to be identified that meets preset requirements; use a feature extraction model to extract features from the data to be identified to obtain data features of the data to be identified; call an abnormal traffic detection model, and use the data features of the data to be identified to determine whether the target advertising traffic is abnormal traffic.

[0115] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0116] Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various storage media that can store program codes.

[0117] The serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0118] The integrated units in the above embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the above computer-readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part or all or part of the technical solutions, can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0119] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0120] In the several embodiments provided by the present application, it should be understood that the disclosed client can be implemented in other ways. Of course, the above device embodiment is only illustrative, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0121] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0122] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0123] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying abnormal traffic, characterized in that: include: Obtaining target advertising traffic, wherein the target advertising traffic is advertising traffic with anomalies to be identified; Preprocessing the target advertising traffic to obtain data to be identified, wherein the preprocessing is used to process the target advertising traffic into the data to be identified that meets preset requirements; Performing feature extraction on the data to be identified using a feature extraction model to obtain data features of the data to be identified, wherein the feature extraction includes performing statistical calculations on the data to be identified, performing feature mapping based on a deep learning model, and extracting a general abstract feature dataset based on an autoencoder for unsupervised learning; Calling an abnormal traffic detection model, and determining whether the target advertising traffic is abnormal traffic using the data features of the data to be identified, including: obtaining the abnormal traffic detection model, wherein the abnormal traffic detection model is trained based on a semi-supervised network model of a graph convolutional neural network and a class prototype vector; calling the abnormal traffic detection model, and identifying the data features of the data to be identified using the saved feature mapping layer parameters and the class prototype vector to determine whether the target advertising traffic is abnormal traffic, wherein the abnormal traffic detection model is trained using a semi-supervised learning method; Among them, constructing a semi-supervised network model based on a graph convolutional neural network and a class prototype vector includes constructing a graph convolutional neural network (GCN): taking each advertising traffic sample as a node in the graph convolutional neural network (GCN), if the user identifiers of the advertising traffic samples of two nodes are the same, then establishing an edge between the two nodes and setting the association weight between the two nodes to 1; if the user identifiers of the advertising traffic samples of the two nodes are different, then not establishing an edge between the two nodes and setting the association weight between the two nodes to 0; The semi-supervised network model includes: an input layer, the input of which includes a data set of data features of the advertising traffic sample; a graph convolution layer, the graph convolution layer is used to convolve the data set to obtain a pseudo-label probability matrix; a feature mapping layer, the feature mapping layer is used to map data features to aggregate the projection vectors of data features of the same category and alienate the projection vectors of data features of different categories; a class contribution layer, the class contribution layer is used to determine the contribution of the sample to the constructed class prototype vector, wherein the closer the projection vector of the sample in the feature space is to the class prototype vector, the greater the contribution; a prototype vector layer, the prototype vector layer is used to determine the class prototype vector; a classifier layer, the classifier layer is used to determine the category to which the sample belongs and the probability of belonging to the category.

2. The method according to claim 1, characterized in that Obtain anomaly traffic detection models, including: Obtaining original advertising traffic, wherein the original advertising traffic is advertising traffic used for training; Preprocessing the original advertisement traffic to convert the original advertisement traffic into an advertisement traffic sample that meets preset requirements; Performing feature extraction and feature combination statistics on the advertising traffic sample to obtain the feature extraction model; A semi-supervised network model based on a graph convolutional neural network and a class prototype vector is constructed, and the data features of the advertising traffic sample are used to train the semi-supervised network model to obtain the abnormal traffic detection model.

3. The method according to claim 1, characterized in that The advertisement traffic samples include marked traffic samples and unmarked traffic samples.

4. The method according to claim 2, characterized in that Preprocessing the original advertisement traffic includes at least one of the following: Performing data alignment on the original advertising traffic; Performing data type conversion on the original advertisement traffic; performing data encoding on the original advertisement traffic; Filling missing values ​​in the original advertising traffic; Performing deduplication processing on the original advertising traffic; Performing data bucketing processing on the original advertising traffic; Perform feature normalization processing on the original advertisement traffic.

5. An abnormal flow identification device, characterized in that: include: An acquiring unit, configured to acquire target advertising traffic, wherein the target advertising traffic is advertising traffic to be identified as abnormal; a preprocessing unit, configured to preprocess the target advertising traffic to obtain data to be identified, wherein the preprocessing is configured to process the target advertising traffic into the data to be identified that meets preset requirements; a feature extraction unit, configured to extract features from the data to be identified using a feature extraction model to obtain data features of the data to be identified, wherein the feature extraction includes performing statistical calculations on the data to be identified, performing feature mapping based on a deep learning model, and extracting a general abstract feature dataset based on an autoencoder for unsupervised learning; an identification unit, configured to call an abnormal traffic detection model and determine whether the target advertising traffic is abnormal traffic using data features of the data to be identified, comprising: obtaining the abnormal traffic detection model, wherein the abnormal traffic detection model is trained using a semi-supervised network model based on a graph convolutional neural network and a class prototype vector; calling the abnormal traffic detection model and identifying the data features of the data to be identified using the saved feature mapping layer parameters and the class prototype vector to determine whether the target advertising traffic is abnormal traffic, wherein the abnormal traffic detection model is trained using a semi-supervised learning method; Among them, constructing a semi-supervised network model based on a graph convolutional neural network and a class prototype vector includes constructing a graph convolutional neural network (GCN): taking each advertising traffic sample as a node in the graph convolutional neural network (GCN), if the user identifiers of the advertising traffic samples of two nodes are the same, then establishing an edge between the two nodes and setting the association weight between the two nodes to 1; if the user identifiers of the advertising traffic samples of the two nodes are different, then not establishing an edge between the two nodes and setting the association weight between the two nodes to 0; The semi-supervised network model includes: an input layer, the input of which includes a data set of data features of the advertising traffic sample; a graph convolution layer, the graph convolution layer is used to convolve the data set to obtain a pseudo-label probability matrix; a feature mapping layer, the feature mapping layer is used to map data features to aggregate the projection vectors of data features of the same category and alienate the projection vectors of data features of different categories; a class contribution layer, the class contribution layer is used to determine the contribution of the sample to the constructed class prototype vector, wherein the closer the projection vector of the sample in the feature space is to the class prototype vector, the greater the contribution; a prototype vector layer, the prototype vector layer is used to determine the class prototype vector; a classifier layer, the classifier layer is used to determine the category to which the sample belongs and the probability of belonging to the category.

6. A storage medium, characterized in that The storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 4 when executed.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor executes the method according to any one of claims 1 to 4 through the computer program.

Images