A Supplier Fraud Detection Method Based on Encoding / Decoding Algorithms

Abstract

This invention discloses a supplier cheating detection method based on encoding / decoding algorithms, comprising: collecting node data information; calculating correlation using mutual information, and selecting combinations based on correlation thresholds to obtain combined data features; constructing a graph structure, training a bandwidth prediction model using GCN on the graph structure, optimizing the model using the mean square error between the model output and the actual bandwidth traffic as a first loss function, and obtaining a graph embedding representation based on the optimized bandwidth prediction model; concatenating the graph embedding representation, data information, and combined data features to obtain abnormal prediction demand data; dividing the dataset based on business and corresponding loss rate thresholds, updating the overall loss function of the abnormal detection algorithm using the first loss function to obtain an abnormal detection model; using the abnormal detection model for detection, manually confirming the detection results for false positives, and processing suppliers or optimizing the abnormal detection model based on the false positive results. This invention can cope with diverse cheating behaviors and achieve efficient cheating detection.

Description

Technical Field

[0001] This invention belongs to the field of edge cloud technology, specifically relating to a supplier cheating detection method based on encoding and decoding algorithms. Background Technology

[0002] The edge cloud resource platform primarily functions as follows: 1. Managing edge resource pools to provide customers with stable computing and storage resources, improving performance and availability through load allocation; 2. Providing edge cloud-based data storage and processing services, reducing latency and ensuring data security; 3. Providing data security safeguards, such as authentication, access control, and encryption, ensuring data confidentiality, integrity, and availability, and preventing malicious code threats; 4. Providing customer support services, including technical support, training, and consulting, ensuring customers fully utilize resources and services. Based on these functions, the edge cloud resource platform can meet the diverse needs of customers in edge computing scenarios.

[0003] Edge computing technology brings data processing closer to users and terminal devices, accelerating decision-making, reducing transmission costs, and enhancing data privacy and security. With the generation of massive amounts of data on the internet, edge computing technology is becoming increasingly prevalent. Applications such as autonomous driving, cloud gaming, and AIGC (AI Generated Content) have different requirements for computing resources. For example, autonomous driving requires real-time response, cloud gaming demands high-performance graphics processing, and AIGC relies on massively parallel computing. Edge cloud resources can be flexibly scheduled according to application needs, improving user experience and optimizing resource utilization. However, edge devices have limited resources; to meet computing demands, edge cloud resources have emerged as resource pools. Edge cloud, through multi-device sharing, can provide greater computing power and storage capacity to meet the data processing needs of these applications on edge devices.

[0004] Providers offering services to edge cloud platforms may resort to the following methods to reduce bandwidth provision and gain revenue exceeding actual bandwidth: 1. Degrading network quality: Providers can reduce bandwidth provision by degrading network quality, such as limiting network bandwidth, reducing network speed, or increasing network latency; 2. Simulating data traffic: Providers can deceive the platform by simulating data traffic, causing it to misjudge bandwidth usage and thus reduce bandwidth provision. For example, automated scripts can be used to simulate data traffic, making it appear as real user traffic when it is actually just simulated data. Resource providers achieve the goal of inflating bandwidth and performance through fake data, thereby inducing customers to purchase their resources and gain more revenue. However, this leads to customers' tasks not being processed effectively, which not only affects customer experience but also damages the reputation of the entire edge cloud resource market; 3. Exploiting technical vulnerabilities: Providers can exploit technical vulnerabilities to bypass platform monitoring and detection, thereby achieving bandwidth below the reported level. For example, proxy servers can be used to hide the real network bandwidth, or hacking techniques can be used to attack the platform's monitoring system, making it unable to accurately detect bandwidth usage. In summary, cheating can lead to resource abuse, affect service quality, cause customer data leakage, and create system security risks. Supplier fraud undermines market fairness and transparency, reduces customer trust and satisfaction, and impacts platform business development and competitiveness. Therefore, edge cloud platforms should implement strict regulatory and control measures to prevent supplier fraud.

[0005] Currently, supplier fraud detection mainly faces the following problems: 1. The accuracy of existing detection algorithms is not high. Most current fraud detection algorithms are based on simple threshold detection or rule detection, which are easily affected by external interference, leading to false positives or false negatives; 2. Data acquisition is difficult. Suppliers may deliberately conceal data and are unwilling to provide complete resource data. Platform stress testing may affect customer service quality and billing, thus data acquisition is difficult, limiting the feasibility and effectiveness of detection algorithms; 3. Weak attack and defense capabilities. Existing detection algorithms are relatively weak in dealing with malicious behavior by suppliers. Suppliers may circumvent the system by modifying data or submitting different data at different times. Detection: To improve offensive and defensive capabilities, existing detection algorithms need further optimization, combining real-time monitoring and historical data analysis to improve the accuracy of identifying abnormal behavior and effectively prevent potential risks; 4. Insufficient generalization ability: Existing detection algorithms may perform well in dealing with specific types of cheating behavior, but when encountering new or unknown cheating behaviors, their generalization ability is insufficient, making it difficult to effectively identify and handle them; therefore, it is necessary to research detection algorithms with stronger generalization ability to better adapt to the ever-changing supplier cheating methods and scenarios; 5. Insufficient intelligence: Existing detection algorithms largely rely on manually designed features and rules, which limits the detection effect to human experience and expertise. When facing complex and ever-changing supplier cheating behaviors, over-reliance on manually designed features may not be able to fully capture potential cheating behaviors. Therefore, it is necessary to develop more intelligent detection algorithms that can automatically learn and mine hidden patterns and information in data, thereby improving detection effectiveness and adaptability. Summary of the Invention

[0006] To address the above problems, this invention proposes a supplier cheating detection method based on encoding and decoding algorithms. This method effectively handles the diverse resource nodes, dynamically changing service demands, and data characteristics in edge cloud scenarios, solving the problems existing in current supplier cheating detection. It provides an efficient, accurate, and intelligent cheating detection solution, achieving fair and healthy development of the edge cloud resource market. The technical solution adopted by this invention to solve the above technical problems is as follows:

[0007] A supplier cheating detection method based on encoding / decoding algorithms includes the following steps:

[0008] S1, data collection In the time period Data information for each edge cloud node at each time point, the data information including node data features and business data features, and all data features are vectorized;

[0009] S2, use mutual information to calculate the correlation between the data features in each data information and the cheating status of the data feature, filter the data features in each data information based on the correlation threshold and combine them in pairs to obtain combined data features, and construct bandwidth prediction demand information based on the combined data features and the data information in step S1.

[0010] S3. Construct a graph structure based on the bandwidth prediction demand information, train the graph structure using GCN to obtain a bandwidth prediction model, and optimize the bandwidth prediction model using the output of the bandwidth prediction model and the mean square error of the actual bandwidth traffic as the first loss function. Based on the optimized bandwidth prediction model, obtain the graph embedding representation of each edge cloud node.

[0011] S4, the graph embedding representation in step S3 and the bandwidth prediction demand information in step S2 are concatenated to obtain the abnormal prediction demand data for each edge cloud node.

[0012] S5. Based on the business and the corresponding loss rate threshold, the abnormal prediction demand data in step S4 is divided into a normal set and an abnormal set. The first loss function in step S3 is used to update the overall loss function of the abnormal detection algorithm model. The abnormal detection algorithm after updating the overall loss function of the model is trained to obtain the abnormal detection model.

[0013] S6. Collect the latest data information of edge cloud nodes and use the anomaly detection model in step S5 to detect it. Manually confirm the detection results for misjudgment. If there is no misjudgment, the supplier is dealt with; otherwise, the normal set is updated and the anomaly detection model is optimized based on the misjudged data.

[0014] The node data features include machine ID, supplier, billing type, node reserve bandwidth, single-line bandwidth, number of lines, TCP packet loss satisfaction, average test bandwidth, TCP packet loss satisfaction, extreme stress test satisfaction, bandwidth utilization, network latency, online rate, number of online lines, and node actual 95% bandwidth.

[0015] The business data characteristics include task ID, TCP retransmission rate, single-line bandwidth, network type, disk bandwidth ratio, and average utilization rate.

[0016] Step S3 includes the following steps:

[0017] S3.1 Calculate connectivity indicators based on the information transmission status between edge cloud nodes, and construct a graph structure based on connectivity indicators and bandwidth prediction demand information in step S2;

[0018] S3.2, Use graph convolutional neural networks to build and train network models;

[0019] S3.3, the bandwidth prediction model is obtained by training the graph structure using the training network model, and the bandwidth prediction model is optimized by the Adam optimizer using the output of the bandwidth prediction model and the mean square error of the actual bandwidth traffic as the first loss function.

[0020] S3.4, use the optimized bandwidth prediction model to obtain the graph embedding representation of each edge cloud node.

[0021] The graph structure is represented by G(V, E), where V represents a vertex (i.e., an edge cloud node), E represents an edge, and e represents a vertex (i.e., an edge cloud node). ij ∈E, e ij Represents edge cloud node v i With edge cloud nodes v j The connectivity index between them is calculated using the following formula:

[0022] e ij =delay ij *w1+TCPReTrans ij *w2+distance ij *w3+ISP ij *w4;

[0023] In the formula, delay ij Represents edge cloud node v i With edge cloud nodes v j Network latency between TCPReTrans ij Represents edge cloud node v i With edge cloud nodes v j TCP retransmission rate between distances ij Represents edge cloud node v i With edge cloud nodes v j The distance between them, ISP ij Represents edge cloud node v i With edge cloud nodes v j Whether they belong to the same operator is an indicator variable, where w1, w2, w3, and w4 all represent weights.

[0024] Step S5 includes:

[0025] S5.1, calculate the total 95 bandwidth of each edge cloud node in each time period based on the business, calculate the loss rate of the edge cloud node in each time period based on the business based on the total 95 bandwidth, and divide the abnormal prediction demand data below the loss rate threshold into the normal set, otherwise divide it into the abnormal set;

[0026] The formula for calculating the loss rate is as follows:

[0027]

[0028] In the formula, This represents the loss rate based on service c on edge cloud node i during the m-th time period. This represents the total bandwidth of edge cloud node i based on service c in the m-th time period, b. m，i This represents the 95% bandwidth of edge cloud node i in the m-th time period;

[0029] S5.2, Use manual methods to analyze the noise causes of the abnormal set in step S5.1 and filter out non-cheating abnormal data volume situations to add to the normal set.

[0030] S5.3, based on the updated normal set and abnormal set, use the Anomaly Transformer model after updating the overall loss function of the model based on the first loss function to train and obtain the anomaly detection model.

[0031] Step S6 includes the following steps:

[0032] S6.1 Input the latest data information of the edge cloud nodes into the anomaly detection model for detection and output the corresponding anomaly value;

[0033] S6.2, based on business needs, group the data information of all edge cloud nodes, and sort the grouped edge cloud node data information in descending order based on the mean of outliers, then... The supplier corresponding to each piece of data is sent to the edge cloud operation platform for processing as a supplier at risk of cheating.

[0034] S6.3, The edge cloud operation platform manually confirms the suppliers with cheating risks. If cheating is confirmed, the supplier is processed according to the preset rules. Otherwise, the suppliers with cheating risks are marked as misjudged, and the marked information is sent to the anomaly detection model and step S6.4 is executed.

[0035] S6.4 The anomaly detection model adds the corresponding misjudged samples to the normal set for training to optimize the anomaly detection model.

[0036] The beneficial effects of this invention are:

[0037] 1. GCN is used to process edge cloud node information, and different resource nodes are identified through node embedding. Specifically, GCN is used to extract and represent node features, and the node representation is passed as input to the codec for anomaly detection. In this way, different resource nodes can still be accurately identified without relying on specific resource node IDs, thereby preventing suppliers from circumventing detection by modifying data or submitting different data at different times. This improves the accuracy of cheat detection and enhances both attack and defense capabilities.

[0038] 2. By combining manual annotation and automatic detection, the training dataset is dynamically updated to ensure the quality and diversity of the training data. By continuously adding non-cheating samples, the model can better learn the characteristics of normal behavior, reduce its sensitivity to abnormal data, and improve detection accuracy. Using an adaptive learning strategy, the model can self-adjust under new data and scenarios, achieving dynamic adaptation and continuous optimization, thereby improving the model's generalization ability and practicality. A quality feedback mechanism is introduced to optimize and adjust the training data based on the model's performance in real-world applications, ensuring that the model can continuously improve its performance during iterative processes.

[0039] 3. The anomaly detection algorithm, which combines the Anomaly Transiormer algorithm and the correlation difference mechanism, improves the processing capability of complex time-series data. At the same time, the strategy of continuous iteration of human and data improves the quality of training dataset and model performance. It can effectively adapt to different edge cloud resource scenarios, better deal with diverse supplier cheating behaviors, achieve efficient cheating detection, help improve the accuracy of anomaly detection, reduce the risk of false positives and false negatives, and ensure fair market competition. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the process of the present invention.

[0042] Figure 2 This is a schematic diagram of the structure for training a network model.

[0043] Figure 3 This is a comparison chart of the false positive rates of this application and other algorithms.

[0044] Figure 4 This is a diagram illustrating bandwidth traffic based on business requirements.

[0045] Figure 5 This is a diagram illustrating bandwidth and traffic under network sharing conditions.

[0046] Figure 6 This is a diagram illustrating bandwidth and traffic under speed-limited conditions.

[0047] Figure 7 This is a diagram illustrating bandwidth and traffic during network anomalies.

[0048] Figure 8 This is a schematic diagram illustrating the change process of the learning rate according to the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Supplier cheating in edge cloud scenarios refers to suppliers gaining unfair competitive advantages through various means that violate fair competition principles when providing cloud computing resources. Network sharing refers to suppliers sharing network connections with multiple customers when providing cloud computing resources such as virtual machines, reducing network bandwidth quality and stability, and harming the interests of other customers. Bandwidth misrepresentation refers to suppliers exaggerating bandwidth capacity or performance indicators when providing cloud computing resources, deceiving customers or platform administrators, and gaining illicit profits. Rate limiting refers to suppliers limiting the rate of certain customers or tasks to prioritize the needs of other customers or tasks, or deliberately restricting the use of resources by customers or tasks to obtain more profit. These behaviors seriously affect fair competition in the edge cloud computing resource market and harm the rights and interests of other customers or platform administrators; therefore, effective technical means are needed to prevent and detect supplier cheating.

[0051] The paper "Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy" (ICLR 2022 Spotlight) proposes a Transformer-based time series anomaly detection algorithm. This algorithm utilizes the Transformer's self-attention mechanism to capture complex patterns in time series data. Simultaneously, it introduces an association discrepancy metric to assess the degree of anomalousness in dependencies between elements in the sequence. Through this combination, AnomalyTransformer can effectively identify and locate outliers in time series data. This method is widely applicable to anomaly detection tasks in various fields, such as finance, industry, and healthcare, to improve the accuracy and efficiency of anomaly identification.

[0052] Graph Convolutional Networks (GCNs) originated from Kipf and Welling's paper "Semi-Supervised Classification with Graph Convolutional Networks" (2017). GCNs utilize the adjacency matrix and node features of a graph structure to perform convolutional operations, extracting information between nodes. Applications include node classification, link prediction, and graph generation, and they are widely used in social network analysis, bioinformatics, and recommender systems.

[0053] Anomaly detection algorithms are methods for identifying data points that significantly differ from normal data patterns. Common methods include statistical methods, machine learning (such as clustering and classification), and deep learning (such as autoencoders, LSTM, and Transformers). Application areas include financial fraud, network intrusion, and fault detection, aiming to discover potential problems and improve system security and stability.

[0054] 95 PM Billing: Node Revenue = 95 PM Billing Points * Node Price P * Daily Effective Factor. A 95 PM billing point can be selected every 5 minutes, and its bandwidth usage is recorded. Therefore, there are a total of 12 * 24 = 288 points per day. Among them, the 36 points between 20:00 and 23:00 are sorted in descending order, and the first 14 (5% * 288, rounded) points are removed. The remaining point with the highest ranking is taken as the 95 PM billing point, and its bandwidth usage (in Mbps) is the billing bandwidth. Daily Effective Factor: Points with data (points online at the time of statistics) / All points on the billing day (288). Node Price: Determined based on the node's pricing model.

[0055] A supplier cheating detection method based on encoding / decoding algorithms, such as Figure 1 As shown, it includes the following steps:

[0056] S1, data collection In the time period Data information for each edge cloud node at each time point, wherein the data information includes node data characteristics and business data characteristics, including the following steps:

[0057] S1.1 Collect node data characteristics and business data characteristics of the services running on each edge cloud node according to the preset time period and preset time node, and construct several data information of each edge cloud node at different times based on the data characteristics;

[0058] The node data features include basic node features and statistical node features. Basic node features include machine ID, provider, billing type, node reserve bandwidth, single-line bandwidth, number of lines, TCP packet loss satisfaction, average test bandwidth, TCP packet loss satisfaction, extreme stress test satisfaction, and bandwidth utilization. It may also include province, city, dial-up type, IP address, number of CPU cores, memory, disk type and size, number of lines, recruitment type, process status, carrier, IOPS, and single-line IOPS. Statistical node features include network latency, online rate, number of online lines, and node's actual 95% bandwidth. It may also include bandwidth utilization and reserve bandwidth fluctuation. Furthermore, the node data features may include node monitoring features, such as disk latency, number of disconnected lines, number of IP change errors, and number of offline events.

[0059] The business data features include business statistical features and business basic features. Business basic features include task ID, TCP retransmission rate, single-line bandwidth, network type, disk bandwidth ratio, and may also include task name, bandwidth requirement, number of CPUs per line, memory size per line, number of SSDs per line, size of SSDs per line, and size of disk per line. Business statistical features include average utilization rate, and may also include sum of actual 95% bandwidth, bandwidth summation, total number of nodes, and average upload 95% bandwidth.

[0060] During the specific collection process, It can be set to 30 or 20 days, with data collected every five or ten minutes each day, for example... It was set to collect data every five minutes, that is, to collect data every day. By analyzing the data features at each time point, 8640 data entries corresponding to different times for each edge cloud node can be obtained.

[0061] S1.2, Perform abnormal data cleaning on all data features in each data information in step S1.1;

[0062] The abnormal data refers to hardware failure data, abnormal supplier login / logouts, periods of low bandwidth usage, missing feature field values, and abnormal feature values. In detecting supplier bandwidth fraud, specialized cleaning (mean supplementation or deletion) of bandwidth-related data can eliminate abnormal fluctuations in bandwidth usage. For example, data showing no bandwidth usage on nodes before a task is activated should be deleted.

[0063] S1.3, Perform feature transformation on the data features in each cleaned data information according to the data type to vectorize the data features;

[0064] The data types include numerical and non-numerical types, i.e., types. Type features, such as cities and operators, can be achieved using one-hot encoding, multi-hot encoding, vectorization, and other techniques. Numerical features, such as node bandwidth and single-line bandwidth, can be achieved using normalization, bucketing, one-hot encoding, or vectorization.

[0065] S2, using mutual information to calculate the correlation between the data features in each data information and the cheating status of that data feature, and then filtering the data features in each data information based on the correlation threshold and combining them in pairs to construct bandwidth prediction demand information;

[0066] S2.1, Manually label each data feature in each data message with cheating status based on 95 bandwidth traffic within a preset time period;

[0067] The cheating status refers to whether the edge cloud node supplier has cheated in terms of bandwidth provision. For example, after statistically analyzing 95% of the bandwidth traffic within a preset time period, the bandwidth measured by a bandwidth stress testing tool is compared with the statistically analyzed bandwidth. If the stress test bandwidth is much smaller than the statistically analyzed bandwidth, for example, if the stress test bandwidth is less than 80% of the bandwidth reported by the supplier, then the supplier is considered to have cheated within that preset time period.

[0068] S2.2 Calculate the correlation between each data feature in each data information and the cheating status using mutual information, and filter out data features with a correlation greater than the correlation threshold to combine them in pairs to obtain combined data features;

[0069] When calculating mutual information, continuous features (i.e., numerical features) are first binned, while discrete features (i.e., categorical features) are not binned. Then, the mutual information between feature X and the cheating target Y is calculated, with each time point representing one data point (X, Y). The formula for calculating mutual information is:

[0070] I(X,Y)=∑∑P(X,Y)log(P(X,Y) / (P(X)P(Y)));

[0071] The data features selected in this application include supplier, node reserve bandwidth, single-line bandwidth, average test bandwidth, extreme stress test satisfaction, TCP packet loss satisfaction, node actual 95% bandwidth, network latency, and online rate. When combining consecutive features in pairs, they need to be bucketed; for example, supplier and node reserve bandwidth can be combined into a new feature.

[0072] S2.3, combine the data features in each data information obtained in step S1 with the corresponding combined data features in step S2.2 to obtain bandwidth prediction demand information;

[0073] The bandwidth prediction demand information uses f i It means that f i =Concat(f1, f2, ... f m ), f i Let m represent the node features of the i-th edge cloud node, and m represent the total number of features of the edge cloud nodes.

[0074] S3. Construct a graph structure based on bandwidth prediction demand information, train the graph structure using GCN to obtain a bandwidth prediction model, and optimize the bandwidth prediction model using the output of the bandwidth prediction model and the mean square error of the actual bandwidth traffic as the first loss function. Based on the optimized bandwidth prediction model, obtain the graph embedding representation of each edge cloud node, including the following steps:

[0075] S3.1 Calculate connectivity indicators based on the information transmission status between edge cloud nodes, and construct a graph structure based on connectivity indicators and bandwidth prediction demand information in step S2;

[0076] The graph structure is represented by G(V, E), where V represents a vertex, i.e., an edge cloud node, and V = {v1, v2, ... v}. n ), v n This represents the nth edge cloud node, where n represents the number of edge cloud nodes, E represents the edge, and e ij ∈E, e ij Represents edge cloud node v i With edge cloud nodes v j The connectivity index between them is obtained through weighted calculation, and the corresponding calculation formula is:

[0077] e ij =delay ij *w1+TCPReTrans ij *w2+distance ij *w3+ISP ij *w4;

[0078] In the formula, e ij Represents edge cloud node v iWith edge cloud nodes v j The connectivity indicator between them, delay ij Represents edge cloud node v i With edge cloud nodes v j Network latency between TCPReTrans ij Represents edge cloud node v i With edge cloud nodes v j TCP retransmission rate between distances ij Represents edge cloud node v i With edge cloud nodes v j The distance between them, ISP ij Represents edge cloud node v i With edge cloud nodes v j Whether they are from the same operator is an indicator variable, when edge cloud node v i With edge cloud nodes v j The value is 0 if they are from the same operator, otherwise it is 1. w1, w2, w3, and w4 all represent weights. Edge cloud node v i With edge cloud nodes v j Network latency, TCP retransmission rate, and distance between them are obtained through pre-collected data.

[0079] S3.2, Use GCN (Graph Convolutional Network) to build and train the network model;

[0080] like Figure 2 As shown, the trained network model includes an input layer, a first graph convolutional layer, a first activation function layer, a second graph convolutional layer, a second activation function layer, a pooling layer, and an output layer connected in sequence. The graph convolutional layer is used for local convolutional operations to capture dependencies between nodes. After the graph convolutional layer, an appropriate activation function, such as ReLU or tanh, is added to increase the model's non-linear expressive power. The pooling layer can reduce feature dimensionality and computational complexity. The input of the pooling layer serves as a node embedding representation for subsequent cheating detection tasks. The output layer outputs the predicted bandwidth flow of the nodes at each time point, used to calculate the auxiliary first loss function. This is prior art and will not be described in detail here.

[0081] S3.3, the bandwidth prediction model is obtained by training the graph structure using the training network model, and the bandwidth prediction model is optimized by the Adam optimizer using the output of the bandwidth prediction model and the mean square error of the actual bandwidth traffic as the first loss function.

[0082] The formula for calculating the first loss function is:

[0083]

[0084] In the formula, U represents actual bandwidth traffic. t This represents the bandwidth prediction value output by the bandwidth prediction model. Specifically, it can be set to predict and measure the actual bandwidth every five minutes to optimize the model. The bandwidth traffic mentioned in this application refers to 95 bandwidth traffic.

[0085] S3.4, Use the optimized bandwidth prediction model to obtain the graph embedding representation of each edge cloud node;

[0086] The bandwidth prediction demand information is re-input into the optimized bandwidth prediction model, and the output of the second graph convolutional layer is extracted as the graph embedding representation of the edge cloud nodes. i , and e i ∈E d E d Let E represent a d-dimensional vector space.

[0087] F: f→E d ;

[0088] In the formula, F represents the graph embedding mapping function, and f represents the bandwidth prediction demand information.

[0089] This application introduces a GCN network for the first time into the problem of supplier cheating detection in edge cloud nodes. Compared with existing detection algorithms that only use information from a single node, the GCN neural network can comprehensively consider neighboring network nodes across connectivity metrics, detecting network anomalies caused by severe fluctuations in regional traffic, thus reducing noise input to anomaly detection algorithms. To be applicable to edge cloud nodes, this application redefines the connectivity metrics applicable to node distance, more reasonably defining proximity in resource scheduling. Figure 3 As shown in the experimental metrics, after using GCN node embedding as input, the misclassification rate of anomaly detection (misclassification rate = number of misclassified samples / total number of samples) decreased from 0.0461 to 0.0347.

[0090] S4, embed the graph representation e from step S3. i And the bandwidth prediction demand information f in step S2 i The corresponding data are stitched together to obtain the anomaly prediction requirements for each edge cloud node;

[0091] x i =Concat(f i e i )

[0092] In the formula, x i This represents the anomaly prediction requirement data for the i-th edge cloud node, and Concat(·) represents the concatenation function.

[0093] S5. Based on the business and the corresponding loss rate threshold, the abnormal prediction demand data in step S4 is divided into a normal set and an abnormal set. The overall loss function of the anomaly detection algorithm model is updated using the first loss function. The anomaly detection algorithm (Anomaly-transformer) after updating the overall loss function is then trained to obtain the anomaly detection model, including the following steps:

[0094] S5.1, calculate the total 95 bandwidth of each edge cloud node in each time period based on the business, calculate the loss rate of the edge cloud node in each time period based on the business based on the total 95 bandwidth, and divide the abnormal prediction demand data below the loss rate threshold into the normal set, otherwise divide it into the abnormal set;

[0095] This application assumes that at most one service can be running on a single edge cloud node at each time point, and the same service may occupy different edge cloud nodes at the same time point, without considering mixed operation scenarios. The formula for calculating the attenuation rate is:

[0096]

[0097] In the formula, This represents the loss rate based on service c on edge cloud node i during the m-th time period. This represents the total bandwidth of edge cloud node i based on service c in the m-th time period, b. m，i This represents the 95% bandwidth of edge cloud node i in the m-th time period. This application uses the 3 / 4 quantile of each loss rate as the corresponding loss rate threshold. The 3 / 4 quantile is the threshold for the loss rate of edge cloud node i based on service c in the m-th time period.

[0098] For example, Figure 4 As shown, in the t1-th time period, if service c runs on edge cloud node a and edge cloud node b, b t1 This represents the total bandwidth of edge cloud node b based on service c during time period t1, which is 95%. This represents node b's contribution to service c. t2 Let t1 represent the 95% bandwidth value for edge cloud node b based on all services. Then, the attenuation rate of edge cloud node b is 1-b. t1 / b t2 .

[0099] S5.2, Use manual methods to analyze the noise causes of the abnormal set in step S5.1 and filter out non-cheating abnormal data volume situations to add to the normal set.

[0100] To reduce the impact of noisy data on model performance, this application analyzed the causes of noisy data and found that abnormal data comes from multiple sources, including: vendor cheating issues such as network sharing, false bandwidth reporting, and rate limiting; network quality anomalies such as line outages and abnormal network latency; hardware problems such as equipment hardware failures; scheduling problems caused by client-side scheduling that are beyond the control of the platform and vendor; and program anomalies usually caused by task switching, such as the inability to scale up during peak hours on the same day after a switch. Network quality anomalies, hardware problems, scheduling problems, and program anomalies are not considered cheating.

[0101] like Figure 5 As shown, network sharing refers to a bandwidth resource being shared by multiple nodes, with these nodes competing for traffic during peak evening hours. Figure 6 As shown, rate limiting refers to a rate limiting strategy specified by the supplier, including full limiting, partial limiting, and alternating limiting between shared nodes. For example... Figure 7 As shown, network quality anomalies refer to abnormal service scheduling waveforms caused by network packet loss and latency, resulting in high-frequency fluctuations in the service curve.

[0102] S5.3, Based on the updated normal set and abnormal set, the anomaly detection model is trained using the Anomaly Transformer model to obtain the anomaly detection model;

[0103] The Anomaly Transformer algorithm employs a novel correlation difference mechanism to measure the correlation differences between anomalous and normal time-series data. Unlike general encoding / decoding anomaly detection algorithms, this mechanism effectively distinguishes between anomalous and normal data, improving the accuracy of anomaly detection. Furthermore, the Transformer's self-attention mechanism captures long-distance dependencies in time-series data, enhancing the model's generalization ability and detection performance. This algorithm is highly adaptable, capable of handling diverse time-series data and anomaly types in edge cloud scenarios. This is of significant value for achieving efficient anomaly detection across different edge devices and application scenarios.

[0104] To adapt to the specific task of anomaly detection, this application selects the Anomaly Transformer, which optimizes and improves the Transformer model. It enhances the accuracy of supplier cheating detection through correlation differences. The model structure is as follows: Rescale performs hierarchical normalization of the sequences, ensuring that the sum of prior correlation values ​​equals 1. Lay-Norm and Feed-Forward are components of the Transformer and remain unchanged in this application.

[0105] x 0 =Embedding(x);

[0106] Initialization: Q, K, V,

[0107] Prior association:

[0108] Sequence association:

[0109] reconstruction:

[0110] Residual connection:

[0111] Output:

[0112] In the formula, x 0 This represents the anomaly prediction requirement data after initialization.

[0113] The loss of the Anomaly Transformer consists of two parts: reconstruction loss and correlation difference. Model output. This is the output of the last transformer layer. Traditional models typically include a cheat detection loss function and an auxiliary loss. This application differs from existing technologies by using the first loss function L... aux It is also added to the overall model loss function. The formula for calculating the updated overall model loss function is as follows:

[0114]

[0115] The differences in correlation are as follows:

[0116]

[0117] Compared to directly using the Transformer model for anomaly detection, this application employs the Anomaly-transformer algorithm, which is based on correlation differences. This algorithm leverages the correlation between local and global time-series data, making it suitable for traffic fluctuations caused by heterogeneous network devices in edge cloud scenarios, significantly improving the anti-interference capability of the anomaly detection algorithm. Experiments show that the misclassification rate of the method GCN-Anomaly_Transformer used in this application is reduced from 0.0520 to 0.0347 compared to GCN-Transformer.

[0118] This application first determines the transformer's hyperparameters, such as dimension and number of layers, using a grid search method. The dimension values ​​are [16, 32, 48, 96], and the number of layers are [4, 8, 16, 32]. Gradient clipping is used to limit the maximum gradient to prevent gradient explosion. A custom learning rate scheduling strategy, "warmup + cosine decay," is employed. The learning rate is gradually increased in the early stages to accelerate convergence and gradually decreased in the later stages to stabilize training. Training begins after setting the number of iterations and the maximum learning rate. In deep learning, the learning rate scheduling strategy is crucial for successful training. Warmup + Cosine Decay is a common learning rate scheduling strategy that combines warmup and cosine decay. Warmup: At the beginning of training, the learning rate gradually increases from a low initial value to the predetermined maximum learning rate. This warm-up phase helps prevent gradient explosion or vanishing in the early stages of training, thus improving training stability. Cosine Decay: After the warmup phase, the learning rate gradually decreases according to the cosine decay formula. Cosine decay can help the model converge to a local optimum better in the later stages of training. warmup_steps = 100, max_learning_rate = 1e-3, initial_learning_rate = 1e-5, final_learning_rate = 1e-4, total_steps = 500. Figure 8 It shows the process of the learning rate changing.

[0119] S6. Collect the latest data information of edge cloud nodes and perform detection using the anomaly detection model in step S5. Manually confirm the detection results for false positives, and process or optimize the anomaly detection model for the supplier based on whether a false positive is found. This includes the following steps:

[0120] S6.1 Input the latest data information of the edge cloud nodes into the anomaly detection model for detection and output the corresponding anomaly value;

[0121] The formula for outliers is:

[0122]

[0123] This outlier is a sequence, and this application uses the sequence mean to represent the overall outlier of the sequence.

[0124] S6.2, based on business requirements, group the data information of all edge cloud nodes, and sort the grouped edge cloud node data information in descending order based on the mean of outliers, then... The supplier corresponding to each piece of data is sent to the edge cloud operation platform for processing as a supplier at risk of cheating.

[0125] In this application, the supplier acts as the hardware provider for the edge cloud nodes, and the edge cloud operation platform manages the hardware devices, facilitating the request and utilization of computing resources by terminals such as Douyin and Kuaishou. This application sets... The calculation formula is:

[0126]

[0127] In the formula, Num k This indicates the number of data entries in the k-th group. This indicates the preset risk ratio.

[0128] S6.3, The edge cloud operation platform manually confirms the suppliers with cheating risks. If cheating is confirmed, the supplier is processed according to the preset rules. Otherwise, the suppliers with cheating risks are marked as misjudged, and the marked information is sent to the anomaly detection model and step S6.4 is executed.

[0129] During manual verification, the method for judging cheating status can be followed, or it can be based on experience. When dealing with suppliers, the supplier can be notified to rectify the situation first. If the rectification is not completed within the specified time, it can be determined whether the node is in a resource-scarce area. If not, billing can be switched to 95 PM to optimize the problematic node and improve the platform's service quality. Otherwise, if so, communication should be made to rectify the situation.

[0130] S6.4, the anomaly detection model adds the corresponding misjudged samples to the normal set for training in order to optimize the anomaly detection model;

[0131] For cases where the model misclassifies, the samples are compiled and added to the training set to optimize model performance and make it compatible with more types of non-cheating anomalies. Through repeated iterations, the model is continuously optimized to improve its performance in cheating prediction and enhance the user experience on both the supply and demand sides.

[0132] The purpose and significance of this application lies in addressing the issue of cheating by edge cloud resource providers. It employs an encoder-decoder-based algorithm to predict the probability of cheating by resource nodes, integrating characteristics of resource nodes, business operations, traffic volume curves, network topology, resource usage, data transmission and processing speed, stability, and reliability indicators. An anomaly detection algorithm based on the encoder-decoder is used to monitor and detect edge cloud resources, identifying nodes suspected of cheating and restricting or eliminating them. Simultaneously, through analysis of historical data and iterative processing between humans and algorithms, usage patterns and behavioral norms of nodes are discovered, thereby improving the accuracy and reliability of the detection algorithm and effectively ensuring the fair allocation and efficient utilization of computing resources. By utilizing GCN and Anomaly Transformer models for self-supervised learning training, this method can detect cheating behaviors by providers when offering cloud computing resources, thus preventing providers from falsely reporting resource performance or providing fake resources. Therefore, this application is of great significance for the standardization and fair competition of the computing resource market and can also improve the security and stability of cloud computing resources. In addition, the cheating detection algorithm in this patent does not rely on complete data provided by the supplier. It only needs to obtain a small amount of resource data to perform cheating detection, which greatly simplifies the difficulty of data acquisition.

[0133] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A supplier cheating detection method based on encoding / decoding algorithms, characterized in that, Includes the following steps: S1, data collection In the time period Data information for each edge cloud node at each time point, the data information including node data features and business data features, and all data features are vectorized; S2, use mutual information to calculate the correlation between the data features in each data information and the cheating status of the data feature, filter the data features in each data information based on the correlation threshold and combine them in pairs to obtain combined data features, and construct bandwidth prediction demand information based on the combined data features and the data information in step S1. S3. Construct a graph structure based on the bandwidth prediction demand information, train the graph structure using GCN to obtain a bandwidth prediction model, and optimize the bandwidth prediction model using the output of the bandwidth prediction model and the mean square error of the actual bandwidth traffic as the first loss function. Based on the optimized bandwidth prediction model, obtain the graph embedding representation of each edge cloud node. S4, the graph embedding representation in step S3 and the bandwidth prediction demand information in step S2 are concatenated to obtain the abnormal prediction demand data for each edge cloud node. S5. Based on the business and the corresponding loss rate threshold, the abnormal prediction demand data in step S4 is divided into a normal set and an abnormal set. The first loss function in step S3 is used to update the overall loss function of the abnormal detection algorithm model. The abnormal detection algorithm after updating the overall loss function of the model is trained to obtain the abnormal detection model. S6. Collect the latest data information of edge cloud nodes and use the anomaly detection model in step S5 to detect it. Manually confirm the detection results for misjudgment. If there is no misjudgment, the supplier is dealt with; otherwise, the normal set is updated and the anomaly detection model is optimized based on the misjudged data.

2. The supplier cheating detection method based on encoding / decoding algorithm according to claim 1, characterized in that, The node data features include machine ID, supplier, billing type, node reserve bandwidth, single-line bandwidth, number of lines, TCP packet loss satisfaction, average test bandwidth, TCP packet loss satisfaction, extreme stress test satisfaction, bandwidth utilization, network latency, online rate, number of online lines, and node actual 95% bandwidth. The business data characteristics include task ID, TCP retransmission rate, single-line bandwidth, network type, disk bandwidth ratio, and average utilization rate.

3. The supplier cheating detection method based on encoding / decoding algorithm according to claim 1, characterized in that, Step S3 includes the following steps: S3.1 Calculate connectivity indicators based on the information transmission status between edge cloud nodes, and construct a graph structure based on connectivity indicators and bandwidth prediction demand information in step S2; S3.2, Use graph convolutional neural networks to build and train network models; S3.3, the bandwidth prediction model is obtained by training the graph structure using the training network model, and the bandwidth prediction model is optimized by the Adam optimizer using the output of the bandwidth prediction model and the mean square error of the actual bandwidth traffic as the first loss function. S3.4, use the optimized bandwidth prediction model to obtain the graph embedding representation of each edge cloud node.

4. The supplier cheating detection method based on encoding / decoding algorithm according to claim 3, characterized in that, The graph structure adopts express, This represents a vertex, also known as an edge cloud node. Represents an edge. , Represents edge cloud nodes With edge cloud nodes The connectivity index between them is calculated using the following formula: ； In the formula, Represents edge cloud nodes With edge cloud nodes Network latency between Represents edge cloud nodes With edge cloud nodes TCP retransmission rate between Represents edge cloud nodes With edge cloud nodes The distance between them Represents edge cloud nodes With edge cloud nodes Whether they are from the same operator is an indicator variable. , , , All represent weights.

5. The supplier cheating detection method based on encoding / decoding algorithm according to claim 1, characterized in that, Step S5 includes: S5.1, calculate the total 95 bandwidth of each edge cloud node in each time period based on the business, calculate the loss rate of the edge cloud node in each time period based on the business based on the total 95 bandwidth, and divide the abnormal prediction demand data below the loss rate threshold into the normal set, otherwise divide it into the abnormal set; The formula for calculating the loss rate is as follows: ； In the formula, Indicates the first Edge cloud nodes during the time period Based on business The loss rate, Indicates the first Edge cloud nodes during the time period Based on business Total bandwidth of 95 Indicates the first Edge cloud nodes during the time period 95 bandwidth; S5.2, Use manual methods to analyze the noise causes of the abnormal set in step S5.1 and filter out non-cheating abnormal data volume situations to add to the normal set. S5.3, based on the updated normal set and abnormal set, use the Anomaly Transformer model after updating the overall loss function of the model based on the first loss function to train and obtain the anomaly detection model.

6. The supplier cheating detection method based on encoding / decoding algorithm according to claim 1, characterized in that, Step S6 includes the following steps: S6.1 Input the latest data information of the edge cloud nodes into the anomaly detection model for detection and output the corresponding anomaly value; S6.2, based on business needs, group the data information of all edge cloud nodes, and sort the grouped edge cloud node data information in descending order based on the mean of outliers, then... The supplier corresponding to each piece of data is sent to the edge cloud operation platform for processing as a supplier at risk of cheating. S6.3, The edge cloud operation platform manually confirms the suppliers with cheating risks. If cheating is confirmed, the supplier is processed according to the preset rules. Otherwise, the suppliers with cheating risks are marked as misjudged, and the marked information is sent to the anomaly detection model and step S6.4 is executed. S6.4 The anomaly detection model adds the corresponding misjudged samples to the normal set for training to optimize the anomaly detection model.

Images