Data exception traceability tracking and positioning method, system and device and storage medium

Abstract

The invention discloses a data exception traceability tracking and positioning method, system and device and a storage medium. Relates to the technical field of power system data management. At present, the problems of insufficient data anomaly detection accuracy, low traceability positioning efficiency, incomplete knowledge association and the like in analysis and application links of power marketing data are solved. The method comprises the following steps: carrying out preprocessing and feature extraction on power marketing data, constructing a data association relationship network through a clustering analysis and statistical method, analyzing the difference of data features, analyzing the propagation path of an abnormal point identified in a data stream by adopting a path reasoning technology, and constructing a data abnormity tracing positioning model. A semi-supervised learning method is adopted to improve the accuracy of the model, result optimization and hyper-parameter adjustment are carried out, and multi-condition factors influencing business data information are introduced into the model as hyper-parameters. According to the technical scheme, the anomaly detection and traceability positioning capability of the power marketing data is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power system data governance, and in particular to a method, system, device and storage medium for tracing and locating data anomalies. Background Art

[0002] In recent years, the technologies of knowledge graph and graph neural network have been widely used in semantic association and knowledge mining of data, realizing efficient analysis of complex data relationships. In this context, the analysis and application of marketing data are becoming increasingly important for business decision-making and service optimization. On the one hand, data anomaly tracing and positioning technology has developed rapidly in the field of big data monitoring, and by building data link association models and causal networks, it is easy to accurately identify and analyze data anomalies; on the other hand, the amount of power marketing data is huge and diverse, and with policy changes and market fluctuations, data anomalies are frequent. Traditional data monitoring methods are difficult to identify the causes of anomalies in a timely manner, resulting in a time-consuming and laborious tracing process. On this basis, a knowledge-driven tracing and positioning method is adopted, and the association network of multi-source heterogeneous data is constructed using knowledge graphs. The graph neural network is used to infer the anomaly propagation path, which can effectively improve the anomaly identification and rapid positioning capabilities of power marketing data.

[0003] However, due to the differences in multi-source data, the lack of data link anomaly diagnosis tools, and high data governance costs, the link anomaly diagnosis method for power marketing data faces many challenges in practical applications: (1) Insufficient accuracy of data anomaly detection. Traditional rule detection methods rely on fixed threshold settings and cannot adapt to the complex changes in data characteristics in a timely manner. Especially in a multi-source heterogeneous data environment, the fixed threshold detection method is prone to false positives or false negatives, which makes the anomaly detection process lack flexibility and difficult to cope with dynamically changing marketing data flows. (2) Low efficiency of source tracing and positioning. In existing source tracing analysis, due to the multiple sources and long links of power marketing data, source tracing and positioning after anomalies often requires a lot of manual intervention and time investment. A single data link monitoring method fails to cover the complete business scenario and cannot build an efficient traceability path, resulting in difficulty in quickly locating the root cause of the anomaly. The low traceability efficiency not only affects the business response speed, but also makes it impossible to timely control potential system risks, greatly affecting the stability of power marketing data and business continuity. (3) Incomplete knowledge association and lack of a global perspective. Existing power marketing data analysis is often limited to a single data dimension, lacking the global perspective supported by knowledge graphs, and unable to capture implicit relationships and associated knowledge between multi-source data, which limits the comprehensiveness of abnormal diagnosis and makes it impossible to effectively link abnormal phenomena with the underlying causes. In addition, in data association analysis, the lack of learning ability for historical abnormal data makes it difficult for the system to form cognition of complex abnormal patterns, affecting the adaptability and flexibility of data processing. Summary of the invention

[0004] The technical problem to be solved and the technical task proposed by the present invention are to improve and perfect the existing technical solutions, and provide a data anomaly tracing and locating method, system, device and storage medium, so as to realize knowledge-driven data anomaly tracing and locating, and improve the accuracy and efficiency of anomaly detection. To this end, the present invention adopts the following technical solutions.

[0005] The first technical solution of the present invention is: a knowledge-driven data anomaly tracing and locating method, which includes the following steps: 1) Preprocessing and feature extraction of power marketing data, including data cleaning, deduplication, and standardization operations, as well as extracting the features of power consumption data using knowledge graphs and screening out key features; 2) Through cluster analysis and statistical methods, build a data association network, analyze the differences in data characteristics, distinguish the similarities and differences between different data results or application scenarios, and compare them; 3) Use path reasoning technology and complex network analysis methods to identify and locate the root cause of data anomalies by analyzing the propagation path of abnormal points in the data flow; 4) Build a data anomaly tracing and positioning model, use graph neural network methods for feature learning, and use knowledge graphs to identify the entire link of abnormal data generation, realize the judgment and marking of raw data anomalies, and perform semi-supervised anomaly positioning evaluation; 5) Optimize results and adjust hyperparameters, introduce multiple conditional factors that affect business data information into the model as hyperparameters, adjust the model's hyperparameters, and use the feedback mechanism to dynamically adjust the model's output results.

[0006] This technical solution provides an efficient, flexible and global perspective knowledge-driven data anomaly traceability tracking and positioning method, which solves the problems of low anomaly detection accuracy, low traceability efficiency and incomplete knowledge association in power marketing data, and greatly improves the anomaly detection and traceability positioning capabilities of power marketing data. Specifically: By using the knowledge graph to extract the key features of power marketing data, and combining data cleaning, deduplication and standardization operations, the present invention can effectively identify potential anomalies in the data. Traditional rule detection methods rely on fixed thresholds and often cannot adapt to complex changes in data characteristics. The present invention combines knowledge graphs for feature extraction and screening, allowing the model to dynamically adapt to changes in the data stream, avoiding the false positives and negatives caused by fixed thresholds, thereby improving the accuracy and flexibility of data anomaly detection.

[0007] In the traditional anomaly tracing process, the tracing and positioning after the anomaly occurs often relies on a lot of manual intervention and time investment, and it is difficult to quickly locate the root cause of the anomaly. The present invention constructs a data association network and combines cluster analysis and statistical methods to analyze the differences in data features from a global perspective and construct an efficient data tracing path. By using path reasoning technology and complex network analysis methods, the root cause of data anomalies can be quickly located by identifying the anomaly propagation path, significantly improving the efficiency of tracing and positioning.

[0008] Traditional power marketing data analysis methods are usually limited to a single data dimension and lack in-depth mining and comprehensive analysis of implicit relationships between multi-source heterogeneous data. The present invention realizes the association of multi-source data through knowledge graphs, which can capture the correlation and implicit relationships between data from a global perspective, thereby improving the comprehensiveness and accuracy of abnormal diagnosis. In addition, feature learning and semi-supervised abnormal location assessment based on graph neural networks can continuously optimize the recognition and judgment capabilities of abnormal patterns, avoiding the problem of incomplete knowledge association in traditional methods.

[0009] The present invention introduces hyperparameter adjustment and feedback mechanisms, so that the model can be dynamically adjusted and optimized in the face of a multi-condition complex data environment. In particular, when there are many important factors affecting the power marketing data business, by introducing external condition characteristics and business hyperparameter adjustment, the continuous adaptability and flexibility of the model can be ensured, the prediction effect of the model can be optimized, thereby reducing system risks and improving the stability of power marketing data and business continuity.

[0010] As a preferred technical means: In step 1), the statistical, computational and analytical relationship system in the knowledge graph is used to strengthen the identification of data features, and key features are screened out through statistical analysis and dimensionality reduction methods to ensure the efficiency and effectiveness of model processing.

[0011] Knowledge graphs can provide deep connections between data through their rich semantic relationships and hierarchical structures. This enables the model to identify potential patterns and key features in the data at a higher level, avoiding the limitations of traditional methods that rely on a single data dimension. With the guidance of knowledge graphs, more implicit connections between data can be captured, improving the accuracy and comprehensiveness of data feature recognition.

[0012] The knowledge graph not only provides rich feature data, but also provides semantic explanations for the selection of each feature, so that the model not only has efficient prediction capabilities, but also has better interpretability, which is easier to understand and debug. Users can trace the source of specific features and the logical relationship behind them through the knowledge graph, thereby improving the transparency of the model.

[0013] In power marketing data, there are usually a lot of redundant and irrelevant features. By combining statistical analysis and dimensionality reduction methods, it is possible to remove redundant features, reduce the dimensions of data, and improve the efficiency of subsequent model processing without losing key information. This efficient feature selection and dimensionality reduction method can effectively avoid the dimensionality disaster, reduce the size of the training data set, and improve the speed and efficiency of model training.

[0014] By screening out the most representative and influential key features and reducing unnecessary feature interference, the model can be trained and predicted more efficiently. Dimensionality reduction methods can reduce the amount of calculation and improve computing efficiency, making the model more efficient when processing large-scale data, giving results in a shorter time, and improving business response speed.

[0015] Power marketing data is highly volatile and may be affected by a variety of factors, such as policy changes and market fluctuations. By using the statistical, computational, and analytical relationship system in the knowledge graph, the model can more flexibly adjust the feature recognition strategy when data characteristics change. Dimensionality reduction and feature screening methods help the model maintain good adaptability and robustness in a complex and changing environment, enabling it to cope with dynamically changing power marketing data streams.

[0016] By screening out key features through statistical analysis and dimensionality reduction, and combining with the rich semantic information provided by the knowledge graph, the model can better capture potential abnormal patterns, making anomaly detection more accurate, avoiding detection methods that rely too much on a single data source, and improving the stability of the system and anomaly detection capabilities.

[0017] As a preferred technical means: in step 2), the specific steps of constructing the data association network include: 2.1) Use the K-means clustering algorithm to divide the data into K clusters, and build a data association network G=(V,E) based on the clustering results, where the node V represents the cluster and the edge E represents the relationship between clusters; 2.2) Perform feature vector comparison and similarity calculation, use cosine similarity to compare the feature differences between different clusters, and identify clusters with obvious feature differences based on the similarity matrix S, and analyze their possible abnormal behaviors.

[0018] The K-means algorithm can automatically identify natural distribution patterns in data and cluster similar data points together. By clustering, complex and high-dimensional power marketing data can be converted into a structured form that is easier to handle, thereby helping the system discover potential patterns in massive data, avoiding the difficulty of manually screening features, and adaptively revealing the inherent structure of the data. The association network constructed through the K-means clustering results converts the relationship between data points from a single data dimension into a high-order association network; this networked structure is more expressive and can help the system more comprehensively understand and process complex relationships between data.

[0019] By mapping data nodes to clusters and establishing edges between these clusters to represent the relationships between them, complex data sets can be simplified into a hierarchical network structure. The structured representation not only makes the clustering results intuitively understandable, but also enables better analysis and location of possible sources of data anomalies. The K-means algorithm iteratively optimizes the cluster centers, making the final clustering results stable and having good generalization capabilities; in a multi-source heterogeneous data environment, K-means can effectively respond to changes in different data types and improve the stability of data processing.

[0020] Cosine similarity can accurately measure the feature differences between different clusters, and is particularly suitable for processing high-dimensional data and sparse data. By comparing feature vectors based on cosine similarity, it is possible to more flexibly and finely discover subtle differences between different clusters, effectively identify clusters with large feature changes, and quickly lock potential abnormal areas. The use of cosine similarity enables anomaly detection to dynamically adapt in feature space. For clusters with obvious feature differences, the model can detect abnormal trends earlier and mark and locate them in a timely manner.

[0021] By constructing the similarity matrix S, we can make a global comparison of the relationships between different clusters and quickly identify abnormal clusters that have large differences in characteristics from most clusters. Without manual intervention, we can automatically detect and distinguish normal and abnormal behavior patterns, thereby improving the efficiency and accuracy of anomaly detection.

[0022] The combination of K-means algorithm and cosine similarity has good computational efficiency in a big data environment. The time complexity of the K-means algorithm is O(nk), where n is the number of data points and k is the number of clusters, and the calculation of cosine similarity can be accelerated through matrix operations, which makes the entire analysis process more efficient when processing large-scale power marketing data, and can complete data processing and anomaly detection in a shorter time. The clustered data has reduced the complexity of the data through aggregation and classification, reduced the computational cost, and improved the efficiency of subsequent analysis.

[0023] Cluster analysis helps to automatically group and, by comparing the characteristics of different clusters, can reveal the characteristics of abnormal patterns, which is particularly important for identifying complex and subtle abnormal behaviors. This method can capture small data deviations caused by factors such as dynamic changes in the power marketing environment and market fluctuations, thereby more accurately identifying potential abnormal behaviors.

[0024] The combination of K-means algorithm and similarity calculation has good scalability and can be flexibly applied to different power marketing data scenarios, and the number of clusters and similarity calculation methods can be adjusted according to demand. In addition, as the amount of data increases, the algorithm can be optimized by adjusting parameters (such as K value) to improve the accuracy of clustering and the reliability of anomaly detection.

[0025] As a preferred technical means: in step 3), the specific steps of identifying the abnormal propagation path include: 3.1) Construct a weighted directed graph G=(V,E), where the node V represents the data point and the edge E represents the influence relationship between the data. Use the graph traversal algorithm to find the anomaly propagation path; 3.2) Based on complex network analysis methods, network indicators are calculated, including the degree, centrality and influence of nodes, and a propagation model is established to simulate the process of anomalies propagating from one node to another.

[0026] A weighted directed graph can clearly represent the influence relationship between data points; the node V represents the data point, and the edge E represents the influence or relationship between the data points. By weighting, the influence between different data points can be more accurately portrayed. In anomaly detection, the use of a weighted directed graph can reveal the propagation path of anomalies in the data stream, helping to accurately identify the root cause of the anomaly and its scope of influence.

[0027] Through graph traversal algorithms (such as depth-first search DFS or breadth-first search BFS), the propagation path starting from the abnormal point can be efficiently found in the graph. Graph traversal algorithms can help the system dynamically identify the process of abnormal propagation, quickly locate the abnormal propagation trajectory and its key nodes, and reduce the time required for manual troubleshooting of abnormal paths in traditional methods.

[0028] By constructing a weighted directed graph, the model can accurately identify the origin node of the anomaly and track the propagation of the anomaly from one data point to other data points. The weight of each node and edge represents the influence relationship to varying degrees, accurately determining which nodes play a key role in the propagation of the anomaly, thereby helping to find the root cause of the data anomaly. Compared with traditional rule detection methods, this graph structure can handle complex data propagation paths and provide more sophisticated and accurate anomaly tracing and positioning.

[0029] The propagation model simulates the diffusion process of anomalies in the network, further enhancing the understanding of the anomaly propagation path. By establishing a propagation model, we can better capture the dynamic process of anomaly propagation and help decision makers understand the potential causes behind data anomalies, so as to carry out targeted interventions.

[0030] Through network analysis of data relationship graphs, we can dig deeper and understand the propagation patterns of anomalies. Network analysis methods (such as node degree, centrality, influence, etc.) not only help identify the propagation paths of anomalies, but also help identify key nodes or data points in the network, which may be the center or source of anomaly propagation. With network analysis, we can reveal which nodes play an important role in the data flow by quantifying the network structure, which helps to locate the root cause of data anomalies. Network indicators provide strong support for the identification of anomaly propagation paths.

[0031] Propagation models based on complex networks (such as the SIR model) can simulate the propagation process of anomalies in the network. Through this simulation, the system can predict how anomalies spread from one node to other nodes and evaluate the speed and scope of their propagation. With this information, measures can be taken in advance to prevent the further spread of anomalies, thereby preventing large-scale failures or data errors in the system. By combining the propagation model with actual anomaly data, the system can adjust the anomaly detection mechanism more accurately, effectively predict future anomaly propagation trends, and improve anomaly early warning capabilities.

[0032] The weighted directed graph and complex network analysis methods can dynamically adapt to different power marketing data streams without the need for fixed rules or threshold settings. They are highly adaptable and can automatically adjust anomaly detection strategies according to different characteristics of data streams (such as market fluctuations, policy changes, etc.), respond to the occurrence of abnormal events in real time, and avoid false positives and false negatives. Through graph traversal and complex network analysis, anomalies can be detected in real time in data streams and their propagation paths can be tracked, ensuring that the system can respond and locate anomalies in a timely manner, reducing manual intervention and improving the efficiency of exception handling.

[0033] Graph-based analysis methods provide a global perspective that can see the relationships between different data points, thereby helping decision makers fully understand the propagation process of anomalies at the system level. This not only helps to quickly locate the root cause of the anomaly, but also provides more comprehensive information support for subsequent decision-making.

[0034] The construction of weighted directed graphs and complex network analysis methods are very suitable for processing large-scale power marketing data. In the case of extremely large amounts of data, this method can efficiently construct relationships between data and quickly identify abnormal propagation paths through graph traversal and network analysis, adapting to the needs of massive data processing. The structured representation of graphs and the calculation of network indicators can be implemented through efficient algorithms, with good computing performance and scalability, and can maintain high real-time processing capabilities in complex big data environments.

[0035] As a preferred technical means: In step 3.1), first construct a weighted directed graph , where the node V represents the data point and the edge E represents the influence relationship between the data; secondly, the graph traversal algorithm is used to find the abnormal propagation path. Given an abnormal point , calculated from To other nodes The shortest path is:

[0036] in, is the weight of each edge in the path; By traversing all nodes, we can identify the outliers. Most relevant nodes , thereby determining the propagation path of the anomaly.

[0037] In step 3.2), First, network indicators are calculated, including node degree, centrality, and influence indicators, to help identify key nodes. Defined as The number of connected edges, the centrality calculation formula is: ,in, Is a node and The distance between Secondly, based on the propagation path, a propagation model is established, assuming that the way data anomalies propagate between nodes is the SIR model, and the formula is: ,in, is the number of susceptible nodes, is the number of infected nodes, is the number of recovery nodes, is the transmission rate, is the recovery rate, which simulates the process of anomaly propagation from one node to another with the help of the propagation model.

[0038] By constructing a weighted directed graph, the weights of nodes and edges can more accurately represent the influence relationship between data points. The weight of each edge reflects the strength of association or influence between data points, thus providing a reliable basis for tracking the propagation path of anomalies. By traversing all nodes, the system can accurately identify the nodes most relevant to the anomaly, thereby determining the propagation path and impact range of the anomaly. This graph-based structured analysis is more flexible and accurate than traditional rule-based or fixed threshold methods.

[0039] The shortest path algorithm can help find the shortest path in the propagation process from the anomaly point, thereby identifying the core node of the anomaly propagation. Through path calculation, the system can effectively lock the key links in the anomaly propagation chain, reduce false positives and false negatives, and improve the efficiency of anomaly tracing.

[0040] By using the SIR propagation model to simulate the propagation process of anomalies in data streams, we can better understand how anomalies spread from one node to other nodes. The SIR model can simulate the dynamic propagation of anomalies between nodes by modeling the relationship between susceptible nodes (S), infected nodes (I), and recovery nodes (R). By adjusting the propagation rate (β) and recovery rate (γ), the speed and scope of anomaly propagation can be predicted, and decision support can be provided for anomaly detection. The SIR model provides dynamic modeling capabilities for the propagation of anomalies, can capture the diffusion trend of anomalies in the system, and help predict the possible impact range of anomalies in the future. This dynamic simulation capability provides support for the timely discovery of potential anomalies and system risks, and helps to warn and take measures in advance to avoid system failures or business interruptions.

[0041] By calculating the degree, centrality, and influence of nodes, key nodes in the network can be effectively identified. These network indicators help to discover which nodes play a key role in the propagation of anomalies, thereby providing support for anomaly tracing and impact assessment.

[0042] Degree: The degree (k_i) indicates the number of connections a node has with other nodes. Nodes with higher degrees usually have greater influence and may play an important role in anomaly propagation.

[0043] Centrality: Centrality calculations can help identify central nodes of abnormal transmission by evaluating the location of nodes in the network. These nodes may be the source of transmission or the key link of transmission.

[0044] Influence: Nodes with high influence may play a greater role in promoting the propagation of anomalies. Calculating influence can help prioritize the identification of these nodes, thereby speeding up the location and processing of anomalies.

[0045] By constructing a weighted directed graph and calculating network indicators, we can provide a global perspective on anomaly propagation, not just focusing on local data points, but identifying anomaly paths from the perspective of the overall network structure. This global perspective can reveal potential, implicit anomaly propagation chains in data streams, and effectively identify anomalies that are difficult to detect due to the mutual influence of multiple nodes. Through the combination of graph traversal, network indicator calculation, and propagation model, the anomaly tracing process is not limited to a single data stream, but is comprehensively analyzed from multiple dimensions. In this way, the anomaly source and propagation path can be located more comprehensively and accurately, greatly improving the accuracy of anomaly identification.

[0046] The SIR model and network indicator calculation can dynamically adjust the identification and evaluation strategy of abnormal propagation paths as the data flow changes, allowing the system to adapt to different types of data flows and abnormal patterns, avoiding the limitations of traditional methods that rely on fixed rules and thresholds. In a complex power marketing data environment, abnormal patterns can be very complex and difficult to predict. By combining graph structures and propagation models, it is possible to identify potential complex abnormal behaviors and respond accordingly, enhancing the system's ability to adapt to different types of abnormalities.

[0047] Through graph traversal algorithms and propagation models, the system can quickly identify anomalies and key nodes, significantly improving the efficiency of anomaly tracing. Especially in the power marketing environment with large data volumes and complex structures, traditional methods often require a lot of manual intervention, while this solution can effectively reduce labor costs and improve tracing efficiency through automated graph analysis and propagation models. Traditional rule detection methods may cause false positives or negative positives due to over-reliance on fixed thresholds, while this solution can more accurately identify the propagation path of anomalies through weighted directed graphs and complex network analysis, reduce false detections, and improve the accuracy of anomaly location.

[0048] As a preferred technical means: the specific steps of step 4) include: 4.1) Graph neural network feature learning: Use graph neural networks to extract features from data nodes and capture the complex relationships between nodes: First, aggregate neighbor information: , in Is a node In the The layer representation, Representation Node Neighbors, is a trainable weight matrix, is the activation function, is an aggregate function; Secondly, through multi-layer aggregation, the feature representation of each node is obtained , where L represents the number of layers, Represents a knowledge graph; Finally, the extracted feature representation is used as the input for subsequent anomaly detection. In the last layer, the node features is passed to a classifier for anomaly detection: , in represents the predicted anomaly label; 4.2) Abnormal feature labeling and semi-supervised positioning evaluation: First, the features are judged using the set threshold and abnormal situations are marked; Secondly, semi-supervised learning technology is used to train the model on labeled and unlabeled data, and data with known anomalies are used to guide model learning to improve the ability to detect unknown anomalies.

[0049] Graph neural networks have an excellent ability to capture complex relationships and dependencies between nodes. Through multi-layer aggregation (from aggregation of neighbor node information to learning of global information), GNN is able to learn high-dimensional features of nodes from a local to global hierarchy, thereby accurately representing the attributes of each data point and its role in the network. For power marketing data, which has complex structures and multi-level relationships, graph neural networks can effectively identify potential abnormal patterns.

[0050] By aggregating node neighbors, GNN can combine local information of each node with global information to form a feature representation with context-awareness. This enables the model to capture the complex relationships between nodes, thereby better identifying abnormal data and abnormal propagation paths.

[0051] Through the multi-level graph neural network structure, the node features finally extracted can deeply mine the potential characteristics of the data and provide them for subsequent anomaly detection tasks. This high-quality feature representation can improve the accuracy and stability of the anomaly detection model.

[0052] Graph neural networks can not only extract features from static data, but also further adapt to dynamically changing data through multi-layer learning. In practical applications, power marketing data is affected by a variety of external factors, and data characteristics often change. The multi-level aggregation characteristics of GNN enable the system to adapt to these changes and adjust its understanding of data in real time, thereby enhancing the ability to identify new and unseen anomalies.

[0053] Traditional anomaly detection methods often rely on fixed rules or thresholds, while graph neural networks can better adapt to complex anomaly patterns by learning complex relationships and propagation patterns between nodes, reduce false positives and negatives, and significantly improve the accuracy and robustness of anomaly detection.

[0054] By setting thresholds to judge node features, the model can automatically mark possible abnormal data. This effectively reduces manual intervention and improves the automation level of the system. At the same time, feature extraction based on graph neural networks can provide higher accuracy for abnormal marking, avoiding the problems of fixed thresholds and manual adjustments that traditional rules rely on.

[0055] Traditional anomaly detection usually requires a large amount of labeled data, which may be difficult to obtain in practical applications. Semi-supervised learning technology can improve anomaly detection capabilities by combining a small amount of labeled data with a large amount of unlabeled data. Data with known anomalies is used to guide model learning, thereby improving the ability to detect unknown anomalies and adapting to complex and incompletely labeled data environments. Semi-supervised learning significantly reduces the cost of manual labeling by combining unlabeled data, allowing anomaly detection models to be trained and deployed more efficiently in practical applications. Between labeled and unlabeled data, semi-supervised learning can gradually discover new abnormal patterns through self-learning, especially in scenarios where data is scarce or difficult to label, which can effectively improve the ability to identify new anomalies.

[0056] The feature learning of graph neural networks is not limited to the attributes of local nodes. It can capture the relationships, interdependencies and global structures between nodes. Therefore, the system can identify anomalies in a wider context. It can not only identify local anomalies, but also find the sources of anomalies that may have an impact in the global structure, thereby providing more accurate judgments for anomaly location. Through the deep features extracted by graph neural networks, combined with the anomaly labeling and evaluation of semi-supervised learning, the system can accurately locate the source of anomalies and reduce the ambiguity of the root cause of anomalies in traditional methods. This is crucial for the anomaly monitoring of large-scale power marketing data, which can respond quickly and take measures.

[0057] Power marketing data is affected by many factors, with large amounts of data and constant changes. The flexibility and adaptability of graph neural networks enable the system to adapt to data changes in a timely manner, and maintain efficient anomaly detection capabilities when facing complex dynamic data streams, allowing the system to run for a long time without frequent adjustments.

[0058] Through automated feature learning and anomaly detection, the system can efficiently identify abnormal nodes and propagation paths, thereby helping decision makers focus resources on processing the most critical anomalies and avoiding unnecessary waste of resources.

[0059] As a preferred technical means: the specific steps of step 5) include: 5.1) Define multiple hyperparameters that affect business data, denoted as , where each hyperparameter Represents a specific business condition and combines these hyperparameters with the input features H of the graph neural network:

[0060] in, The eigenvector representing the external conditions, Represents the join operation of features; 5.2) Use cross-validation method to optimize and select hyperparameters: divide the data into k subsets, take each subset as validation set in turn, and use the remaining k-1 subsets as training sets. In each round, adjust the hyperparameter H and train the model, record the performance index of the validation set, and select the performance index P to evaluate the model effect:

[0061] in, is a true positive, It is a false positive; grid search and Bayesian optimization are used to find the best hyperparameter combination : ; 5.3) Through the feedback mechanism, the output results of the model are fed back to the model to adjust and optimize the parameters and hyperparameters of the model; the feedback model is:

[0062] in is the predicted output of the model, is the actual label, is the feedback function used to calculate the model performance difference; Further parameter adjustment is performed to update the model parameters W and hyperparameters H according to the feedback information:

[0063]

[0064] in, is the learning rate, is the gradient of the loss function, is the feedback influence coefficient.

[0065] In the task of tracing and locating anomalies in power marketing data, the diversity and complexity of the data require that the model can be flexibly adjusted under different business conditions. By defining multiple data hyperparameters that affect the business (such as business environment, market fluctuations, policy changes, etc.), this technical solution can optimize the model for different actual situations, so that the effect of anomaly detection is more in line with actual needs. By combining hyperparameters with external conditions, the model can consider changes in both data and the external environment. For example, when the power market fluctuates or policies change, the system can adjust the input features in real time and optimize the model to adapt to the changing business needs and external environment, thereby improving the accuracy and flexibility of anomaly detection and tracing and positioning.

[0066] The cross-validation method can effectively avoid overfitting or underfitting of the model and ensure the generalization ability of the model. By dividing the data into multiple subsets, the model is trained on different training sets and validation sets, thereby ensuring the adaptability of the model to different data sets, making the final selected hyperparameter combination more stable. By selecting TP (true positive) / (TP+FP) (false positive rate) as the performance indicator for model evaluation, the accuracy of the model in the anomaly detection task can be effectively monitored and optimized. Ensure that the true accuracy of anomaly detection is maximized during the optimization process of the model. Grid search and Bayesian optimization methods can efficiently search for the best hyperparameter combination, especially when the parameter space is large, which can avoid the tedious steps of manually adjusting hyperparameters, thereby improving the efficiency of model optimization. When dealing with high-dimensional complex problems, Bayesian optimization can automatically select the optimal hyperparameters through statistical models, reduce computational costs and improve model performance.

[0067] By comparing the model's prediction results with the actual labels, the model is adjusted in real time using the feedback mechanism. This feedback mechanism can significantly improve the model's adaptability and avoid wrong decisions caused by large deviations between the model's predictions and the actual situation. This mechanism enables the model to dynamically adjust parameters according to changes in the actual situation, making the system more intelligent and adaptive. During the model optimization process, the model parameters and hyperparameters are adjusted by calculating the gradient of the loss function and feedback information. This step enables the model to adjust itself more accurately after each iteration, avoiding the error accumulation problem caused by static parameter settings. At the same time, the introduction of the feedback coefficient λ further enhances the flexibility and accuracy of the model's parameter adjustment during the feedback process. Since the system can automatically adjust parameters based on actual feedback, frequent manual intervention is not required. Users only need to pay attention to the performance indicators of the model (such as TP, FP), and the model itself can be automatically optimized in practice, thereby improving efficiency and reducing labor costs.

[0068] The business environment, market fluctuations, policy changes and other factors of power marketing data are dynamic and complex, and traditional models may not be able to effectively cope with these rapid changes. By introducing external condition features and hyperparameter adjustments, the model can quickly respond to environmental changes and optimize decisions. This flexible adaptability is particularly important when facing a complex and ever-changing marketing environment. The model can dynamically adjust hyperparameters according to business needs and actual data in each iteration to ensure that anomaly detection can maintain high accuracy under different conditions. Through this dynamic adjustment, the system can adapt to various uncertainties and continue to provide high-quality detection results as data and business needs change.

[0069] By continuously optimizing hyperparameters and combining feedback mechanisms, the model's accuracy in identifying anomalies can be continuously improved, and false positives and missed reports can be reduced, thereby improving the efficiency and accuracy of anomaly detection. This is especially important for extremely large and complex data sets such as power marketing data, which can help quickly identify potential risks and take effective countermeasures. Through accurate anomaly detection and efficient feedback mechanisms, the model can provide more accurate information support to power marketing decision makers, helping them to respond quickly to sudden market changes and reduce possible losses.

[0070] Through precise hyperparameter adjustment and feedback mechanisms, the model can identify anomalies that really need attention and avoid wasting resources on irrelevant or unimportant anomalies. In the power marketing scenario, this refined management helps improve overall operational efficiency.

[0071] The second technical solution of the present invention is to provide a data anomaly tracing and locating system, which applies the aforementioned knowledge-driven data anomaly tracing and locating method, and the system includes: Data preprocessing and feature extraction module: used to preprocess power marketing data, including data cleaning, deduplication, and standardization operations, and use the statistical, computational, and analytical relationship system in the knowledge graph to strengthen the identification of data features, and select key features through statistical analysis and dimensionality reduction methods to ensure the efficiency and effectiveness of model processing; Similarity and Difference Analysis Module: The K-means clustering algorithm is used to divide the data into K clusters, and a data association network is constructed based on the clustering results. The feature differences between different clusters are compared by comparing feature vectors and calculating similarity, and the cosine similarity is used to compare the feature differences between different clusters. Clusters with obvious feature differences are identified based on the similarity matrix, and their possible abnormal behaviors are analyzed. A data association network is constructed through cluster analysis and statistical methods to identify data clusters of the same location or type. The differences in data features are analyzed, and the similarities and differences of different data results or application scenarios are distinguished and compared based on the data influence relationship position. The Similarity and Difference Analysis Module constructs a weighted directed graph, uses a graph traversal algorithm to find the abnormal propagation path, and calculates network indicators based on complex network analysis methods. The network indicators include the degree, centrality and influence of the node, and establishes a propagation model to simulate the process of abnormal propagation from one node to another. Abnormal propagation path reasoning module: It adopts path reasoning technology and complex network analysis methods to identify and locate the root cause of data anomalies by analyzing the propagation path of abnormal points in the data flow; the abnormal propagation path reasoning module uses graph neural networks to extract features from data nodes, capture the complex relationship between nodes, and use the set threshold to judge the features, mark abnormal situations, and use semi-supervised learning technology to train the model to improve the detection ability of unknown anomalies; Abnormal tracing and positioning model construction module: Build a data abnormality tracing and positioning model, use graph neural network methods for feature learning, and identify the full link of abnormal data generation through knowledge graphs, realize the judgment and marking of raw data abnormalities, and perform semi-supervised abnormality positioning evaluation; Result optimization and hyperparameter adjustment module: introduces multiple conditional factors that affect business data information into the model as hyperparameters, adjusts the model's hyperparameters, and uses the feedback mechanism to dynamically adjust the model output results; the result optimization and hyperparameter adjustment module defines multiple hyperparameters of data that affect the business through the construction module, and combines them with the input features of the graph neural network, uses the cross-validation method to optimize and select the hyperparameters, and feeds the model's output results back to the model through the feedback mechanism to adjust and optimize the model's parameters and hyperparameters.

[0072] The data preprocessing and feature extraction module effectively solves the noise, duplication and scale difference problems in the original data by cleaning, deduplication and standardization of power marketing data. By adopting the statistical, computational and analytical relationship system in the knowledge graph, it is possible to deeply explore the potential relationship between data.

[0073] Statistical analysis and dimensionality reduction methods are used to screen out key features, ensuring that the model can focus on the most valuable information during processing, improving computational efficiency and the accuracy of anomaly detection. This ensures efficient data processing and captures potential abnormal patterns.

[0074] By using the K-means clustering algorithm to divide the data, the power marketing data is divided into K clusters. Such a division helps to identify the potential relationship between the data and reduce the interference of noise data.

[0075] The cosine similarity is used to analyze the feature differences between clusters, so that similar data can be classified into the same category, and different data can be identified in time. The similarity matrix helps to identify clusters with obvious differences and further analyze possible abnormal behaviors.

[0076] By constructing a weighted directed graph to represent the influence relationship between data and using a graph traversal algorithm to find the anomaly propagation path, this method can accurately simulate the propagation process of anomalies in data streams, identify the root nodes of anomalies, and track the diffusion process of anomalies in the network.

[0077] By calculating network indicators (such as node degree, centrality, influence, etc.), this system can identify key nodes and key paths of data flows, help analyze the trend and scope of anomaly propagation, and improve the accuracy of anomaly source location.

[0078] By learning the features of data nodes through graph neural networks (GNNs), it is possible to capture the complex relationships between nodes. Especially in the multi-dimensional and multi-source environment of power marketing data, GNNs can effectively handle non-Euclidean relationships between data and improve the accuracy of anomaly detection.

[0079] In the abnormal feature labeling and semi-supervised positioning evaluation stage, semi-supervised learning is used to improve the model's ability to detect unknown anomalies by combining labeled and unlabeled data. This not only marks known anomalies, but also improves the model's ability to identify new or uncommon anomalies.

[0080] By using the knowledge graph and incorporating the related information of each link and data node into the analysis framework, the system can achieve full-link anomaly tracing and location. This not only improves the accuracy of identifying the anomaly source, but also captures more complex anomaly patterns and interrelated abnormal behaviors.

[0081] A semi-supervised anomaly localization and evaluation method is adopted, which combines labeled and unlabeled data for training, thus enhancing the flexibility of anomaly detection, which is especially suitable for complex systems such as power marketing data that are changing dynamically.

[0082] By introducing multiple hyperparameters that affect business data information into the model and combining them with the input features of the graph neural network, the model's processing method can be adjusted in a dynamic environment to adapt to different business needs and changes.

[0083] The cross-validation method is used to optimize and select hyperparameters, and the model output is compared with the actual results through the feedback mechanism to dynamically adjust the model parameters and hyperparameters. This can continuously improve the adaptability and accuracy of the model, reduce human intervention and optimize system performance.

[0084] The model output is compared with the actual label through the feedback mechanism, and the model is adjusted and optimized in real time using the feedback function. Combined with hyperparameter adjustment, the accuracy of the model can be continuously optimized to ensure that the system can cope with the rapidly changing business needs in the power marketing data environment.

[0085] The third technical solution of the present invention is: a computer device, which includes one or more processors and one or more memories, and the one or more memories store at least one program code. When the program code is executed by the one or more processors, the aforementioned knowledge-driven data anomaly tracing and locating method is implemented.

[0086] The fourth technical solution of the present invention is: a storage medium, in which at least one program code is stored, characterized in that when the program code is executed by a processor, the steps of the aforementioned knowledge-driven data anomaly tracing and locating method are implemented.

[0087] Beneficial effects: 1. Improve the accuracy of anomaly detection: This invention adopts a knowledge-driven data anomaly tracing and positioning method, and overcomes the shortcomings of traditional rule detection that relies on fixed thresholds by constructing a knowledge graph and graph neural network of multi-source heterogeneous data. By using dynamic feature analysis and deep learning models, the accuracy of anomaly detection is significantly improved, false positives and missed positives are reduced, and the system can more flexibly adapt to complex and changeable power marketing data flows, thereby ensuring timely response to potential risks and improving the effectiveness of overall data monitoring.

[0088] 2. Optimize the efficiency of source tracing and positioning: By establishing a data anomaly source tracing and positioning model with graph neural network as the core, the present invention significantly improves the efficiency of source tracing and positioning. Combining path reasoning and complex network analysis technology, the source tracing process after the anomaly occurs no longer relies on a large amount of manual intervention, and the root cause of the anomaly can be quickly identified and located. This efficient source tracing analysis method not only reduces time and labor costs, but also improves the stability of power marketing data and business continuity, thereby optimizing the response speed of business decisions.

[0089] 3. Enhanced knowledge association analysis capabilities: This invention realizes deep association analysis between multi-source data through the construction of knowledge graphs, breaking through the limitations of traditional analysis methods. Through the knowledge-driven mechanism from a global perspective, the system can capture the implicit relationships and potential knowledge between data. This enhanced knowledge association analysis capability not only enables effective linkage analysis of abnormal phenomena and the reasons behind them, but also improves the system's ability to recognize complex abnormal patterns, thereby optimizing the adaptability and flexibility of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0091] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0092] Step 1: Data preprocessing and feature extraction. In view of the multi-source heterogeneity of power marketing data, data cleaning, deduplication and standardization operations are performed to ensure data consistency and integrity. At the same time, combined with the existing power marketing knowledge graph associated with multi-source heterogeneous data, the characteristics of electricity consumption data are extracted, and the statistical, computational and analytical relationship system in the knowledge graph is used to strengthen the identification of data features. Key features are screened out through statistical analysis and dimensionality reduction methods to ensure the efficiency and effectiveness of model processing.

[0093] Assume that the electricity consumption data matrix is , then the feature extraction process generates features represented by H through the neural network model, .in, is the original electricity consumption data, is the normalized electricity consumption data matrix, W is the weight matrix, and f(·) is the activation function.

[0094] Step 2: Similarities and differences analysis. Combined with the power consumption feature analysis, statistics and calculation results, a data association network is constructed to cluster data of similar locations or types. By analyzing the data impact relationship, the similarities and differences of different data results or application scenarios are distinguished and compared, the differences in data features are identified, and then the similarities and differences are analyzed. The specific contents are as follows: Step 2.1: Clustering to build a data association network. When performing similarity and difference analysis, the extracted feature data must first be clustered to identify data clusters of the same location or type. The K-means clustering algorithm is used to divide the data into K clusters so that the data similarity within the cluster is maximized and the data difference between clusters is maximized. First, randomly select K initial cluster centers. ; Secondly, for each data point , calculate its distance to each cluster center , assign it to the cluster with the nearest distance; then, update the cluster center and calculate the new cluster center as ,in for all data points in cluster j; finally, repeat the above steps until the cluster center no longer changes or the change is less than the set threshold.

[0095] Through the above steps, we finally get K clusters Next, we build a data association network based on the clustering results. , where nodes V represent clusters and edges E represent the relationship between clusters. The weight of the edge can be calculated based on the distance between the cluster centers. To define, specifically: ,in To prevent the denominator from being a constant of 0.

[0096] Step 2.2: Perform data similarity and difference analysis. After clustering is completed, it is necessary to compare the similarities and differences of different data results or application scenarios based on the data influence relationship position. By constructing the data association network G, the relationship and feature differences between clusters can be analyzed. The specific method is as follows: First, perform feature vector comparison. For each cluster , calculate its eigenvector The mean and variance of , ; Secondly, similarity calculation is performed, and cosine similarity is used to compare the feature differences between different clusters, which is defined as ,in and Cluster and Finally, we conduct data similarity and difference analysis. Based on the similarity matrix S, we can identify clusters with obvious feature differences. For example, we can set a threshold ,if , it is considered a cluster and clustering There are significant differences in features. At this point, we can further analyze the differences between the two clusters in application scenarios and identify possible abnormal behaviors.

[0097] Step 3: Anomaly propagation path reasoning. The goal of anomaly propagation path reasoning is to identify and locate the root cause of data anomalies. To this end, we will use path reasoning technology and complex network analysis methods to analyze the propagation path of anomalies in the data flow and reveal their impact relationships: Step 3.1: Study the path reasoning technology for data anomaly propagation. First, build a weighted directed graph. , where the node V represents the data point, the edge E represents the influence relationship between the data, and the weight of the edge Calculated from the result of the previous step 2. Secondly, use the graph traversal algorithm to find the abnormal propagation path. Given an abnormal point , we can calculate from To other nodes The shortest path is as follows:

[0098] in, is the weight of each edge in the path.

[0099] By traversing all nodes, we can identify the outliers Most relevant nodes , thereby determining the propagation path of the anomaly.

[0100] Step 3.2: Complex network analysis. After completing the path reasoning, further analyze the propagation characteristics of the outliers, and understand the interaction between the nodes in the data flow and the mechanism of anomaly propagation based on the complex network analysis method. First, calculate the network indicators, calculate the degree, centrality and influence of the nodes, and help identify the key nodes. Defined as The number of connected edges, the centrality is calculated as follows: ,in, Is a node and The distance between them; secondly, based on the propagation path, a propagation model is established, assuming that the way data anomalies propagate between nodes is the SIR (susceptible-infected-recovered) model, and the formula is as follows: ,in, is the number of susceptible nodes, is the number of infected nodes, is the number of recovery nodes, is the transmission rate, is the recovery rate. With this model, we can simulate the process of anomaly propagation from one node to another.

[0101] Step 4: Anomaly tracing and positioning model construction. In the process of data anomaly tracing and positioning, we will comprehensively use the graph neural network method to build a preliminary data anomaly tracing and positioning model, and identify the full link of abnormal data generation through the knowledge graph to realize the judgment and marking of raw data anomalies, so as to conduct semi-supervised anomaly positioning evaluation. The specific contents are as follows: Step 4.1: Graph neural network feature learning. Use graph neural network to extract features from data nodes and capture the complex relationships between nodes. First, aggregate neighbor information. ,in Is a node In the The layer representation, Representation Node Neighbors, is a trainable weight matrix, is the activation function, is the aggregation function; secondly, through multi-layer aggregation, the feature representation of each node is obtained , where L represents the number of layers, Represents the knowledge graph; finally, the extracted feature representation is used as the input for subsequent anomaly detection. In the last layer, the node features is passed to a classifier for anomaly detection: ,in Represents the predicted anomaly label.

[0102] Step 4.2: Abnormal feature labeling and semi-supervised positioning evaluation. When the original data is found to be abnormal, we will judge the data problem and mark the abnormal features of the original data. First, use the set threshold to judge the features and mark the abnormalities; second, use semi-supervised learning technology to train the model on the labeled and unlabeled data, and use the known abnormal data to guide the model learning, so as to improve the detection ability of unknown anomalies.

[0103] Step 5: Result optimization and hyperparameter adjustment: Introduce multiple conditional factors that affect business data information (such as policy changes, abnormal events, weather changes, etc.) into the model as hyperparameters to enhance its adaptability and accuracy.

[0104] Step 5.1: Define multiple hyperparameters that affect the business data, denoted as , where each hyperparameter Represents a specific business condition (such as policy changes, weather, etc.), and combines these hyperparameters with the input features H of the graph neural network:

[0105] in, The eigenvector representing the external conditions, Represents a join operation on a feature.

[0106] Step 5.2: In order to obtain better model performance, further use cross-validation and other methods to optimize and select hyperparameters: divide the data into k subsets, take each subset as the validation set in turn, and use the remaining k-1 subsets as the training set. In each round, adjust the hyperparameter H and train the model, and record the performance indicators of the validation set. For example, select the performance indicator P to evaluate the model effect:

[0107] in, is a true positive, It is a false positive. Based on this, grid search, Bayesian optimization and other methods are used to find the best hyperparameter combination. : .

[0108] Step 5.3: Feed the model’s output back to the model through the feedback mechanism to adjust and optimize the model’s parameters and hyperparameters. The feedback model is as follows:

[0109] in is the predicted output of the model, is the actual label, is the feedback function used to calculate the model performance difference.

[0110] Further parameter adjustment is performed to update the model parameters W and hyperparameters H according to the feedback information:

[0111]

[0112] in, is the learning rate, is the gradient of the loss function, is the feedback influence coefficient.

[0113] Through the above steps, the data anomaly tracing and positioning model can be effectively optimized to better adapt to the changing power marketing environment and complex data characteristics, thereby improving overall business efficiency and decision-making support capabilities.

[0114] In order to verify the technical solution of the present invention, an experimental simulation test of the data anomaly tracing and locating method was carried out. The simulation data set includes power marketing data, business scenario simulation data, historical anomaly data, etc. The amount of data used in the simulation exceeds 500,000. In order to ensure the representativeness of the simulation results, the data is divided into a training set and a test set, where the training set contains 80% of the data and the test set contains 20% of the data.

[0115] Table 1 Application effect of knowledge-driven data anomaly tracing and location method

[0116] Through multiple rounds of simulation comparison, the superiority of the present invention in anomaly detection and source tracing and positioning was verified. The test results in different scenarios show that: in dynamically changing power marketing data, the false alarm rate and missed alarm rate of the present invention method are significantly lower than those of traditional methods, and it can accurately capture complex abnormal patterns; the source tracing and positioning time is significantly shortened, and the present invention method greatly improves the speed and accuracy of abnormal source positioning through optimized data association path reasoning and graph neural network feature learning; processing accuracy and cost-effectiveness are optimized. Compared with traditional manual intervention and rule setting methods, the present invention not only improves the accuracy of abnormal processing, but also reduces the workload of business personnel and reduces operating costs through automated abnormal tracing.

[0117] Embodiment 2 Accordingly, this embodiment provides a knowledge-driven data anomaly tracing and locating system, including a data preprocessing and feature extraction module, a similarity and difference analysis module, an anomaly propagation path reasoning module, an anomaly tracing and locating model construction module, and a result optimization and hyperparameter adjustment module.

[0118] The data preprocessing and feature extraction module mainly applies data cleaning, deduplication and standardization techniques to ensure the consistency and integrity of power marketing data. At the same time, statistical analysis methods are used to calculate the mean and variance of key features, and dimensionality reduction techniques such as principal component analysis are used to reduce data dimensions. Finally, a high-quality power consumption data feature matrix is ​​generated, providing a reliable basis for subsequent analysis. This module is used to implement the function of step 1 in Example 1, which will not be repeated here.

[0119] The similarity and difference analysis module uses a clustering algorithm to identify data clusters of the same location or type. By clustering the extracted feature data, a data association network is formed, and the feature differences between different clusters are calculated using cosine similarity to help analyze the data impact relationship. The module ultimately outputs the clustering results and the data association network, clarifying the similarities and differences between different data results or application scenarios. This module is used to implement the function of step 2 in Example 1, which will not be repeated here.

[0120] The abnormal propagation path reasoning module uses path reasoning technology and complex network analysis methods to construct a weighted directed graph to represent the influence relationship between data points. By traversing the nodes in the graph, the propagation path of the abnormal point is identified, and the shortest path between the nodes is calculated. In addition, the SIR model is used to analyze the propagation characteristics of the abnormality, thereby revealing the abnormal propagation mechanism in the data stream. This module is used to implement the function of step 3 in Example 1, which will not be repeated here.

[0121] The abnormal source tracing and positioning model construction module uses a combination of graph neural networks and semi-supervised learning technology to extract features from data nodes and establish a preliminary abnormal source tracing and positioning model. This module uses the knowledge graph to identify the full-link link of abnormal data generation, realizes the judgment and marking of raw data anomalies, improves the detection capability of unknown anomalies, and enhances the practicality of the model. This module is used to implement the function of step 4 in Example 1, which will not be repeated here.

[0122] The result optimization and hyperparameter adjustment module adjusts the hyperparameters of the model through cross-validation and optimization methods. Combined with the multi-conditional factors that affect the business, the module introduces these hyperparameters into the graph neural network to improve the accuracy and adaptability of the model. At the same time, the feedback mechanism is used to dynamically adjust the model output results to ensure that the parameters and hyperparameters of the model are continuously optimized to adapt to the complex power marketing environment. This module is used to implement the function of step 5 in Example 1, which will not be repeated here.

[0123] Embodiment 3 This embodiment provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the knowledge-driven data anomaly tracing and locating method as described in any embodiment of the present invention.

[0124] Embodiment 4 This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the knowledge-driven data anomaly tracing and locating method as described in any embodiment of the present invention is implemented.

[0125] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed in the present invention can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0126] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0128] The data anomaly tracing and locating method, system, device and storage medium shown above are specific embodiments of the present invention, which have embodied the substantial characteristics and progress of the present invention. According to actual use needs and under the guidance of the present invention, equivalent modifications in shape, structure, etc. can be made to them, which are all within the protection scope of this scheme.

Claims

1. A knowledge-driven data anomaly tracing and location method, characterized in that: The following steps are involved: 1) Preprocessing and feature extraction of power marketing data, including data cleaning, deduplication, and standardization operations, as well as extracting the features of power consumption data using knowledge graphs and screening out key features; 2) Through cluster analysis and statistical methods, build a data association network, analyze the differences in data characteristics, distinguish the similarities and differences between different data results or application scenarios, and compare them; 3) Use path reasoning technology and complex network analysis methods to identify and locate the root causes of data anomalies by analyzing the propagation paths of abnormal points in the data flow; 4) Build a data anomaly tracing and positioning model, use graph neural network methods for feature learning, and use knowledge graphs to identify the entire link of abnormal data generation, realize the judgment and marking of raw data anomalies, and perform semi-supervised anomaly positioning evaluation; 5) Optimize results and adjust hyperparameters, introduce multiple conditional factors that affect business data information into the model as hyperparameters, adjust the model's hyperparameters, and use the feedback mechanism to dynamically adjust the model's output results.

2. According to claim 1, a knowledge-driven data anomaly tracing and positioning method is characterized by: In step 1), the statistical, computational, and analytical relationship system in the knowledge graph is used to strengthen the identification of data features, and key features are screened out through statistical analysis and dimensionality reduction methods to ensure the efficiency and effectiveness of model processing.

3. A knowledge-driven data anomaly tracing and positioning method according to claim 2, characterized in that: In step 2), the specific steps of constructing the data association network include: 2.1) Use the K-means clustering algorithm to divide the data into K clusters, and build a data association network G=(V,E) based on the clustering results, where the node V represents the cluster and the edge E represents the relationship between clusters; 2.2) Perform feature vector comparison and similarity calculation, use cosine similarity to compare the feature differences between different clusters, and identify clusters with obvious feature differences based on the similarity matrix S, and analyze their possible abnormal behaviors.

4. A knowledge-driven data anomaly tracing and positioning method according to claim 3, characterized in that: In step 3), the specific steps of identifying the abnormal propagation path include: 3.1) Construct a weighted directed graph G=(V,E), where the node V represents the data point and the edge E represents the influence relationship between the data. Use the graph traversal algorithm to find the anomaly propagation path; 3.2) Based on complex network analysis methods, network indicators are calculated, including the degree, centrality and influence of nodes, and a propagation model is established to simulate the process of anomalies propagating from one node to another.

5. A knowledge-driven data anomaly tracing and positioning method according to claim 4, characterized in that: In step 3.1), we first construct a weighted directed graph , where the node V represents the data point and the edge E represents the influence relationship between the data; secondly, the graph traversal algorithm is used to find the abnormal propagation path. Given an abnormal point , calculated from To other nodes The shortest path is: in, is the weight of each edge in the path; By traversing all nodes, we can identify the outliers. Most relevant nodes , thereby determining the propagation path of the anomaly; In step 3.2), First, network indicators are calculated, including node degree, centrality, and influence indicators, to help identify key nodes. Defined as The number of connected edges, the centrality calculation formula is: ,in, Is a node and The distance between Secondly, based on the propagation path, a propagation model is established, assuming that the way data anomalies propagate between nodes is the SIR model, and the formula is: ,in, is the number of susceptible nodes, is the number of infected nodes, is the number of recovery nodes, is the transmission rate, is the recovery rate, which simulates the process of anomaly propagation from one node to another with the help of the propagation model.

6. A knowledge-driven data anomaly tracing and positioning method according to claim 5, characterized in that: The specific steps of step 4) include: 4.1) Graph neural network feature learning: Use graph neural networks to extract features from data nodes and capture the complex relationships between nodes: First, aggregate neighbor information: ， in Is a node In the The layer representation, Representation Node Neighbors, is a trainable weight matrix, is the activation function, is an aggregate function; Secondly, through multi-layer aggregation, the feature representation of each node is obtained , where L represents the number of layers, Represents a knowledge graph; Finally, the extracted feature representation is used as the input for subsequent anomaly detection. In the last layer, the node features is passed to a classifier for anomaly detection: ， in represents the predicted anomaly label; 4.2) Abnormal feature labeling and semi-supervised positioning evaluation: First, the features are judged using the set threshold and abnormal situations are marked; Secondly, semi-supervised learning technology is used to train the model on labeled and unlabeled data, and data with known anomalies are used to guide model learning to improve the ability to detect unknown anomalies.

7. A knowledge-driven data anomaly tracing and positioning method according to claim 6, characterized in that: The specific steps of step 5) include: 5.1) Define multiple hyperparameters that affect the business data, denoted as , where each hyperparameter Represents a specific business condition and combines these hyperparameters with the input features H of the graph neural network: in, The eigenvector representing the external conditions, Represents the join operation of features; 5.2) Use cross-validation method to optimize and select hyperparameters: divide the data into k subsets, take each subset as validation set in turn, and use the remaining k-1 subsets as training sets. In each round, adjust the hyperparameter H and train the model, record the performance index of the validation set, and select the performance index P to evaluate the model effect: in, is a true positive, It is a false positive; grid search and Bayesian optimization are used to find the best hyperparameter combination : ; 5.3) Through the feedback mechanism, the output results of the model are fed back to the model to adjust and optimize the parameters and hyperparameters of the model; the feedback model is: in is the predicted output of the model, is the actual label, is the feedback function used to calculate the model performance difference; Further parameter adjustment is performed to update the model parameters W and hyperparameters H according to the feedback information: in, is the learning rate, is the gradient of the loss function, is the feedback influence coefficient.

8. A knowledge-driven data anomaly tracing and positioning system, characterized in that: A knowledge-driven data anomaly tracing and positioning method as described in any one of claims 1 to 7 is applied, wherein the system comprises: Data preprocessing and feature extraction module: used to preprocess power marketing data, including data cleaning, deduplication, and standardization operations, and use the statistical, computational, and analytical relationship system in the knowledge graph to strengthen the identification of data features, and select key features through statistical analysis and dimensionality reduction methods to ensure the efficiency and effectiveness of model processing; Similarity and Difference Analysis Module: The K-means clustering algorithm is used to divide the data into K clusters, and a data association network is constructed based on the clustering results. The feature differences between different clusters are compared by comparing feature vectors and calculating similarity, and the cosine similarity is used to compare the feature differences between different clusters. Clusters with obvious feature differences are identified based on the similarity matrix, and their possible abnormal behaviors are analyzed. A data association network is constructed through cluster analysis and statistical methods to identify data clusters of the same location or type. The differences in data features are analyzed, and the similarities and differences of different data results or application scenarios are distinguished and compared based on the data influence relationship position. The Similarity and Difference Analysis Module constructs a weighted directed graph, uses a graph traversal algorithm to find the abnormal propagation path, and calculates network indicators based on complex network analysis methods. The network indicators include the degree, centrality and influence of the node, and establishes a propagation model to simulate the process of abnormal propagation from one node to another. Abnormal propagation path reasoning module: It adopts path reasoning technology and complex network analysis methods to identify and locate the root cause of data anomalies by analyzing the propagation path of abnormal points in the data flow; the abnormal propagation path reasoning module uses graph neural networks to extract features from data nodes, capture the complex relationship between nodes, and use the set threshold to judge the features, mark abnormal situations, and use semi-supervised learning technology to train the model to improve the detection ability of unknown anomalies; Abnormal tracing and positioning model construction module: Build a data abnormality tracing and positioning model, use graph neural network methods for feature learning, and identify the full link of abnormal data generation through knowledge graphs, realize the judgment and marking of raw data abnormalities, and perform semi-supervised abnormality positioning evaluation; Result optimization and hyperparameter adjustment module: introduces multiple conditional factors that affect business data information into the model as hyperparameters, adjusts the model's hyperparameters, and uses the feedback mechanism to dynamically adjust the model output results; the result optimization and hyperparameter adjustment module defines multiple hyperparameters of data that affect the business through the construction module, and combines them with the input features of the graph neural network, uses the cross-validation method to optimize and select the hyperparameters, and feeds the model's output results back to the model through the feedback mechanism to adjust and optimize the model's parameters and hyperparameters.

9. A computer device, characterized in that: The device includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories. When the program code is executed by the one or more processors, a knowledge-driven data anomaly tracing and locating method as described in any one of claims 1 to 7 is implemented.

10. A storage medium, wherein at least one program code is stored in the storage medium, characterized in that: When the program code is executed by a processor, the steps of a knowledge-driven data anomaly tracing and locating method as described in any one of claims 1 to 7 are implemented.

Images