Method for checking abnormal charging data by using data consanguinity topological relation

Abstract

The invention provides a method for checking abnormal charging data by using a data consanguinity topological relation, relates to the technical field of data processing, and forms a data consanguinity network by collecting multi-dimensional data from a charging system, constructing a data consanguinity topological graph and marking key nodes. And constructing a batch anomaly detection model based on a random forest, a user behavior anomaly detection model of a support vector machine, and a periodic anomaly detection model based on STL decomposition and a Prophet model, and identifying abnormal behaviors, equipment faults and seasonal load changes in charging. Meanwhile, a real-time anomaly detection system is designed, charging data flow is analyzed in real time based on a sliding window, and sudden anomalies such as equipment shutdown are quickly responded. After the abnormality is detected, the propagation path and influence of the abnormal data are traced and analyzed by utilizing the topological graph, processing is carried out aiming at different abnormality types, the system is timely fed back and optimized, and the stability and data processing efficiency of the charging and battery swapping system are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method for checking abnormal charging and swapping data using data lineage topological relationships. Background Art

[0002] With the rapid development of electric vehicles and charging infrastructure, the charging and swapping field has ushered in a comprehensive challenge in the era of big data. The amount of data processed by enterprises and charging station operators has increased dramatically, including not only basic information such as user charging behavior, power consumption, and device status, but also complex related data such as geographic location, time pattern, and device interoperability. Faced with this massive and highly interconnected data set, how to efficiently manage, effectively utilize, and promptly identify potential abnormal conditions, such as abnormal charging behavior and equipment failure warnings, has become the focus of industry attention.

[0003] In view of the characteristics of charging and swapping data, a data anomaly detection strategy based on data lineage topology can be used to optimize the data processing and monitoring process. Specifically, the strategy includes: Data analysis method based on the blood relationship of charging and swapping data: By pre-building a blood relationship diagram between charging and swapping data, the source, flow and interdependence of each charging record, equipment status change and other data are clarified. When data analysis or anomaly detection is required, the system can quickly locate and extract data sets that are closely related to the target data field from the data warehouse or real-time data stream. Based on these comprehensive and accurate data associations, in-depth analysis can significantly improve the accuracy and response speed of anomaly detection, such as quickly identifying high failure rates or abnormal charging patterns of charging stations in a certain area within a certain period of time.

[0004] Anomaly detection method for charging and swapping that integrates topological structure and attribute information: Given that the charging and swapping network has natural spatial distribution characteristics and complex interactions between devices, we can draw on the network topology analysis method and combine the attribute information of the charging and swapping facilities (such as power, voltage, temperature, etc.) to build a charging and swapping attribute network. Using graph theory, machine learning and other algorithms, we can detect abnormal nodes or abnormal patterns in the network, such as abnormal charging time distribution, abnormally high power loss, or abnormally frequent switching of equipment. This method can not only effectively identify problems with a single device, but also reveal potential risks in the entire charging and swapping network, providing strong support for operational optimization and fault prevention.

[0005] In the field of charging and swapping data, although data analysis methods based on data lineage have shown significant advantages in improving data processing and analysis efficiency, they may face some challenges when directly applied to data anomaly detection, especially when dealing with large and complex data sets generated by charging and swapping networks. These challenges are mainly reflected in the real-time and accuracy of anomaly detection, because data lineage mainly focuses on the source and flow of data, and is not necessarily optimized directly for the identification of abnormal patterns. This means that additional mechanisms may be needed to enhance the sensitivity and timeliness of anomaly detection in terms of quickly identifying sudden failures, abnormal charging behaviors, or equipment performance degradation during the charging and swapping process.

[0006] On the other hand, the anomaly detection method based on the deep combination of topological structure and attribute information does have advantages in capturing the complexity of charging and swapping networks and the characteristics of equipment attributes. However, this method may not fully take into account the uniqueness of the blood relationship of charging and swapping data, such as the flow of data between charging stations, users, and vehicles, and the business logic and rules behind these data flows. Therefore, directly applying this method to the anomaly inspection of charging and swapping data warehouses or business data may ignore some abnormal patterns caused by specific business scenarios or data blood relationships, thereby affecting the comprehensiveness and accuracy of anomaly detection.

[0007] To overcome these challenges, a more ideal solution is to combine data lineage analysis with anomaly detection methods based on topology and attribute information to form a comprehensive charging and swapping data anomaly detection framework. This framework should be able to fully utilize data lineage to track the source and propagation path of abnormal data, while using in-depth analysis of topology and attribute information to identify and verify potential abnormal patterns. This can not only ensure the real-time and accuracy of anomaly detection, but also comprehensively cover various abnormal situations in the charging and swapping network, providing operators with more accurate and efficient decision support. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a method for checking abnormal charging and swapping data by utilizing data lineage topological relationships. By integrating the visual presentation of data lineage maps, the construction of sophisticated data topological relationships, and efficient data anomaly detection and instant feedback mechanisms, the data development and management anomaly problems encountered in the production process of charging and swapping big data can be effectively solved.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for checking abnormal charging and swapping data using data kinship topological relationship, comprising the following steps: S1. Data collection: collect relevant data from various data sources in the charging and swapping system, organize them into data sets, and preprocess the data sets to obtain a preprocessed charging and swapping multi-dimensional data set; S2. Constructing a topological graph of the blood relationship between charging and swapping, establishing a topological graph of the blood relationship between each data item in the charging and swapping system, and classifying and marking the nodes in the topological graph of the data blood relationship; then, using a depth-first search algorithm to obtain key nodes, and analyzing the data blood relationship; finally, after completing the analysis, generating a topological graph of the blood relationship between charging and swapping; S3, construction of anomaly detection model group; S4, abnormal location and tracing, using the constructed data lineage topology map to trace the detected anomalies, clarify the source of abnormal data and its propagation path in the entire charging and swapping network; S5. Handle charging and swapping data anomalies and formulate a handling strategy based on the type, severity and impact scope of the anomaly.

[0010] Preferably, the data sources include a charging station control system, a user payment platform, and a vehicle information system.

[0011] Preferably, the step S1 specifically includes the following steps: S11. Collect charging and swapping data from different time periods, different locations, and different platforms, and organize them into data sets; S12. Manually identify the dimensions in the data set, and perform dimensionality reduction, remove duplicate values, and remove outliers to obtain the charging and swapping data table DA, the most recent fault location table DB, the area table DC, the protection unit table DD, the first olt data summary table 1DE, the second olt data summary table 2DF, the third olt data summary table 3DG, the fourth olt data summary table 4DH, and the data application table DI.

[0012] Preferably, step S2 comprises the following steps: S21. Use natural language processing and regular expressions to accurately identify the charging and swapping data lineage node set VDA, the nearest fault location lineage node set VDB, the regional lineage node set VDC, the protection unit lineage node set VDD, the first olt data summary lineage node set VDE, the second olt data summary lineage node set VDF, the third olt data summary lineage node set VDG, the fourth olt data summary lineage node set VDH, and the data application lineage node set VDI from SQL query statements, data flow diagrams, and ETL scripts. At the same time, identify the dependencies and transmission paths E between data items to provide basic information for the subsequent construction of data lineage relationships. S22. Use the charging and swapping data lineage node set VDA, the nearest fault location lineage node set VDB, the regional lineage node set VDC, and the protection unit lineage node set VDD as the starting node of the data lineage topology map, and the data application lineage node set VDI as the end node. Use the sets VDE, VDF, VDG, and VDH between the starting node and the end node as the backbone nodes of the data lineage topology map to construct the skeleton of the data lineage topology map; S23. Automatically generate a topology graph G based on the topology graph skeleton and in combination with the blood relationship between the sets.

[0013] Preferably, step S3 comprises the following steps: S31. Construct an abnormal classification model based on a data lineage topology map, collect multi-dimensional data in the charging and swapping system, and mark key nodes through the construction of a data lineage topology map to form a data lineage network; S32. Build a user behavior anomaly detection model based on user behavior analysis, establish real-time and batch anomaly detection models, the batch detection model extracts features from the topology map, and determines node anomalies through the random forest algorithm. The user behavior analysis uses the support vector machine SVM to identify abnormal charging behavior; S33, build a periodic anomaly detection model, and identify abnormal trends such as seasonal load changes through the periodic anomaly detection model STL decomposition and Prophet model; S34. Build a real-time anomaly detection model. The real-time detection model analyzes the charging data stream in real time based on a sliding window to locate sudden anomalies that cause equipment shutdown.

[0014] Preferably, the step S31 specifically includes: first, extracting each node and path, that is, the key features of the device description, from the constructed data kinship topology diagram, selecting, combining and generating derivative features for the features, adding specific labels, and constructing a charging and swapping data set containing normal samples and abnormal samples, and then; dividing the cleaned charging and swapping data set into a training set and a test set according to a certain ratio; importing the training set into the random forest model for learning; before the random forest model performs anomaly detection on the features of each node and path in the topological structure, the number of sample feature subset dimensions of each decision tree of the random forest model is set to , p is the total number of features; in the model training process, the candidate parameter set param_grid is set by cross-validation method, and the optimal number of decision trees M is determined by grid search; for each decision tree, the importance of node features is calculated respectively, and the abnormal probability is calculated according to the status of different paths; the output result of each decision tree is 0 or 1, indicating normal and abnormal respectively; the output of each decision tree is combined by majority voting method to obtain the final judgment result of the node; In terms of anomaly scoring and calculation, the prediction results of multiple decision trees are integrated to define the topological anomaly score S topology : ; Where M represents the number of trees in the random forest model, and Tm(x) is the classification result of the mth tree for sample x; Finally, the anomaly threshold Ttopology is set; when the topological anomaly score Stopology>Ttopology, the random forest model will mark the corresponding node as an anomaly and give the corresponding anomaly label.

[0015] Preferably, the step S32 specifically includes: first, collecting user behavior data, including charging time, frequency, charging pile status, and user historical behavior, cleaning and preprocessing the original user behavior data, extracting a feature vector composed of multiple key features from the user behavior data, and constructing a key feature matrix , where n is the number of samples and d is the feature dimension; Then, the matrix features are input into the nonlinear support vector machine SVM for training; by introducing the Gaussian kernel: ; in, K () represents the Gaussian kernel function, γ is the parameter that controls the kernel width, and an optimal hyperplane is found so that normal charging behavior and abnormal charging behavior can be effectively distinguished, and a classifier is obtained that can classify the input behavior data into "normal charging behavior" or "abnormal charging behavior"; After training is completed, the trained SVM model is used to classify new user behaviors; for each input sample x, the model will output a label , where -1 indicates abnormal charging behavior and +1 indicates normal charging behavior; by analyzing the classification results, the abnormal charging behavior is identified; When abnormal charging behavior is detected, the information of all nodes and edges currently related to the abnormal charging behavior is captured through the data lineage topology graph, and the data is input into the data lineage topology graph to further identify the specific cause of the abnormality.

[0016] Preferably, the step S33 specifically includes: first, preprocessing the charging and swapping data by processing missing values, removing obvious outliers, standardizing, and removing long-term trends to ensure data stability and reduce noise levels; then using STL decomposition to decompose the time series, and the decomposition formula is: ; in, Y t represents a time series, Represents the trend component, reflecting the long-term trend of charging and swapping demand over time; Represents the seasonal component, reflecting the fixed periodic changes in charging and swapping demand, The residual part represents other short-term fluctuations or noise that cannot be captured by trends and seasonality. After the STL decomposition is completed, the seasonal components are periodically modeled. According to the seasonal characteristics of charging and swapping demand, the Prophet model first separates the trend term from the seasonal term and fits the long-term trend changes of the data. Subsequently, the Prophet model generates sine and cosine basis functions through Fourier series to represent periodic changes, and uses the user-specified period to automatically adapt to the seasonal components of daily fluctuations and differences between weekends and weekdays to generate repeatable periodic functions. For special demand fluctuations caused by holidays, the Prophet model adds holidays as features through the holiday effect module and learns the unique impact of holidays on charging demand to adjust the expected value and capture changes in holiday demand. After the Prophet model training is completed, the Prophet model will generate future forecast data as a reference for the periodic normal value; then calculate the standardized residual: ; Among them, Residual t is the difference between the actual observation value and the Prophet model prediction value, μ is the residual mean, and σ is the standard deviation of the residual; set the threshold , for each time point t, when When the Prophet model enters the anomaly detection phase; it identifies moments that are abnormal based on cyclical trends and seasonal changes.

[0017] Preferably, the step S34 is specifically as follows: a real-time anomaly detection system is used to continuously monitor the charging and swapping data flow, and the acquired data is imported into an anomaly detection model based on the data lineage topology structure to obtain the current status of the device to ensure a rapid response, including sudden shutdown of the device and sudden anomalies of abnormal charging behavior.

[0018] Preferably, the method further includes step S6 of abnormal feedback, collecting feedback information during the abnormal detection process and transmitting it back.

[0019] The present invention provides a method for checking abnormal charging and swapping data using data kinship topology, which has the following beneficial effects: 1. Improve the efficiency of abnormal detection of charging and swapping data: The technical solution of the present invention achieves accurate tracking of data flow during the charging and swapping process by constructing a topological relationship of charging and swapping data. During anomaly detection, this topological relationship can be used to quickly locate potential problem areas, significantly reducing the time and cost of checking massive charging and swapping data one by one. This not only speeds up the response to anomalies, but also improves overall operational efficiency.

[0020] 2. Improve the accuracy of abnormal detection of charging and swapping data: The technical solution of the present invention innovatively combines the topological structure and specific attribute information of charging and swapping data (such as charging amount, charging time, device status, etc.) for anomaly detection. This multi-dimensional and in-depth analysis method can more accurately identify abnormal values ​​and abnormal patterns in the data, whether it is the abnormal behavior of a single charging station or potential problems in the entire charging and swapping network, which can be captured in time and accurately analyzed.

[0021] 3. Enhance the stability and reliability of charging and swapping systems: The technical solution of the present invention enables data anomalies in the charging and swapping system to be discovered in a timely manner and effectively handled. This avoids the propagation and accumulation of abnormal data within the system, thereby reducing business interruptions or decision-making errors caused by data errors. At the same time, the rapid response and processing of abnormal data also improves the system's self-repair capabilities and enhances the system's stability and reliability. Against the background of the rapid development of the charging and swapping business, this advantage is of great significance for ensuring user experience and maintaining brand image. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0023] like Figure 1 As shown, a method for checking abnormal charging and swapping data using data lineage topology relationship includes the following steps: S1. Data collection: collect relevant data from various data sources in the charging and swapping system, organize them into data sets, and preprocess the data sets to obtain a preprocessed charging and swapping multi-dimensional data set; S2. Constructing a topological graph of the blood relationship between charging and swapping, establishing a topological graph of the blood relationship between each data item in the charging and swapping system, and classifying and marking the nodes in the topological graph of the data blood relationship; then, using a depth-first search algorithm to obtain key nodes, and analyzing the data blood relationship; finally, after completing the analysis, generating a topological graph of the blood relationship between charging and swapping; S3, construction of anomaly detection model group; S4, abnormal location and tracing, using the constructed data lineage topology map to trace the detected anomalies, clarify the source of abnormal data and its propagation path in the entire charging and swapping network; S5. Handle charging and swapping data anomalies and formulate a handling strategy based on the type, severity and impact scope of the anomaly.

[0024] Preferably, the data sources include a charging station control system, a user payment platform, and a vehicle information system.

[0025] Preferably, the step S1 specifically includes the following steps: S11. Collect charging and swapping data from different time periods, different locations, and different platforms, and organize them into data sets; S12. Manually identify the dimensions in the data set, and perform dimensionality reduction, remove duplicate values, and remove outliers to obtain the charging and swapping data table DA, the most recent fault location table DB, the area table DC, the protection unit table DD, the first olt data summary table 1DE, the second olt data summary table 2DF, the third olt data summary table 3DG, the fourth olt data summary table 4DH, and the data application table DI.

[0026] Preferably, step S2 comprises the following steps: S21. Use natural language processing and regular expressions to accurately identify the charging and swapping data lineage node set VDA=(VA1, VA2,...VAa)from SQL query statements, data flow diagrams and ETL scripts, where a represents the number of lineage nodes in the set; the nearest fault location lineage node set VDB=(VB1, VB2,...VBb), where b represents the number of lineage nodes in the set; the regional lineage node set VDC=(VC1, VC2,...VCc), where c represents the number of lineage nodes in the set; the protection unit lineage node set VDD=(VD1, VD2,...VDd), where d represents the number of lineage nodes in the set; the first olt data summary lineage node set VDE=(VE1, VE2,...VEe), where e represents the lineage node in the set The number of points; the second olt data summary lineage node set VDF=(VF1, VF2, ... VFf), where f represents the number of lineage nodes in the set; the third olt data summary lineage node set VDG=(VG1, VG2, ... VGg), where g represents the number of lineage nodes in the set; the fourth olt data summary lineage node set VDH=(VH1, VH2, ... VHh), where h represents the number of lineage nodes in the set; the data application lineage node set VDI=(VI1, VI2, ... VIi), where i represents the number of lineage nodes in the set; at the same time, identify the dependency relationship and transmission path E=(E1, E2, ... Ej) between data items, where j represents the number of dependency relationships and transmission paths, providing basic information for the subsequent data lineage relationship construction; S22. Use the charging and swapping data lineage node set VDA, the nearest fault location lineage node set VDB, the regional lineage node set VDC, and the protection unit lineage node set VDD as the starting node of the data lineage topology map, and the data application lineage node set VDI as the end node. Use the sets VDE, VDF, VDG, and VDH between the starting node and the end node as the backbone nodes of the data lineage topology map to construct the skeleton of the data lineage topology map; S23. Automatically generate a topology graph G based on the topology graph skeleton and in combination with the blood relationship between the sets.

[0027] Preferably, step S3 comprises the following steps: S31. Construct an abnormal classification model based on a data lineage topology map, collect multi-dimensional data in the charging and swapping system, and mark key nodes through the construction of a data lineage topology map to form a data lineage network; S32. Build a user behavior anomaly detection model based on user behavior analysis, establish real-time and batch anomaly detection models, the batch detection model extracts features from the topology map, and determines node anomalies through the random forest algorithm. The user behavior analysis uses the support vector machine SVM to identify abnormal charging behavior; S33, build a periodic anomaly detection model, and identify abnormal trends such as seasonal load changes through the periodic anomaly detection model STL decomposition and Prophet model; S34. Build a real-time anomaly detection model. The real-time detection model analyzes the charging data stream in real time based on a sliding window to locate sudden anomalies that cause equipment shutdown.

[0028] STL decomposition (Seasonal-Trend decomposition using LOESS): STL decomposition is a time series decomposition technique that uses the locally weighted regression (LOESS) method to separate the time series into trend components, seasonal components, and residual components. STL decomposition is particularly suitable for time series data with complex seasonal and trend changes.

[0029] Prophet Model: is a statistical model for time series forecasting, designed to capture trends, seasonality, and holiday effects in time series. The model uses Fourier series to generate periodic changes and automatically adjusts seasonal components according to user-specified periods (such as years, months, weeks), which is suitable for handling holiday effects and regular seasonal fluctuations.

[0030] Anomaly Detection Model: A model used to identify and classify data that does not conform to expected patterns. In the charging and swapping system, the anomaly detection model can identify possible equipment failures or abnormal charging behaviors by analyzing the characteristics and behaviors of the data.

[0031] Seasonal Anomaly Detection: It is a method based on time series analysis to identify abnormal data that does not conform to seasonal trends and periodic changes. It models the seasonal components and trends of the data to identify abnormal patterns that appear on the basis of normal periodic changes. It is particularly suitable for identifying problems such as abnormal load caused by seasonal demand changes.

[0032] Preferably, the step S31 specifically includes: first, extracting each node and path from the constructed data kinship topology diagram, that is, key features of device description, such as node attributes (abnormal fluctuation of charging amount, surge in equipment failure rate, distribution of charging time, power consumption trend, etc.), edge attributes (data transmission time, data transmission success rate, etc.), selecting, combining and generating derivative features for the features, adding specific labels, and constructing a charging and swapping data set containing normal samples and abnormal samples, and then; dividing the cleaned charging and swapping data set into a training set and a test set according to a certain ratio; importing the training set into a random forest model for learning; before the random forest model performs anomaly detection on the features of each node and path in the topological structure, the number of sample feature subset dimensions of each decision tree of the random forest model is set to , p is the total number of features; in the model training process, the candidate parameter set param_grid = {'n_estimators':[10, 50, 100, 200, 500]} is set through cross-validation method, and the optimal number of decision trees M = grid_search.best_params_['n_estimators'] is determined by grid search; for each decision tree, the importance of node features is calculated respectively, and the abnormal probability is calculated according to the status of different paths; the output result of each decision tree is 0 or 1, indicating normal and abnormal respectively; the output of each decision tree is combined by majority voting method to obtain the final judgment result of the node; In terms of anomaly scoring and calculation, the prediction results of multiple decision trees are integrated to define the topological anomaly score S topology : ; Where M represents the number of trees in the random forest model, and Tm(x) is the classification result of the mth tree for sample x; Finally, the anomaly threshold Ttopology=Percentile95(Stopology,normal) is set; when the topological anomaly score Stopology>Ttopology, the random forest model will mark the corresponding node as abnormal and give the corresponding abnormal label.

[0033] Preferably, the step S32 specifically includes: first, collecting user behavior data, including charging time, frequency, charging pile status, and user historical behavior, cleaning and preprocessing the original user behavior data, extracting a feature vector composed of multiple key features from the user behavior data, and constructing a key feature matrix , where n is the number of samples and d is the feature dimension; Then, the matrix features are input into the nonlinear support vector machine SVM for training; by introducing the Gaussian kernel: ; in, K () represents the Gaussian kernel function, γ is the parameter that controls the kernel width, and an optimal hyperplane is found so that normal charging behavior and abnormal charging behavior can be effectively distinguished, and a classifier is obtained that can classify the input behavior data into "normal charging behavior" or "abnormal charging behavior"; After training is completed, the trained SVM model is used to classify new user behaviors; for each input sample x, the model will output a label , where -1 indicates abnormal charging behavior and +1 indicates normal charging behavior; by analyzing the classification results, the abnormal charging behavior is identified; When abnormal charging behavior is detected, the information of all nodes and edges currently related to the abnormal charging behavior is captured through the data lineage topology graph, and the data is input into the data lineage topology graph to further identify the specific cause of the abnormality.

[0034] Preferably, the step S33 specifically includes: first, preprocessing the charging and swapping data by processing missing values, removing obvious outliers, standardizing, and removing long-term trends to ensure data stability and reduce noise levels; then using STL decomposition to decompose the time series, and the decomposition formula is: ; in, Y t represents a time series, Represents the trend component, reflecting the long-term trend of charging and swapping demand over time; Represents the seasonal component, reflecting the fixed periodic changes in charging and swapping demand, The residual part represents other short-term fluctuations or noise that cannot be captured by trends and seasonality. After the STL decomposition is completed, the seasonal components are periodically modeled. According to the seasonal characteristics of charging and swapping demand, the Prophet model first separates the trend term from the seasonal term and fits the long-term trend changes of the data. Subsequently, the Prophet model generates sine and cosine basis functions through Fourier series to represent periodic changes, and uses the user-specified period (such as year, month, week) to automatically adapt to the seasonal components of daily fluctuations and differences between weekends and weekdays to generate repeatable periodic functions. For special demand fluctuations caused by holidays, the Prophet model adds holidays as features through the Holiday Effect module and learns the unique impact of holidays on charging demand to adjust the expected value and capture changes in holiday demand. After the Prophet model training is completed, the Prophet model will generate future forecast data (including synthetic values ​​of trend and seasonal changes) as a reference for the cyclical normal value; then calculate the standardized residual: ; Among them, Residual t is the difference between the actual observation value and the Prophet model prediction value, μ is the residual mean, and σ is the standard deviation of the residual; set the threshold , for each time point t, when When the Prophet model enters the anomaly detection phase; it identifies moments that are abnormal based on cyclical trends and seasonal changes.

[0035] Preferably, step S34 is as follows: adopt a real-time anomaly detection system to continuously monitor the charging and swapping data stream, and import the acquired data into an anomaly detection model based on the data lineage topology structure to obtain the current status of the equipment, and ensure rapid response, including sudden anomalies such as sudden equipment shutdown and abnormal charging behavior. Feedback the results of anomaly detection and processing to the data management department and business operation team of the charging and swapping system in a timely manner to help optimize the data processing process and improve data quality and service level. Continuously optimize anomaly detection algorithms and strategies to adapt to the ever-changing development of charging and swapping business.

[0036] Preferably, the method further includes step S6 of abnormal feedback, collecting feedback information during the abnormal detection process and transmitting it back.

[0037] The present invention provides a method for checking abnormal charging and swapping data using data lineage topological relationships, which relates to the field of data processing technology. By collecting multi-dimensional data from the charging and swapping system, and constructing a data lineage topological map, key nodes are marked to form a data lineage network. A batch anomaly detection model based on random forests, a user behavior anomaly detection model based on support vector machines, and a periodic anomaly detection model based on STL decomposition and Prophet models are constructed to identify abnormal behaviors, equipment failures, and seasonal load changes in charging and swapping. At the same time, a real-time anomaly detection system is designed to analyze the charging data stream in real time based on a sliding window, and quickly respond to sudden anomalies such as equipment shutdowns. After the anomaly is detected, the topological map is used to trace the source, analyze the propagation path and impact of the abnormal data, and implement processing for different anomaly types, timely feedback and optimize the system, and improve the stability of the charging and swapping system and data processing efficiency.

[0038] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limiting the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. A method for checking abnormal charging and swapping data using data lineage topological relationships, characterized in that: The following steps are involved: S1. Data collection: collect relevant data from various data sources in the charging and swapping system, organize them into data sets, and preprocess the data sets to obtain a preprocessed charging and swapping multi-dimensional data set; S2. Constructing a topological graph of the blood relationship between charging and swapping, establishing a topological graph of the blood relationship between each data item in the charging and swapping system, and classifying and marking the nodes in the topological graph of the data blood relationship; then, using a depth-first search algorithm to obtain key nodes, and analyzing the data blood relationship; finally, after completing the analysis, generating a topological graph of the blood relationship between charging and swapping; S3, construction of anomaly detection model group; S4, abnormal location and tracing, using the constructed data lineage topology map to trace the detected anomalies, clarify the source of abnormal data and its propagation path in the entire charging and swapping network; S5. Handle charging and swapping data anomalies and formulate a handling strategy based on the type, severity and impact scope of the anomaly.

2. According to claim 1, a method for checking abnormal charging and swapping data using data lineage topology relationship is characterized in that: The data sources include a charging station control system, a user payment platform, and a vehicle information system.

3. A method for checking abnormal charging and swapping data using data lineage topology according to claim 1 or 2, characterized in that: The step S1 specifically includes the following steps: S11. Collect charging and swapping data from different time periods, different locations, and different platforms, and organize them into data sets; S12. Manually identify the dimensions in the data set, and perform dimensionality reduction, remove duplicate values, and remove outliers to obtain the charging and swapping data table DA, the most recent fault location table DB, the area table DC, the protection unit table DD, the first olt data summary table 1DE, the second olt data summary table 2DF, the third olt data summary table 3DG, the fourth olt data summary table 4DH, and the data application table DI.

4. According to claim 3, a method for checking abnormal charging and swapping data using data lineage topology relationship is characterized in that: The step S2 comprises the following steps: S21. Use natural language processing and regular expressions to accurately identify the charging and swapping data lineage node set VDA, the nearest fault location lineage node set VDB, the regional lineage node set VDC, the protection unit lineage node set VDD, the first olt data summary lineage node set VDE, the second olt data summary lineage node set VDF, the third olt data summary lineage node set VDG, the fourth olt data summary lineage node set VDH, and the data application lineage node set VDI from SQL query statements, data flow diagrams, and ETL scripts. At the same time, identify the dependencies and transmission paths E between data items to provide basic information for the subsequent construction of data lineage relationships. S22. Use the charging and swapping data lineage node set VDA, the nearest fault location lineage node set VDB, the regional lineage node set VDC, and the protection unit lineage node set VDD as the starting node of the data lineage topology map, and the data application lineage node set VDI as the end node. Use the sets VDE, VDF, VDG, and VDH between the starting node and the end node as the backbone nodes of the data lineage topology map to construct the skeleton of the data lineage topology map; S23. Automatically generate a topology graph G based on the topology graph skeleton and in combination with the blood relationship between the sets.

5. According to claim 1, a method for checking abnormal charging and swapping data using data lineage topology relationship is characterized in that: The step S3 comprises the following steps: S31. Construct an abnormal classification model based on a data lineage topology map, collect multi-dimensional data in the charging and swapping system, and mark key nodes through the construction of a data lineage topology map to form a data lineage network; S32. Build a user behavior anomaly detection model based on user behavior analysis, establish real-time and batch anomaly detection models, the batch detection model extracts features from the topology map, and determines node anomalies through the random forest algorithm. The user behavior analysis uses the support vector machine SVM to identify abnormal charging behavior; S33, build a periodic anomaly detection model, and identify abnormal trends such as seasonal load changes through the periodic anomaly detection model STL decomposition and Prophet model; S34. Build a real-time anomaly detection model. The real-time detection model analyzes the charging data stream in real time based on a sliding window to locate sudden anomalies that cause equipment shutdown.

6. According to claim 5, a method for checking abnormal charging and swapping data using data lineage topology relationship is characterized in that: The step S31 specifically includes: first, extracting each node and path, i.e., the key features of the device description, from the constructed data kinship topology diagram, selecting, combining and generating derivative features for the features, adding specific labels, and constructing a charging and swapping data set containing normal samples and abnormal samples, and then; dividing the cleaned charging and swapping data set into a training set and a test set according to a certain ratio; importing the training set into a random forest model for learning; before the random forest model performs anomaly detection on the features of each node and path in the topological structure, the number of sample feature subset dimensions of each decision tree of the random forest model is set to , p is the total number of features; in the model training process, the candidate parameter set param_grid is set by cross-validation method, and the optimal number of decision trees M is determined by grid search; for each decision tree, the importance of node features is calculated respectively, and the abnormal probability is calculated according to the status of different paths; the output result of each decision tree is 0 or 1, indicating normal and abnormal respectively; the output of each decision tree is combined by majority voting method to obtain the final judgment result of the node; In terms of anomaly scoring and calculation, the prediction results of multiple decision trees are integrated to define the topological anomaly score S topology : ； Where M represents the number of trees in the random forest model, and Tm(x) is the classification result of the mth tree for sample x; Finally, the anomaly threshold Ttopology is set; when the topological anomaly score Stopology>Ttopology, the random forest model will mark the corresponding node as an anomaly and give the corresponding anomaly label.

7. According to claim 5, a method for checking abnormal charging and swapping data using data lineage topology relationship is characterized in that: The step S32 specifically includes: first, collecting user behavior data, including charging time, frequency, charging pile status, and user historical behavior, cleaning and preprocessing the original user behavior data, extracting a feature vector consisting of multiple key features from the user behavior data, and constructing a key feature matrix , where n is the number of samples and d is the feature dimension; Then, the matrix features are input into the nonlinear support vector machine SVM for training; by introducing the Gaussian kernel: ； in, K () represents the Gaussian kernel function, γ is the parameter that controls the kernel width, and an optimal hyperplane is found so that normal charging behavior and abnormal charging behavior can be effectively distinguished, and a classifier is obtained that can classify the input behavior data into "normal charging behavior" or "abnormal charging behavior"; After training is completed, the trained SVM model is used to classify new user behaviors; for each input sample x, the model will output a label , where -1 indicates abnormal charging behavior and +1 indicates normal charging behavior; by analyzing the classification results, the abnormal charging behavior is identified; When abnormal charging behavior is detected, the information of all nodes and edges currently related to the abnormal charging behavior is captured through the data lineage topology map, and the data is input into the data lineage topology map to further identify the specific cause of the abnormality.

8. According to claim 5, a method for checking abnormal charging and swapping data using data lineage topology relationship is characterized in that: The step S33 specifically includes: first, pre-processing the charging and swapping data by processing missing values, removing obvious outliers, standardizing, and removing long-term trends to ensure data stability and reduce noise levels; then using STL decomposition to decompose the time series, and the decomposition formula is: ； in, Y t represents a time series, Represents the trend component, reflecting the long-term trend of charging and swapping demand over time; Represents the seasonal component, reflecting the fixed periodic changes in charging and swapping demand, The residual part represents other short-term fluctuations or noise that cannot be captured by trends and seasonality. After the STL decomposition is completed, the seasonal components are periodically modeled. According to the seasonal characteristics of charging and swapping demand, the Prophet model first separates the trend term from the seasonal term and fits the long-term trend changes of the data. Subsequently, the Prophet model generates sine and cosine basis functions through Fourier series to represent periodic changes, and uses the user-specified period to automatically adapt to the seasonal components of daily fluctuations and differences between weekends and weekdays to generate repeatable periodic functions. For special demand fluctuations caused by holidays, the Prophet model adds holidays as features through the holiday effect module and learns the unique impact of holidays on charging demand to adjust the expected value and capture changes in holiday demand. After the Prophet model training is completed, the Prophet model will generate future forecast data as a reference for the periodic normal value; then calculate the standardized residual: ； Among them, Residual t is the difference between the actual observation value and the Prophet model prediction value, μ is the residual mean, and σ is the standard deviation of the residual; set the threshold , for each time point t, when When the Prophet model enters the anomaly detection phase; it identifies moments that are abnormal based on cyclical trends and seasonal changes.

9. According to claim 5, a method for checking abnormal charging and swapping data using data lineage topology relationship is characterized in that: The specific details of step S34 are as follows: use a real-time anomaly detection system to continuously monitor the charging and swapping data flow, and import the acquired data into an anomaly detection model based on the data lineage topology structure to obtain the current status of the device to ensure a rapid response, including sudden anomalies such as sudden shutdown of the equipment and abnormal charging behavior.

10. According to claim 1, a method for checking abnormal charging and swapping data using data lineage topology relationship, characterized in that: The process also includes step S6 of abnormal feedback, which collects feedback information during the abnormal detection process and transmits it back.

Images