New energy data anomaly detection method and related equipment
Through the SpectralNet-NAS-TFT method, the problems of nonlinear feature mining and abnormal diagnosis in new energy data processing are solved, efficient data acquisition and abnormal detection are achieved, and data processing capabilities of smart grids are improved.
Patent Information
- Application Number
- CN202510485332.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-17
- Publication Date
- 2025-07-18
AI Technical Summary
Existing new energy data processing methods are difficult to effectively mine nonlinear features, are low in robustness, cannot fully identify and locate data channel interruption abnormalities, traditional models have poor generalization capabilities, are difficult to adapt to changes in complex scenarios, and are inefficient in data acquisition and processing.
Nonlinear feature extraction and weighted aggregation is used for nonlinear feature extraction and weighted aggregation, combined with NAS and TFT data channel abnormality diagnosis models for detection and traceability, and the new energy data comprehensive optimization model is used to optimize the system operating parameters.
It improves the quality of new energy data acquisition and processing, accurately identify channel interrupt abnormalities, improves abnormal detection and prediction accuracy, optimizes data acquisition efficiency, reduces redundancy, and provides strong data support for the operation of smart grids.
Smart Images
Figure CN120337087A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data detection, and more specifically, to a new energy data anomaly detection method and related devices. Background Art
[0002] In the new energy field, data mainly comes from sensors, monitoring devices, etc. However, factors such as measurement errors, transmission noises, and equipment failures often make these data abnormal. If these abnormal data are not detected and processed, they will have a negative impact on the analysis results, and may also lead to data loss, discontinuity, or deviation, destroying the coherence and reliability of the data.
[0003] Current implementation solutions have many limitations in aspects such as data acquisition and processing, data channel interruption anomaly diagnosis, and data optimization models. In terms of data acquisition and processing, new energy data is high-dimensional and complex. Existing methods are difficult to extract non-linear features, have low robustness to noise and abnormal data, cannot fully integrate multi-modal data, and have insufficient real-time acquisition and processing capabilities. In data channel interruption anomaly diagnosis, it is difficult to comprehensively identify and locate complex anomaly types. Traditional models have poor generalization ability, rely on manually designed features and network structures, are inefficient, and are difficult to adapt to scene changes. In terms of data optimization models, they mostly focus on single objectives, cannot handle complex constraints in new energy scenarios, and are difficult to achieve real-time optimization of dynamic characteristics.
[0004] Based on this, this application provides a more comprehensive new energy data anomaly detection solution, effectively making up for the deficiencies of the existing technology and realizing the detection and location of channel anomalies. Summary of the Invention
[0005] In view of this, this application provides a new energy data anomaly detection method and related devices, which optimize the new energy data acquisition efficiency, reduce the redundancy of data acquisition and processing, improve the processing speed and storage efficiency, and improve the detection and prediction accuracy of data anomalies, providing data support for the operation of the smart grid.
[0006] A new energy data anomaly detection method includes:
[0007] Using a data processing model based on SpectralNet to perform non-linear feature extraction and weighted aggregation processing on the obtained real-time device data to generate new energy aggregated data;
[0008] Inputting the new energy aggregated data into a data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and tracing to obtain an anomaly detection result and a tracing trigger chain;
[0009] Using the new energy data comprehensive optimization model, based on the traceability trigger chain, under the constraints of data transmission, data cleaning, anomaly detection accuracy, and anomaly traceability time, the system operation parameters are optimized with the goal of minimizing the total system cost.
[0010] Optionally, the data processing model includes a data acquisition network, a data communication network, a data cleaning network, and a data flow processing network;
[0011] The data acquisition network obtains device real-time data from multiple devices through a data acquisition system;
[0012] The data communication network transmits the device real-time data through a containerized communication module at an effective data transmission rate calculated based on the single-channel throughput and the data packet loss rate, obtaining transmission data;
[0013] The data cleaning network cleans the transmission data for outliers and noise signals, obtaining cleaned data;
[0014] The data flow processing network uses the SpectralNet algorithm to cluster and extract features from the cleaned data, generating a non-linear feature relationship function, and performing weighted aggregation on the extracted feature results to generate new energy aggregated data.
[0015] Optionally, the generation method of the transmission data is:
[0016]
[0017]
[0018] The generation method of the cleaned data is:
[0019]
[0020] The non-linear feature relationship function is:
[0021]
[0022] The generation method of the new energy aggregated data is:
[0023]
[0024] Among them, is the transmission data, is the effective data transmission rate, is the set of device real-time data at time t collected, is the single-channel throughput, is the data packet loss rate, is the cleaned data, is an outlier, is a noise signal, is a feature extraction function based on SpectralNet, is the k-th feature result extracted by the SpectralNet algorithm, is new energy aggregated data, is the weight of the k-th feature.
[0025] Optionally, the data channel anomaly diagnosis model includes an anomaly detection network, a root cause analysis network, and an anomaly traceability network;
[0026] The anomaly detection network is based on the new energy aggregated data and uses the NAS algorithm to detect the anomaly state and obtain the anomaly detection result;
[0027] The root cause analysis network constructs a root cause set based on the anomaly detection result and the device dependency matrix;
[0028] The anomaly traceability network combines the root cause set and the new energy aggregated data and uses the TFT algorithm to perform time series traceability analysis to generate a traceability trigger chain.
[0029] Optionally, the generation method of the anomaly detection result is:
[0030]
[0031] The construction method of the root cause set is:
[0032]
[0033] The generation method of the traceability trigger chain is:
[0034]
[0035] where is the anomaly detection result, is an anomaly state detection function based on NAS, is the new energy aggregated data, is the root cause set, is a function for solving the corresponding parameter when taking the maximum value, is the dependency matrix between device i and device j, is the traceability trigger chain, is a data channel anomaly traceability function based on TFT, where t is the time.
[0036] Optionally, the data transmission constraint is:
[0037]
[0038]
[0039] The data cleaning constraint is as follows:
[0040]
[0041] The anomaly detection accuracy constraint is as follows:
[0042]
[0043] The anomaly traceability time constraint is as follows:
[0044]
[0045] Among them, the effective transmission rate should meet the minimum requirement , is the throughput, is the packet loss rate, is the maximum network bandwidth, and the proportion of effective data should not be lower than the preset minimum threshold , is the transmitted data, is the cleaned data, and the detection accuracy and recall rate of the anomaly detection network should meet the accuracy precision and recall rate precision requirements. The time for the anomaly traceability network to analyze and generate a traceability trigger chain should be lower than the maximum diagnostic delay allowed by the system
[0046] A new energy data anomaly detection device, comprising:
[0047] A data acquisition unit, configured to perform non - linear feature extraction and weighted aggregation processing on the acquired real - time device data by using a data processing model based on SpectralNet to generate new energy aggregated data;
[0048] A data detection unit, configured to input the new energy aggregated data into a data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and traceability, and obtain an anomaly detection result and a traceability trigger chain;
[0049] A comprehensive optimization unit, configured to use a new energy data comprehensive optimization model, and based on the traceability trigger chain, optimize the system operation parameters with the goal of minimizing the total system cost under data transmission constraints, data cleaning constraints, anomaly detection accuracy constraints, and anomaly traceability time constraints.
[0050] A new energy data anomaly detection device, comprising a memory and a processor;
[0051] The memory is used to store programs;
[0052] The processor is used to execute the program to implement each step of the new energy data anomaly detection method described in any one of the above.
[0053] A readable storage medium, on which a computer program is stored, characterized in that when the computer program is executed by a processor, each step of the new energy data anomaly detection method described in any one of the above is implemented.
[0054] A computer program product, comprising a computer program, characterized in that when the computer program is run by a processor, each step of the new energy data anomaly detection method described in any one of the above is executed.
[0055] As can be seen from the above technical solutions, a new energy data anomaly detection method and related devices provided by embodiments of the present application first perform non - linear feature extraction and weighted aggregation processing on the obtained real - time device data by using a data processing model based on SpectralNet to generate new energy aggregated data. Then, the new energy aggregated data is input into a data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and traceability, obtaining an anomaly detection result and a traceability trigger chain. Finally, a new energy data comprehensive optimization model can also be used to optimize the system operation parameters with the goal of minimizing the total system cost based on the traceability trigger chain under data transmission constraints, data cleaning constraints, anomaly detection accuracy constraints, and anomaly traceability time constraints.
[0056] The method of the present application based on SpectralNet - NAS - TFT effectively overcomes the deficiencies of the prior art. By using the non - linear dimensionality reduction ability of SpectralNet, it can deeply mine the deep features of new energy data, improve the quality of data collection and processing, and solve the problem that it is difficult to mine non - linear features when dealing with high - dimensional complex data in existing methods. By automatically designing an anomaly diagnosis model adapted to new energy data through NAS, the adaptive ability of the model is improved, getting rid of the dependence on manual design and better adapting to complex scenario changes. Combining the time - series characteristics of TFT to process new energy data, it can accurately identify and predict channel interruption anomalies, comprehensively solving the problems that it is difficult to identify and locate complex anomaly types and the poor generalization ability in anomaly diagnosis of existing methods. In addition, this method can also optimize the data collection efficiency, reduce redundancy, improve the accuracy of anomaly detection and prediction, and provide strong data support for the operation of the smart grid. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.
[0058] Figure 1 Flow chart of a new energy data anomaly detection method disclosed in an embodiment of the present application;
[0059] Figure 2 Schematic diagram of a new energy data anomaly detection device disclosed in an embodiment of the present application;
[0060] Figure 3 Hardware structure block diagram of a new energy data anomaly detection device disclosed in an embodiment of the present application. Detailed implementation manners
[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.
[0062] The present application can be used in many general or special computing device environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, and so on.
[0063] Next, the solution of the present application will be introduced. The present application proposes the following technical solution. For details, please refer to the following text.
[0064] Figure 1 Flow chart of a new energy data anomaly detection method disclosed in an embodiment of the present application.
[0065] As Figure 1 shown, the method may include:
[0066] Step S1: Use the data processing model based on SpectralNet to perform non-linear feature extraction and weighted aggregation processing on the acquired real-time device data to generate new energy aggregated data.
[0067] Specifically, in the new energy field, a large amount of data generated by devices in real time contains rich but complex information. To more effectively process and utilize this data, first, a data processing model based on SpectralNet is required.
[0068] Device real-time data acquisition: New energy devices (such as wind turbines, solar panels, etc.) continuously generate various real-time data during operation. These data cover the operating status of the devices (such as rotational speed, temperature, voltage, current, etc.), environmental parameters (wind speed, light intensity, temperature, humidity, etc.), and energy output-related data (power generation, power, etc.). These data are collected through various sensors and transmitted to the data processing center, providing the raw data basis for subsequent analysis and processing.
[0069] Nonlinear feature extraction: SpectralNet is constructed based on relevant theories and technologies such as spectral analysis. Its unique network structure can handle data with complex nonlinear relationships. When facing device real-time data, traditional linear methods often struggle to uncover the deep features hidden in the data. SpectralNet uses its nonlinear transformation ability to map the original high-dimensional device real-time data to a new feature space. In this new space, the internal structure and relationships of the data can be presented more clearly.
[0070] For example, regarding the relationship between wind speed and power generation of a wind turbine, due to the influence of various complex factors such as aerodynamics, it is not a simple linear relationship. SpectralNet can extract features that can accurately reflect their complex nonlinear relationship through comprehensive analysis of multiple relevant parameters such as wind speed, wind direction, and blade angle, thus more accurately describing the wind power generation process.
[0071] Weighted aggregation processing: After extracting the nonlinear features, the importance of each feature for describing the operating status and energy production of new energy devices is different. SpectralNet assigns a weight to each feature according to its importance or relevance.
[0072] For example, in a solar power generation system, the influence degrees of light intensity and temperature on power generation may be different. Light intensity may be a more critical factor. Therefore, during the weighted aggregation processing, higher weights will be assigned to the features related to light intensity, while relatively lower weights will be assigned to other features such as temperature.
[0073] Through weighted aggregation, each feature is combined according to its weight to generate new energy aggregated data. This aggregated data can more comprehensively reflect the overall operating conditions and energy output status of new energy devices, providing more valuable inputs for subsequent analysis and decision-making.
[0074] Finally, after the nonlinear feature extraction and weighted aggregation processing of the data processing model based on SpectralNet, the obtained new energy aggregated data can be more effectively used for subsequent tasks such as anomaly detection and performance evaluation, improving the management and operation efficiency of the new energy system.
[0075] Step S2: Input the new energy aggregated data into the data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and tracing, and obtain the anomaly detection result and the tracing trigger chain.
[0076] Specifically, step S2 is to input the new energy aggregated data into the data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and tracing, so as to obtain the anomaly detection result and the tracing trigger chain. This step is based on the unique technical principles of NAS (Neural Architecture Search) and TFT (Temporal Fusion Transformer). NAS can automatically search for the optimal neural network architecture. It uses reinforcement learning, evolutionary algorithms, etc. in the predefined architecture search space, and continuously adjusts the architecture according to the performance of the model on the validation set, avoiding the complexity and time-consuming of manual design; TFT, relying on the self-attention mechanism and gated recurrent units, can effectively capture the long-term and short-term dependencies in time series data such as new energy data.
[0077] In terms of the operation process, first input the new energy aggregated data after non-linear feature extraction and weighted aggregation into the model. In the anomaly detection stage, NAS first automatically searches for the optimal neural network structure, continuously tries different architectures in the search space, trains the model with the training set and evaluates the performance on the validation set, and iteratively finds the optimal architecture; then uses this architecture to train the anomaly detection model with the training set so that it learns the normal data pattern; finally, input the test set into the model, and judge whether there is an anomaly according to the learned normal pattern and output the detection result. In the anomaly tracing stage, combine the device dependency matrix and root cause analysis to generate a set of possible anomaly causes, and then use TFT to perform time series analysis on the new energy aggregated data, and combine the set of anomaly causes to predict the trigger chain of the anomaly event.
[0078] The diagnostic model constructed in this step has significant application effects. It can accurately detect and trace the interruption anomaly of the new energy data channel, quickly and accurately find the anomaly and locate the root cause; by timely discovering and solving the data channel anomaly problem, it improves the reliability and stability of the new energy data; provides detailed anomaly detection results and tracing trigger chains for the operation and maintenance personnel of the new energy system, provides strong decision-making support for system recovery and optimization, helps to improve the system design and operation strategy, and reduces the occurrence of future anomaly events.
[0079] Step S3: Use the new energy data comprehensive optimization model, based on the tracing trigger chain, optimize the system operation parameters with the goal of minimizing the total system cost under the constraints of data transmission, data cleaning, anomaly detection accuracy, and anomaly tracing time.
[0080] Specifically, step S3 focuses on using the new energy data comprehensive optimization model. Based on the traceability trigger chain obtained in step S2, under a series of constraints, it aims to optimize the system operation parameters with the goal of minimizing the total system cost. This step aims to propose a new energy data comprehensive optimization model considering multiple constraints to minimize the total cost of the new energy data collection and diagnosis system while ensuring system performance and stability.
[0081] This comprehensive optimization model sets the minimization of the total system cost as the core goal. The total system cost is affected by various factors such as throughput, latency, packet loss rate, and the abnormal traceability chain. In actual operation, the magnitude of throughput, data transmission latency, packet loss situation, and the complexity of the abnormal traceability chain will directly or indirectly increase the system cost. To ensure the normal operation of the system while reducing costs, the model sets multiple constraints.
[0082] The data transmission constraint requires that the effective transmission rate is not lower than the minimum requirement, which can ensure the quality and efficiency of data during transmission, avoid data loss or errors caused by transmission problems, and thus reduce the cost and difficulty of subsequent processing. The data cleaning constraint ensures that the proportion of valid data meets the standard. Only by ensuring the proportion of valid data can reliable basis be provided for subsequent analysis and decision-making, and prevent waste of computing resources and time due to excessive invalid data. The abnormal detection accuracy constraint puts forward requirements for the accuracy rate and recall rate of the model to ensure that abnormal situations can be accurately detected and avoid potential losses caused by missed detection or false detection. The abnormal traceability time constraint limits the maximum time delay of traceability calculation, enabling the system to quickly locate the root cause of the abnormality, take timely measures for repair, and reduce the impact of the abnormality on the system. In addition, load balancing and system volatility constraints can also be set, which help to maintain the stable operation of the system and avoid performance degradation or failures caused by excessive local load or excessive system volatility.
[0083] Based on the traceability trigger chain obtained in step S2, the model can clarify the causes and processes of abnormal occurrences and adjust the system operation parameters accordingly. By optimizing these parameters, such as adjusting the data transmission rate and frequency, optimizing the data cleaning algorithms and strategies, and improving the model parameters for abnormal detection and traceability, this model provides an efficient and reliable solution for new energy data collection and processing. Under the premise of meeting various constraints, it maximally reduces the total system cost and improves the overall performance of the new energy data collection and diagnosis system.
[0084] The data transmission constraint is:
[0085]
[0086]
[0087] The data cleaning constraint is as follows:
[0088]
[0089] The anomaly detection accuracy constraint is as follows:
[0090]
[0091] The anomaly traceability time constraint is as follows:
[0092]
[0093] Among them, the effective transmission rate should meet the minimum requirement , is the throughput, is the packet loss rate, is the maximum network bandwidth, and the effective data ratio should not be lower than the preset minimum threshold , is the transmitted data, is the cleaned data, and the detection accuracy and recall rate of the anomaly detection network should meet the accuracy precision and recall rate precision requirements. The time for the anomaly traceability network to analyze and generate the traceability trigger chain should be lower than the maximum diagnostic delay allowed by the system, where t is the moment.
[0094] As can be seen from the above technical solutions, for a new energy data anomaly detection method and related devices provided by the embodiments of the present application, first, a data processing model based on SpectralNet is used to perform non-linear feature extraction and weighted aggregation processing on the acquired device real-time data to generate new energy aggregation data. Then, the new energy aggregation data is input into a data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and traceability to obtain anomaly detection results and a traceability trigger chain. Finally, a new energy data comprehensive optimization model can also be used to optimize the system operation parameters with the goal of minimizing the total system cost under the constraints of data transmission, data cleaning, anomaly detection accuracy, and anomaly traceability time.
[0095] This application is based on the SpectralNet-NAS-TFT method, which effectively overcomes the deficiencies of the prior art. By utilizing the non-linear dimensionality reduction ability of SpectralNet, it can deeply explore the deep features of new energy data, improve the quality of data collection and processing, and solve the problem that existing methods are difficult to mine non-linear features when dealing with high-dimensional complex data. Through NAS, an anomaly diagnosis model adapted to new energy data is automatically designed, enhancing the adaptive ability of the model, getting rid of the dependence on manual design, and better adapting to complex scenario changes. Combining the time series characteristics of TFT to process new energy data, it accurately identifies and predicts channel interruption anomalies, comprehensively solving the problems that existing methods are difficult to identify and locate complex anomaly types and have poor generalization ability during anomaly diagnosis. In addition, this method can also optimize data collection efficiency, reduce redundancy, improve anomaly detection and prediction accuracy, and provide strong data support for the operation of smart grids.
[0096] In some embodiments of this application, the data processing model is further introduced, which may specifically include:
[0097] The data processing model includes a data collection network, a data communication network, a data cleaning network, and a data stream processing network;
[0098] The data collection network obtains device real-time data from multiple devices through a data collection system;
[0099] The data communication network transmits the device real-time data through a containerized communication module at an effective data transmission rate calculated based on single-channel throughput and data packet loss rate to obtain transmitted data;
[0100] The data cleaning network cleans outliers and noise signals from the transmitted data to obtain cleaned data;
[0101] The data stream processing network uses the SpectralNet algorithm to cluster and extract features from the cleaned data, generates a non-linear feature relationship function, and performs weighted aggregation on the extracted feature results to generate new energy aggregated data.
[0102] Specifically, as the starting link of data processing, the data collection network obtains device real-time data from multiple devices with the help of a data collection system. These devices have a wide range of sources, covering various sensors and monitoring devices in the new energy system, and the operation data they generate in real time provides the basic original information for subsequent analysis.
[0103] From multiple devices Obtain real-time data , the sampling time interval is , the total number of devices is N, and the collected data is as follows:
[0104]
[0105] The data communication network is responsible for transmitting the real-time device data obtained by the data acquisition network. Through the containerized communication module, the data is sent out according to specific transmission rules to form transmission data. This transmission method takes into account the effective data transmission rate to ensure the effectiveness and reliability of the data during transmission.
[0106] The generation method of the said transmission data is as follows:
[0107]
[0108]
[0109] The data cleaning network processes the transmission data, removes the outliers and noise signals therein to obtain the cleaned data, improves the quality of the data, and provides a purer data sample for subsequent analysis.
[0110] The generation method of the said cleaned data is as follows:
[0111]
[0112] The data stream processing network adopts the SpectralNet algorithm. First, it clusters and extracts features from the cleaned data to generate a non-linear feature relationship function, and mines the potential patterns and features in the data. Then, it performs weighted aggregation on the extracted feature results to generate new energy aggregated data, making the data more representative and facilitating subsequent operations such as anomaly detection and system optimization based on these aggregated data.
[0113] The said non-linear feature relationship function is as follows:
[0114]
[0115] The generation method of the said new energy aggregated data is as follows:
[0116]
[0117] In the above formula, is the transmission data, is the effective data transmission rate, is the set of real-time device data at time t collected, is the single-channel throughput, is the data packet loss rate, is the cleaned data, is the outlier, is the noise signal, is the feature extraction function based on SpectralNet, It is the k-th feature result extracted by the SpectralNet algorithm, which is the new energy aggregated data, and is the weight of the k-th feature.
[0118] In some embodiments of the present application, the data channel anomaly diagnosis model is further introduced, which may specifically include:
[0119] The data channel anomaly diagnosis model includes an anomaly detection network, a root cause analysis network, and an anomaly traceability network;
[0120] The anomaly detection network is based on the new energy aggregated data and uses the NAS algorithm to detect the anomaly state, obtaining the anomaly detection result;
[0121] The root cause analysis network constructs a root cause set based on the anomaly detection result and the device dependency matrix;
[0122] The anomaly traceability network combines the root cause set and the new energy aggregated data and uses the TFT algorithm to perform time series traceability analysis, generating a traceability trigger chain.
[0123] Specifically, the anomaly detection network takes the new energy aggregated data as input and uses the NAS (Neural Architecture Search) algorithm to detect the anomaly state. The NAS algorithm can automatically explore and optimize the neural network architecture within a preset architecture search space through methods such as reinforcement learning and evolutionary algorithms. Based on this, the anomaly detection network can accurately learn the normal patterns in the new energy aggregated data, and then compare the input data with them. When the deviation of the data from the normal pattern exceeds a certain threshold, the anomaly detection result can be output to determine whether there is an anomaly in the data channel.
[0124] The generation method of the anomaly detection result is as follows:
[0125]
[0126] The root cause analysis network constructs a root cause set based on the anomaly detection result output by the anomaly detection network and combines it with the device dependency matrix. In the new energy system, there are complex interdependencies between devices, and this matrix quantitatively describes these relationships. The root cause analysis network analyzes the possible causes of the data channel anomaly by comprehensively considering the anomaly situation and the associations between devices, generating a root cause set to provide a direction for subsequent in-depth analysis.
[0127] The construction method of the root cause set is as follows:
[0128]
[0129] The root cause set obtained by combining the anomaly traceability network and the root cause analysis network and the new energy aggregation data are used for time series traceability analysis by the TFT (Time Fusion Transformer) algorithm. The TFT algorithm can effectively capture the long-term and short-term dependence relationships in time series data, and is particularly applicable to data with obvious time characteristics such as new energy data. Through the time series analysis of the new energy aggregation data and referring to the root cause set at the same time, a traceability trigger chain is generated, clearly presenting the complete process of the abnormal event from occurrence to development and the trigger relationships between various factors.
[0130] The generation method of the traceability trigger chain is as follows:
[0131]
[0132] Among them, is the anomaly detection result, is the anomaly status detection function based on NAS, is the new energy aggregation data, is the root cause set, is the parameter solving function corresponding to taking the maximum value, is the dependence relationship matrix between device i and device j, is the traceability trigger chain, is the data channel anomaly traceability function based on TFT, where t is the time.
[0133] Next, a new energy data anomaly detection device provided by an embodiment of the present application will be described. The new energy data anomaly detection device described below can be mutually corresponded and referred to the new energy data anomaly detection method described above.
[0134] See Figure 2 , Figure 2 which is a schematic diagram of a new energy data anomaly detection device disclosed in an embodiment of the present application.
[0135] As Figure 2 shown, the new energy data anomaly detection device may include:
[0136] A data acquisition unit 110, configured to perform non-linear feature extraction and weighted aggregation processing on the acquired device real-time data by using a data processing model based on SpectralNet, and generate new energy aggregation data;
[0137] A data detection unit 120, configured to input the new energy aggregation data into a data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and traceability, and obtain an anomaly detection result and a traceability trigger chain;
[0138] The comprehensive optimization unit 130 is used to utilize the new energy data comprehensive optimization model, based on the traceability trigger chain, to optimize the system operating parameters with the optimization goal of minimizing the total system cost under the constraints of data transmission, data cleaning, anomaly detection accuracy and anomaly tracing time.
[0139] It can be seen from the above technical solutions that a new energy data anomaly detection method and related equipment provided in the embodiment of the present application first use a data processing model based on SpectralNet to perform nonlinear feature extraction and weighted aggregation processing on the acquired real-time data of the device to generate new energy aggregate data. After that, the new energy aggregate data is input into the data channel anomaly diagnosis model based on NAS and TFT to perform data channel interruption anomaly detection and tracing, and obtain anomaly detection results and tracing trigger chain. Finally, the new energy data comprehensive optimization model can also be used, based on the tracing trigger chain, under the constraints of data transmission, data cleaning, anomaly detection accuracy and anomaly tracing time, to optimize the system operating parameters with the optimization goal of minimizing the total system cost.
[0140] This application is based on the SpectralNet-NAS-TFT method, which effectively overcomes the shortcomings of the prior art. Utilizing the nonlinear dimensionality reduction capability of SpectralNet, it is possible to deeply explore the deep features of new energy data, improve the quality of data acquisition and processing, and solve the problem that existing methods are difficult to mine nonlinear features when processing high-dimensional complex data. By automatically designing an abnormality diagnosis model adapted to new energy data through NAS, the adaptive ability of the model is improved, the dependence on manual design is eliminated, and it can better adapt to changes in complex scenarios. Combined with the time series characteristics of TFT processing new energy data, channel interruption anomalies can be accurately identified and predicted, which comprehensively solves the problem that existing methods are difficult to identify and locate complex anomaly types and have poor generalization capabilities during abnormal diagnosis. In addition, this method can also optimize data acquisition efficiency, reduce redundancy, improve anomaly detection and prediction accuracy, and provide strong data support for smart grid operation.
[0141] Optionally, the data processing model includes a data acquisition network, a data communication network, a data cleaning network and a data stream processing network;
[0142] The data acquisition network acquires real-time data of devices from multiple devices through a data acquisition system;
[0143] The data communication network transmits the real-time data of the device through the containerized communication module at an effective data transmission rate calculated based on a single-channel throughput and a data packet loss rate to obtain transmission data;
[0144] The data cleaning network cleans the transmission data of abnormal values and noise signals to obtain cleaned data;
[0145] The data stream processing network uses the SpectralNet algorithm to cluster and extract features from the cleaned data, generate a non-linear feature relationship function, and perform weighted aggregation on the extracted feature results to generate new energy aggregation data.
[0146] Optionally, the generation method of the transmission data is as follows:
[0147]
[0148]
[0149] The generation method of the cleaned data is as follows:
[0150]
[0151] The non-linear feature relationship function is:
[0152]
[0153] The generation method of the new energy aggregation data is as follows:
[0154]
[0155] Where, is the transmission data, is the effective data transmission rate, is the set of real-time device data at time t collected, is the single-channel throughput, is the data packet loss rate, is the cleaned data, is the outlier, is the noise signal, is the feature extraction function based on SpectralNet, is the k-th feature result extracted by the SpectralNet algorithm, is the new energy aggregation data, is the weight of the k-th feature.
[0156] Optionally, the data channel anomaly diagnosis model includes an anomaly detection network, a root cause analysis network, and an anomaly tracing network;
[0157] The anomaly detection network is based on the new energy aggregation data and uses the NAS algorithm to detect the anomaly state and obtain the anomaly detection result;
[0158] The root cause analysis network constructs a root cause set based on the anomaly detection result and the device dependency matrix;
[0159] The abnormal traceability network combines the root cause set and the new energy aggregated data, and uses the TFT algorithm to perform time series traceability analysis to generate a traceability trigger chain.
[0160] Optionally, the generation method of the anomaly detection result is:
[0161]
[0162] The construction method of the root cause set is:
[0163]
[0164] The generation method of the traceability trigger chain is:
[0165]
[0166] Wherein, is the anomaly detection result, is the NAS-based anomaly status detection function, is the new energy aggregated data, is the root cause set, is the parameter solving function corresponding to taking the maximum value, is the dependency relationship matrix between device i and device j, is the traceability trigger chain, is the data channel anomaly traceability function based on TFT, where t is the time.
[0167] Optionally, the data transmission constraint is:
[0168]
[0169]
[0170] The data cleaning constraint is:
[0171]
[0172] The anomaly detection accuracy constraint is:
[0173]
[0174] The anomaly traceability time constraint is:
[0175]
[0176] Wherein, the effective transmission rate should meet the minimum requirement , is the throughput, is the packet loss rate, For the maximum network bandwidth and the proportion of valid data It should not be lower than the preset minimum threshold , For the transmitted data For the cleaned data, the detection accuracy of the anomaly detection network and the recall rate should meet the accuracy precision and the recall rate precision requirements. The time for the anomaly traceability network to analyze and generate a traceability trigger chain should be lower than the maximum diagnostic delay allowed by the system , where t is the moment.
[0177] The new energy data anomaly detection device provided by the embodiments of this application can be applied to new energy data anomaly detection equipment. Figure 3 Shows the hardware structure block diagram of the new energy data anomaly detection equipment. Refer to Figure 3 , the hardware structure of the new energy data anomaly detection equipment may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;
[0178] In the embodiments of this application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 complete mutual communication through the communication bus 4;
[0179] The processor 1 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;
[0180] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, etc., such as at least one disk memory;
[0181] Among them, the memory stores a program, and the processor can call the program stored in the memory. The program is used for:
[0182] Using the data processing model based on SpectralNet to perform non-linear feature extraction and weighted aggregation processing on the acquired device real-time data to generate new energy aggregation data;
[0183] Inputting the new energy aggregation data into the data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and traceability to obtain an anomaly detection result and a traceability trigger chain;
[0184] Using the new energy data comprehensive optimization model, based on the traceability trigger chain, under the constraints of data transmission, data cleaning, anomaly detection accuracy, and anomaly traceability time, optimize the system operation parameters with the goal of minimizing the total system cost.
[0185] Optionally, the refinement function and expansion function of the program can be referred to the above description.
[0186] The embodiment of the present application also provides a readable storage medium, which can store a program suitable for execution by a processor, and the program is used for:
[0187] Using the data processing model based on SpectralNet to perform non-linear feature extraction and weighted aggregation processing on the acquired device real-time data to generate new energy aggregation data;
[0188] Input the new energy aggregation data into the data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and traceability, and obtain the anomaly detection result and the traceability trigger chain;
[0189] Using the new energy data comprehensive optimization model, based on the traceability trigger chain, under the constraints of data transmission, data cleaning, anomaly detection accuracy, and anomaly traceability time, optimize the system operation parameters with the goal of minimizing the total system cost.
[0190] Optionally, the refinement function and expansion function of the program can be referred to the above description.
[0191] The embodiment of the present application also provides a computer program product, including a computer program, and the method executed when the computer program is run by a processor is:
[0192] Using the data processing model based on SpectralNet to perform non-linear feature extraction and weighted aggregation processing on the acquired device real-time data to generate new energy aggregation data;
[0193] Input the new energy aggregation data into the data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and traceability, and obtain the anomaly detection result and the traceability trigger chain;
[0194] Using the new energy data comprehensive optimization model, based on the traceability trigger chain, under the constraints of data transmission, data cleaning, anomaly detection accuracy, and anomaly traceability time, optimize the system operation parameters with the goal of minimizing the total system cost.
[0195] Optionally, the refinement function and expansion function of the program can be referred to the above description.
[0196] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.
[0197] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For the same or similar parts among the various embodiments, reference may be made to each other.
[0198] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for detecting abnormal new energy data, characterized in that, Including: Using a data processing model based on SpectralNet to perform non - linear feature extraction and weighted aggregation on the acquired real - time device data, generating new - energy aggregated data; Inputting the new - energy aggregated data into a data - channel anomaly diagnosis model based on NAS and TFT for data - channel interruption anomaly detection and tracing, obtaining an anomaly detection result and a tracing trigger chain; Using a new - energy data comprehensive optimization model, based on the tracing trigger chain, under data - transmission constraints, data - cleaning constraints, anomaly - detection accuracy constraints, and anomaly - tracing time constraints, optimizing the system operation parameters with the goal of minimizing the total system cost.
2. The method according to claim 1, characterized in that, The data processing model includes a data acquisition network, a data communication network, a data cleaning network, and a data - flow processing network. Using a data processing model based on SpectralNet to perform non - linear feature extraction and weighted aggregation on the acquired real - time device data, generating new - energy aggregated data, includes: The data acquisition network acquires real - time device data from multiple devices; The data communication network transmits the real - time device data through a containerized communication module at an effective data transmission rate calculated based on single - channel throughput and data packet loss rate, obtaining transmitted data; The data cleaning network cleans the transmitted data for outliers and noise signals, obtaining cleaned data; The data - flow processing network uses the SpectralNet algorithm to perform clustering and feature extraction on the cleaned data, generating a non - linear feature relationship function, and performing weighted aggregation on the extracted feature results, generating new - energy aggregated data.
3. The method according to claim 2, characterized in that The generation method of the transmitted data is: The generation method of the cleaned data is: The non-linear characteristic relation function is as follows: The generation method of the new - energy aggregated data is: Among them, is for transmitting data, is the effective data transmission rate, is the real-time data set of the device at time t collected, is the single-channel throughput, is the data packet loss rate, is the cleaned data, is the outlier, is the noise signal, is the feature extraction function based on SpectralNet, is the k-th feature result extracted by the SpectralNet algorithm, is the new energy aggregation data, is the weight of the k-th feature.
4. The method according to claim 1, wherein The data - channel anomaly diagnosis model includes an anomaly detection network, a root - cause analysis network, and an anomaly tracing network. Inputting the new - energy aggregated data into a data - channel anomaly diagnosis model based on NAS and TFT for data - channel interruption anomaly detection and tracing, obtaining an anomaly detection result and a tracing trigger chain, includes: The anomaly detection network performs anomaly - state detection on the new - energy aggregated data using the NAS algorithm, obtaining an anomaly detection result; The root - cause analysis network constructs a root - cause set based on the anomaly detection result and the device - dependency matrix; The anomaly tracing network combines the root - cause set and the new - energy aggregated data, performs time - series tracing analysis using the TFT algorithm, generating a tracing trigger chain.
5. The method according to claim 4, wherein The generation method of the anomaly detection result is: The construction method of the root - cause set is: The generation method of the tracing trigger chain is: Among them, is the anomaly detection result, is the NAS-based anomaly status detection function, is the new energy aggregation data, is the root cause set, is the parameter solution function when taking the maximum value, is the dependency relationship matrix between device i and device j, is the traceability trigger chain, is the data channel anomaly traceability function based on TFT, where t is the time.
6. The method according to claim 1, wherein The data - transmission constraint is: The data - cleaning constraint is: The anomaly - detection accuracy constraint is: The anomaly - tracing time constraint is: Among them, the effective transmission rate shall meet the minimum requirement , where is the throughput is the packet loss rate is the maximum network bandwidth, and the effective data ratio shall not be lower than the preset minimum threshold , is the transmitted data is the cleaned data, and the detection accuracy and recall rate of the anomaly detection network shall meet the accuracy and recall rate precision requirements. The time for the anomaly traceability network to analyze and generate the traceability trigger chain shall be lower than the maximum diagnostic delay allowed by the system, where t is the time instant 7. A new energy data anomaly detection device, characterized in that, Including: A data acquisition unit, used to use a data processing model based on SpectralNet to perform non - linear feature extraction and weighted aggregation on the acquired real - time device data, generating new - energy aggregated data; A data detection unit, configured to input the new energy aggregated data into a data channel anomaly diagnosis model based on NAS and TFT for data channel interruption anomaly detection and traceability, and obtain an anomaly detection result and a traceability trigger chain; A comprehensive optimization unit, configured to use a new energy data comprehensive optimization model, and based on the traceability trigger chain, optimize the system operation parameters with the goal of minimizing the total system cost under data transmission constraints, data cleaning constraints, anomaly detection accuracy constraints, and anomaly traceability time constraints.
8. A new energy data anomaly detection device, characterized in that, It includes a memory and a processor; The memory is configured to store programs; The processor is configured to execute the programs to implement each step of the new energy data anomaly detection method according to any one of claims 1-6.
9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the new energy data anomaly detection method according to any one of claims 1-6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program runs on the processor, it executes each step of the new energy data anomaly detection method according to any one of claims 1-6.