Distributed storage control method and system for computer big data

Abstract

The invention relates to the technical field of data processing, in particular to a distributed storage control method and system for computer big data, and the method comprises the steps: obtaining electrical time sequence data to be stored in a distributed storage node, and enabling the electrical time sequence data to comprise current time sequence data and voltage time sequence data; performing window sliding on the electrical time sequence data to obtain a power window of each data point, performing window sliding on the power window to obtain a plurality of power sub-windows, and obtaining the electrical data authenticity of the power sub-windows according to the association between the current time sequence data and the voltage time sequence data of the power sub-windows and the electrical data stability degree of the power sub-windows. According to the method, the authenticity of the electrical data of each power sub-window is quantified and evaluated, so that the power sub-windows corresponding to the noise data are identified and eliminated, the influence of noise factors such as electromagnetic interference on part of the power sub-windows is reduced, and the accuracy of noise data detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a distributed storage control method and system for computer big data. Background Art

[0002] With the rapid development of information technology, the storage demand for computer big data has gradually increased, and the traditional single storage architecture can no longer effectively meet the storage demand for massive data. To solve this problem, the distributed storage method has become an important technical means to support large-scale data storage; the distributed storage system can achieve high scalability, reliability, and high performance by dispersing data storage on multiple nodes; however, due to the particularity of its data distribution and storage method, various data quality problems are likely to occur in the distributed storage system, such as data corruption, data loss, access latency, abnormal access behavior, etc. These abnormal phenomena not only affect the stability of the system, but may also lead to inaccurate data processing results, thereby affecting subsequent data analysis and decision-making.

[0003] The distributed storage system is commonly used for storing electrical data of a virtual power plant, and the electrical data includes current data and voltage data. Dispersing the electrical data storage in different distributed storage nodes provides high scalability and high availability, thus supporting the storage and management requirements of the virtual power plant for electrical data.

[0004] Before storing the electrical data on each distributed storage node, it is necessary to perform anomaly detection on the electrical data to reduce the interference caused by abnormal data in subsequent processing. Currently, the fuzzy entropy method is commonly used to perform anomaly detection on the electrical data on each distributed storage node. However, when using the fuzzy entropy method to perform anomaly detection on the electrical data on each distributed storage node, it is determined whether each data point is abnormal by analyzing the variable value of the fuzzy entropy value corresponding to each data point and its immediately preceding data point within their respective windows. And the calculation of the fuzzy entropy value within the window corresponding to each data point is based on the electrical data within the data window of the data point. However, when collecting electrical data in a virtual power plant, due to external electromagnetic fields, especially the radiation from high-frequency electronic devices, it may interfere with the collection of electrical data. The electromagnetic interference will affect the measurement instrument's electronic circuit through induced current or directly, resulting in noise in the collected data. Then when there is noise data within the window of a data point, the accuracy of the subsequently calculated fuzzy entropy value will be reduced, thereby affecting the data anomaly detection result and the subsequent data storage reliability. Summary of the Invention

[0005] To solve the technical problem of low accuracy in detecting noise data of electrical data on existing distributed storage nodes, the purpose of the present invention is to provide a distributed storage control method and system for computer big data. The specific technical solutions are as follows: In the first aspect of the present invention, a distributed storage control method for computer big data is provided, including: Obtain electrical timing data to be stored in a distributed storage node, where the electrical timing data includes current timing data and voltage timing data; Slide the electrical timing data with a first preset window length to obtain a power window for each data point, and slide the power window with a second preset window length to obtain multiple power sub-windows, where the first preset window length is greater than the second preset window length; Based on the correlation between the current timing data and voltage timing data of the power sub-window, and the stability degree of the electrical data of the power sub-window, obtain the authenticity of the electrical data of the power sub-window; Determine noise data based on the authenticity of the electrical data.

[0006] In an exemplary embodiment, the process of obtaining the stability degree of the electrical data of the power sub-window includes: Obtain the overall similarity between a first power sub-window and other power sub-windows belonging to the same power window; the first power sub-window is any one of the power sub-windows; Obtain the data difference of the first power sub-window, where the data difference is the difference between the root mean square value and the mean value of the electrical data of the first power sub-window; Based on the data difference and the overall similarity, obtain the stability degree of the electrical data of the first power sub-window; the stability degree of the electrical data is inversely proportional to the data difference and directly proportional to the overall similarity.

[0007] In an exemplary embodiment, the calculation process of the stability degree of the electrical data is as follows: Normalize the overall similarity, perform negative correlation normalization on the data difference, and calculate the product of the normalized overall similarity and the negatively correlated normalized data difference as the stability degree of the electrical data.

[0008] In an exemplary embodiment, the process of obtaining the correlation between the current timing data and voltage timing data of the power sub-window includes: obtaining the correlation coefficient between the current timing data and voltage timing data of the power sub-window.

[0009] In an exemplary embodiment, the process of obtaining the authenticity of the electrical data of the power sub-window includes: Based on the electrical data stability degree of the power sub-window and the correlation coefficient, the authenticity of the electrical data of the power sub-window is obtained, and the authenticity of the electrical data is proportional to the electrical data stability degree and proportional to the correlation coefficient.

[0010] In an exemplary embodiment, determining noise data according to the authenticity of the electrical data includes: Comparing the authenticity of the electrical data of the power sub-window with a preset authenticity threshold. If it is less than the preset authenticity threshold, the power sub-window is a noise sub-window.

[0011] In an exemplary embodiment, the distributed storage control method further includes: Obtaining target power sub-windows in the power windows of each data point, where the target power sub-windows are the power sub-windows remaining after screening out the noise sub-windows in the power windows; Calculating the fuzzy entropy values of each data point based on the target power sub-windows in the power windows of each data point; Obtaining the entropy value difference between the fuzzy entropy values of two adjacent data points; Performing anomaly judgment according to the entropy value difference.

[0012] In an exemplary embodiment, performing anomaly judgment according to the entropy value difference includes: Comparing the normalized value of the entropy value difference with a preset difference threshold. If the entropy value difference is greater than the preset difference threshold, it is determined that an anomaly occurs at the latter data point among the two adjacent data points.

[0013] In an exemplary embodiment, the distributed storage control method further includes: Marking the data points where anomalies occur and storing the marked data points and the corresponding electrical timing data in the distributed storage nodes.

[0014] In a second aspect of the present invention, a distributed storage control system for computer big data is provided, including: a memory and a processor; the memory is connected to the processor; the memory is used for storing program instructions; the processor is used for implementing the above-mentioned distributed storage control method for computer big data when the program instructions are executed.

[0015] The present invention has the following beneficial effects: The electrical timing data to be stored in the distributed storage nodes is windowed to obtain the power windows of each data point, and then the power windows are windowed with another window length to obtain multiple power sub-windows corresponding to the power windows of each data point. By combining the correlation between the current timing data and the voltage timing data of the power sub-windows and the stability degree of the electrical data of the power sub-windows, the authenticity of the electrical data of the power sub-windows is obtained, realizing the quantification and evaluation of the authenticity of the electrical data of each power sub-window, thereby identifying and eliminating the power sub-windows corresponding to the noise data, reducing the influence of noise factors such as electromagnetic interference on some power sub-windows, improving the accuracy of noise data detection, avoiding the adverse impact of these noise data on the calculation of the fuzzy entropy value, and thus being able to effectively improve the accuracy of fuzzy entropy calculation to enhance the effect of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flowchart of a distributed storage control method for computer big data provided by an embodiment of the present invention; Figure 2 is a flowchart for obtaining the degree of correlation provided by an embodiment of the present invention; Figure 3 is a flowchart for obtaining the degree of association provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following combines the drawings and preferred embodiments to detail the specific implementation manners, structures, features and effects of the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The data information collected in this application is obtained through full consent and authorization, and the collection, use and processing of relevant information need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0019] An application scenario of the distributed storage control method for computer big data provided in this embodiment is as follows: A distributed storage system is set up for a virtual power plant, and the distributed storage system is used to store the electrical data of the virtual power plant. The distributed storage system includes multiple distributed storage nodes. The hardware devices corresponding to the distributed storage nodes can be conventional data storage devices, such as computer hard disks. The number of distributed storage nodes set by the implementer is specifically set and not limited.

[0020] The electrical data stored in the distributed storage nodes in this embodiment includes current data and voltage data. Moreover, a distributed storage control method for computer big data provided in this embodiment is applied to the data acquisition and storage of direct current electricity in a virtual power plant, that is, the current data is specifically direct current data, and the voltage data is specifically direct current voltage data.

[0021] A direct current sensor and a direct current voltage sensor are provided at the positions to be detected in the virtual power plant. The current data and voltage data are collected according to a preset sampling frequency. The sampling duration and sampling frequency are specifically set according to the actual situation. In an exemplary embodiment, the sampling duration is taken as 1 hour, the acquisition frequency is ten times per second, and the current data and voltage data are sampled synchronously, that is, for any sampling moment, the current data and voltage data at this sampling moment can be obtained. In addition, for the convenience of data processing, an analog-to-digital conversion device is used to digitally convert the above-mentioned collected current and voltage data to obtain a digital representation of the above data.

[0022] The purpose of a distributed storage control method for computer big data provided in this embodiment is: by analyzing the numerical change characteristics of the current data and voltage data on each distributed storage node in the distributed storage system of the virtual power plant, the authenticity of each power sub-window is obtained, and the power sub-windows are screened according to the authenticity of each power sub-window, and the power sub-windows with poor authenticity are eliminated to improve the accuracy of noise data detection. Subsequently, according to all the power sub-windows with high authenticity of a data point, the optimized fuzzy entropy value of this data point is calculated, and then a more accurate anomaly detection result is obtained.

[0023] As Figure 1 shown, a distributed storage control method for computer big data provided in this embodiment includes the following steps: Step 1: Obtain electrical time-series data to be stored in the distributed storage node, and the electrical time-series data includes current time-series data and voltage time-series data; Step 2: Slide the electrical time-series data according to a first preset window length to obtain power windows for each data point, and then slide the power windows according to a second preset window length to obtain multiple power sub-windows, where the first preset window length is greater than the second preset window length; Step 3: Obtain the authenticity of the electrical data of the power sub-window according to the correlation between the current time-series data and voltage time-series data of the power sub-window and the stability degree of the electrical data of the power sub-window; Step 4: Determine the noise data according to the authenticity of the electrical data.

[0024] The following specifically explains each step.

[0025] Step 1: Obtain the electrical timing data to be stored in the distributed storage node. The electrical timing data includes current timing data and voltage timing data.

[0026] Since the data processing process on each distributed storage node in the distributed storage system is the same, in this embodiment, any one of the distributed storage nodes is taken as an example.

[0027] Obtain the electrical timing data to be stored in the distributed storage node. Among them, the electrical timing data includes current timing data and voltage timing data. The current timing data includes multiple current data obtained according to the sampling frequency, and the voltage timing data includes multiple voltage data obtained according to the sampling frequency.

[0028] Step 2: Slide the electrical timing data with a first preset window length to obtain the power window of each data point, and then slide the power window with a second preset window length to obtain multiple power sub-windows. The first preset window length is greater than the second preset window length.

[0029] The detection of noise data is mainly based on analyzing the numerical change characteristics of the current data and voltage data on each distributed storage node in the distributed storage system of the virtual power plant. Therefore, first, it is necessary to slide the electrical timing data with a first preset window length to obtain the power window of each data point. The data point represents a certain sampling moment, and the power window of each data point is essentially the power window of each sampling moment. The first preset window length is set according to actual needs. In an exemplary embodiment, the first preset window length is 600. Then, the power window of each data point is a window with a data volume of 600. And the position of each data point in its corresponding power window is set according to actual needs. For example, it can be in the center position of the power window, or in the left edge or right edge position. In an exemplary embodiment, the last data position in the power window is used as the position of the corresponding data point. Moreover, the sliding step length of sliding the electrical timing data with a first preset window length is 1.

[0030] In the above sliding window operation mode, the first part of the data points in the sampling duration will not be able to obtain a complete-length power window. To avoid the above situation, on the basis of obtaining the electrical timing data of the sampling duration, the power data of a certain historical time period before the sampling duration is also obtained, and the power window of each data point in the sampling duration is obtained according to the power data of the sampling duration + historical time period, so as to ensure that each data point in the sampling duration has a corresponding power window.

[0031] For any power window, sliding the window according to the second preset window length to obtain multiple power sub - windows. Among them, the first preset window length is greater than the second preset window length. To facilitate obtaining noise sub - windows and avoid deleting too much normal data as much as possible, the second preset window length can be set smaller, and the gap between the first preset window length and the second preset window length can be larger. For example, if the second preset window length is 20, then each power sub - window is a window with a data volume of 20. Moreover, the sliding step of sliding the power window according to the second preset window length is 1. Thus, multiple power sub - windows of each data point are obtained.

[0032] It should be understood that since the power time - series data includes current time - series data and voltage time - series data, each data point corresponds to each current data point and voltage data point in the current time - series data and voltage time - series data respectively. Therefore, the current time - series data and voltage time - series data are respectively window - slid in the above - mentioned manner, and the window - sliding methods for both are exactly the same. Power windows of each current data point are obtained, where the power window includes multiple power sub - windows, and power windows of each voltage data point are obtained, where the power window includes multiple power sub - windows. Since each current data point and each voltage data point are synchronously collected, each current data point and each voltage data point are collectively referred to as each data point.

[0033] Using the above - mentioned method, the calculation method for the number of power sub - windows is: power window length - power sub - window length + 1.

[0034] In an exemplary embodiment, the similarity tolerance when calculating the fuzzy membership degree of each power sub - window with other power sub - windows belonging to the same power window is preset to an empirical value of 0.25.

[0035] Step 3: Obtain the authenticity of the electrical data of the power sub - window according to the correlation between the current time - series data and voltage time - series data of the power sub - window and the stability degree of the electrical data of the power sub - window.

[0036] In the distributed storage system of a virtual power plant, the electrical data obtained on each distributed storage node reflects the changes in the electrical data. By analyzing the stability of this electrical data, it can help monitor whether the relevant equipment is operating normally, whether there are abnormal fluctuations, or whether it is affected by external interference, etc. The smaller the difference between the root mean square value and the mean value of the electrical data in each power sub-window, and the more similar the overall situation of each power sub-window is to all other power sub-windows belonging to the same power window, the more stable the data of the power sub-window can be indicated, the lower the possibility of the existence of noise data, and the greater the corresponding degree of stability. Moreover, under normal operating conditions, the electrical data should be relatively stable. Even under the influence of external factors such as load fluctuations and weather changes, the fluctuations of the electrical data will remain within a certain range; in this case, the difference between the root mean square value and the mean value of each power sub-window should be small, indicating that the data changes little and is stable. Due to the strong similarity between normal operating data, it indicates that the electrical data shows high consistency and stability.

[0037] Therefore, in an exemplary embodiment, as Figure 2 shown, the process of obtaining the stability degree of the electrical data of the power sub-window includes: Step 3-1: Obtain the overall similarity situation of the first power sub-window and other power sub-windows belonging to the same power window.

[0038] Since the data processing process for each power window is the same, take any one power window as an example below. And, set the first power sub-window as any one power sub-window in the power window. Since the power window includes multiple power sub-windows, then, among the power sub-windows belonging to the same power window, in addition to the first power sub-window, there are also other multiple power sub-windows.

[0039] Obtain the overall similarity situation of the first power sub-window and other power sub-windows belonging to the same power window. Among them, the overall similarity situation characterizes the overall similarity between the first power sub-window and other power sub-windows belonging to the same power window. The higher the overall similarity, the more similar the first power sub-window is to other power sub-windows belonging to the same power window. The overall similarity situation can be obtained by an existing similarity algorithm. In an exemplary embodiment, set the average value of fuzzy membership as the overall similarity situation, that is, obtain the average value of the fuzzy membership of the first power sub-window and other power sub-windows belonging to the same power window. As other implementation manners, it is also possible to respectively obtain the Pearson correlation coefficient or cosine similarity between the first power sub-window and each of the other power sub-windows belonging to the same power window, and then calculate the average value of the Pearson correlation coefficient or cosine similarity as the overall similarity situation.

[0040] Under the interference of external noise (such as electromagnetic interference, power grid fluctuations, etc.), irregular fluctuations will occur in the electrical data. In the case of strong noise interference, the data of some power sub-windows may show irregular peaks or large fluctuations, which will deteriorate the overall similarity between the first power sub-window and other power sub-windows belonging to the same power window, that is, the fuzzy membership degree will be reduced, because there are significant differences between these power sub-windows with large fluctuations and other normal power sub-windows.

[0041] Step 3-2: Obtain the data difference of the first power sub-window. The data difference is the difference between the root mean square value and the mean value of the electrical data of the first power sub-window.

[0042] Obtain the root mean square value of the electrical data of the first power sub-window and the mean value of the electrical data, and then calculate the data difference between the root mean square value and the mean value. Under the interference of external noise (such as electromagnetic interference, power grid fluctuations, etc.), irregular fluctuations will occur in the electrical data. In the case of strong noise interference, the data of some power sub-windows may show irregular peaks or large fluctuations, so that the data difference between the root mean square value and the mean value of the power sub-window will be relatively large, indicating poor data stability.

[0043] Step 3-3: Obtain the stability degree of the electrical data of the first power sub-window according to the data difference and the overall similarity.

[0044] As can be seen from the above analysis, the stability degree of electrical data is inversely proportional to the data difference and directly proportional to the overall similarity. Therefore, according to the data difference and the overall similarity of the first power sub-window, the stability degree of the electrical data of the first power sub-window is obtained. In an exemplary embodiment, the calculation process of the stability degree of electrical data is as follows: Normalize the overall similarity of the first power sub-window, perform negative correlation normalization on the data difference of the first power sub-window, and calculate the product of the normalized overall similarity and the negatively correlated normalized data difference as the stability degree of the electrical data of the first power sub-window.

[0045] It should be understood that the normalization in this embodiment and the norm normalization function can be specifically set according to the actual situation. For example, linear normalization methods such as maximum-minimum normalization can be used, or the following common methods can also be used: , represents the processing object, and exp represents the exponential function with the natural constant e as the base. The negative correlation normalization method can be: .

[0046] As other embodiments, the stability degree of electrical data can also be variance, that is, the variance of the electrical data of the first power sub-window is obtained, and the stability degree of the electrical data is characterized by the negative correlation normalization coefficient of the variance.

[0047] Since the electrical data includes current data and voltage data. Then, in the distributed storage system of the virtual power plant, the current data obtained on each distributed storage node reflects the change of the current. By analyzing the stability of these current data, it can help monitor whether the relevant equipment is operating normally, whether there are abnormal fluctuations, or whether it is affected by external interference, etc. The smaller the difference between the root mean square value and the mean value of the data in each power sub-window, and the larger the average value of the fuzzy membership degrees of each power sub-window and all other power sub-windows belonging to the same power window, the more stable the current data of the power sub-window can be explained, the lower the possibility of noise data, and the greater the corresponding stability degree. Moreover, under normal operating conditions, the current data should be relatively stable. Even under the influence of external factors such as load fluctuations and weather changes, the fluctuations of the current data will remain within a certain range; in this case, the difference between the root mean square value and the mean value of each power sub-window should be small, indicating that the data changes little and is stable; due to the strong similarity between the normal operating data, the average value of the fuzzy membership degrees between different power sub-windows is also high, indicating that the current data in the entire system shows high consistency and stability.

[0048] Therefore, by adopting the above process, the stability degree of the current data of the first power sub-window is obtained. In an exemplary embodiment, the calculation formula is as follows: ; Wherein, represents the stability degree of the current data of the th current data point (i.e., the th data point, that is, the th power window) of the th power sub-window; represents the average value of the fuzzy membership degrees of the th current data point of the th power sub-window and all other power sub-windows belonging to the th current data point, and this value can be obtained according to the existing fuzzy membership degree algorithm; represents the root mean square value of the current data of the th current data point of the th power sub-window; represents the mean value of the current data within the th current data point of the th power sub-window; represents the normalization function.

[0049] In the formula The smaller it is, it indicates that the current data of the th power sub-window of the

[0050] th current data point is more stable, the lower the possibility of noise data, and the greater the corresponding stability degree. The larger it is, it indicates that the th power sub-window of the th current data point and all other power sub-windows belonging to the same th current data point are closer in distance, that is, the numerical performance is closer, indicating that the th power sub-window of the th current data point and all other power sub-windows belonging to the same th current data point have a better overall similarity. Moreover, can be used as the credibility. The larger it is, it can indicate that the credibility of the smaller absolute value of the difference between the root mean square value and the mean value of the data within the th power sub-window of the th current data point is greater, and it can further indicate that the th power sub-window of the

[0051] th current data point is more stable, the lower the possibility of noise data, and the greater the corresponding stability degree. ; Among them, represents the voltage data stability degree of the th voltage data point (i.e., the th power window) of the th power sub-window; represents the average value of the fuzzy membership degrees of the th voltage data point of the th power sub-window and all other power sub-windows belonging to the same th voltage data point; represents the root mean square value of the voltage data of the th voltage data point of the th power sub-window; represents the mean value of the voltage data within the th voltage data point of the th power sub-window.

[0052] Through the above - step analysis, the stability degrees of current data and voltage data for each power sub - window are obtained. In the distributed power system of a virtual power plant, according to Ohm's law, there is a positive correlation between current and voltage, that is, the change in voltage will affect the magnitude of current. In the stability analysis of power sub - windows, if the change trends of current data and voltage data are consistent, then the authenticity of the electrical data reflected by the power sub - window is relatively high. However, since the analysis of the stability degree of current data is only based on the numerical performance of the current data itself, when quantifying the stability degree of data within a power sub - window, noise data may cause some originally normal data to be wrongly considered as abnormal data, thereby reducing the accuracy of the stability degree of data within a power sub - window.

[0053] In this step, in order to reduce the interference of noise data, the correlation between the numerical change characteristics of current data and voltage data will be further analyzed, and then combined with the stability degrees of current data and voltage data within each power sub - window, the authenticity of data within each power sub - window can be obtained.

[0054] Under the normal operating state of the virtual power plant, current and voltage usually show a positive correlation. When the load changes, voltage and current will fluctuate together, maintaining a certain proportional relationship. For example, an increase in power demand is usually accompanied by synchronous changes in voltage and current, that is, a positive correlation. The greater the voltage, the greater the current. If the fluctuation trends of current and voltage within each power sub - window are consistent, then the stability of this power sub - window is relatively high, the interference of noise data is less, and the credibility of the stability analysis is stronger. Therefore, in this scenario, due to the high consistency of current and voltage data, the influence of noise on data is small, and the evaluation of the stability and authenticity of current data is relatively accurate.

[0055] Therefore, the stronger the correlation between current data and voltage data in each power sub - window, the less noise data exists in the electrical data of this power sub - window, the stronger the credibility of the stability degree of electrical data within the power sub - window, and the stronger the corresponding authenticity.

[0056] Then, obtain the correlation between the current time - series data and voltage time - series data of the first power sub - window. Since noise interference in the power system (such as electromagnetic interference, frequency fluctuations, etc.) usually affects the data of current and voltage, under the influence of noise interference, the current data may have abnormal fluctuations, while the voltage data may not change significantly. At this time, the positive correlation between current and voltage may become weaker. In this case, the poor correlation between current and voltage may be caused by the fluctuations of noise data. By analyzing the change relationship between current and voltage, these inconsistent data can be identified, thereby excluding unreliable data points. And combined with voltage data, abnormal changes caused by noise can be effectively identified, reducing the interference of noise on the stability evaluation of current data.

[0057] In an exemplary embodiment, since the current data of the first power sub-window constitutes current time-series data, that is, it constitutes a current data vector, and the voltage data of the first power sub-window constitutes voltage time-series data, that is, it constitutes a voltage data vector. Then, the correlation coefficient of the current time-series data and the voltage time-series data of the first power sub-window is obtained, that is, the correlation coefficient of the current data vector and the voltage data vector of the first power sub-window. Among them, the correlation coefficient can be cosine similarity, Pearson correlation coefficient, and so on. The larger the similarity coefficient, the more similar the changes of the current time-series data and the voltage time-series data of the first power sub-window are, and the stronger the correlation between the current data and the voltage data in the first power sub-window is.

[0058] Then, according to the stability degree and the correlation coefficient of the electrical data of the first power sub-window, the authenticity of the electrical data of the first power sub-window is obtained. The authenticity of the electrical data is proportional to the stability degree of the electrical data and proportional to the correlation coefficient.

[0059] In an exemplary embodiment, the calculation formula of the authenticity of the electrical data is as follows: ; Among them, represents the authenticity of the electrical data of the th power sub-window of the th power window, represents the correlation coefficient of the current time-series data and the voltage time-series data of the th power sub-window of the th power window.

[0060] and The larger they are, it can be explained that from the perspective of the numerical performance of current and voltage, the greater the stability degree of the corresponding current data and voltage data in the th power sub-window of the th power window, then the greater its corresponding authenticity. Moreover, and The larger they are, the greater the credibility that the correlation between the current time-series data and the voltage time-series data of the th power sub-window of the th power window is stronger. Further, it can be explained that the less noise data exists in the electrical data of the th power sub-window, the stronger the credibility of the stability degree of the electrical data of the mth power sub-window, and the stronger the corresponding authenticity.

[0061] The larger it is, it indicates that the th power sub-window of the The stronger the correlation between the current time-series data and the voltage time-series data of the mth power sub-window, the less noise data exists in the electrical data within the mth power sub-window, the stronger the credibility of the stability of the electrical data in the mth power sub-window, and the stronger the corresponding authenticity.

[0062] Thus, the authenticity of the electrical data of each power sub-window in each data point is obtained. The greater the authenticity of the electrical data, the smaller the possibility that the power sub-window has noise.

[0063] Step 4: Determine the noise data based on the authenticity of the electrical data.

[0064] In an exemplary embodiment, a authenticity threshold is preset. The value range of the preset authenticity threshold is 0 - 1, and the specific value is set according to the judgment requirement. The stricter the judgment, the larger the preset authenticity threshold can be set, such as 0.5. The preset authenticity threshold is used to compare with the authenticity of the electrical data of each power sub-window of each data point. If it is greater than or equal to the preset authenticity threshold, it indicates that the authenticity of the power sub-window is relatively high and there is no noise data; if it is less than the preset authenticity threshold, it indicates that the power sub-window has noise data.

[0065] Therefore, compare the authenticity of the electrical data of each power sub-window of each data point with the preset authenticity threshold, obtain the power sub-window corresponding to the authenticity of the electrical data less than the preset authenticity threshold, and regard the power sub-window corresponding to less than the preset authenticity threshold as the noise sub-window.

[0066] Thus, judge each power sub-window of each data point to determine the noise sub-window therein, and realize the detection of noise data.

[0067] In an exemplary embodiment, as Figure 3 shown, the distributed storage control method further includes: Step 5: Obtain the target power sub-window in the power window of each data point.

[0068] Filter out the noise sub-windows in the power window of each data point. The remaining power sub-windows are all power sub-windows with the authenticity of the electrical data greater than or equal to the preset authenticity threshold, which are power sub-windows with relatively high authenticity of the electrical data and less possibility of noise data. The power sub-windows remaining after filtering out the noise sub-windows in the power window are called target power sub-windows, so as to obtain the target power sub-windows in the power window of each data point.

[0069] Step 6: Calculate the fuzzy entropy value of each data point based on the target power sub-window in the power window of each data point.

[0070] According to the target power sub-windows in the power windows of each data point, that is, all the power sub-windows with high authenticity at each data point, the fuzzy entropy values of each data point are calculated. The specific calculation method of the fuzzy entropy value is the prior art and will not be elaborated here. The fuzzy entropy values of each data point calculated here are the fuzzy entropy values obtained after optimizing the above data processing process.

[0071] Step 7: Obtain the entropy value difference between the fuzzy entropy values of two adjacent data points.

[0072] In an exemplary embodiment, calculate the entropy value difference between the fuzzy entropy value of each data point and the fuzzy entropy value at its immediately preceding data point. The entropy value difference is specifically the absolute value of the difference between the fuzzy entropy values, and the calculation formula is as follows: Where, represents the entropy value difference between the fuzzy entropy value of the i-th data point and the (i - 1)-th data point, represents the fuzzy entropy value of the i-th data point, represents the fuzzy entropy value of the (i - 1)-th data point.

[0073] The magnitude of the absolute value of the difference between the fuzzy entropy values reflects the degree of change in the entropy values of two adjacent data points. By the magnitude of the absolute value of the difference between the fuzzy entropy values, it is judged whether there is an abnormality at the data point.

[0074] Step 8: Perform abnormality judgment according to the entropy value difference.

[0075] Preset a difference threshold. The numerical value of this preset difference threshold is set according to the actual judgment requirements. In this embodiment, 0.15 is taken as an example. This preset difference threshold is used to characterize the magnitude of the entropy value difference between the fuzzy entropy values of two adjacent data points. If the normalized value of the entropy value difference is greater than this preset difference threshold, it means that the entropy value difference between the fuzzy entropy values of two adjacent data points is too large. Then, there is data abnormality between the two adjacent data points.

[0076] Then, compare the normalized value of the entropy value difference between the fuzzy entropy values of every two adjacent data points with the preset difference threshold. If the normalized value of the entropy value difference is greater than the preset difference threshold, it is determined that an abnormality has occurred at the latter data point among the corresponding two adjacent data points. Thus, abnormality judgment is performed on each data point to locate the abnormal data points. Among them, the normalization method is a well-known technology and will not be introduced in detail here.

[0077] Taking the above calculation formula as an example: If , represents the preset difference threshold, indicating that the degree of change between the fuzzy entropy value of the i-th data point and the (i - 1)-th data point is greater and exceeds the threshold, indicating that an abnormality has occurred at the i-th data point.

[0078] In an exemplary embodiment, the distributed storage control method further includes: After identifying each abnormal data point, mark the data points with anomalies and store the marked data points and the corresponding electrical timing data in the distributed storage nodes. For other normal data points, there is no need to mark them, and directly store the other normal data points and the corresponding electrical timing data in the distributed storage nodes, which is convenient for relevant staff to query and process.

[0079] Therefore, for the distributed storage control method of computer big data provided in this embodiment, by evaluating the authenticity of each power sub-window, power sub-windows with large noise can be identified and excluded, avoiding the adverse effects of these distorted data on the calculation of the fuzzy entropy value. The optimized fuzzy entropy value is calculated only based on the real and reliable power sub-windows, thereby reducing the interference of noise data and enhancing the reliability and accuracy of the detection results. This method can effectively extract meaningful signals in a noisy environment, improve the sensitivity and stability of anomaly detection, reduce the false alarm rate, and ultimately make the anomaly detection of current data more accurate and effective in a complex and dynamic measurement environment.

[0080] This embodiment also provides a distributed storage control system for computer big data, including: a memory and a processor; the memory is connected to the processor, and the memory is used to store program instructions; the processor is used to implement the steps in the above-mentioned distributed storage control method embodiment of computer big data when the program instructions are executed.

[0081] In an exemplary embodiment, the present invention provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps in the above-mentioned distributed storage control method embodiment of computer big data.

[0082] It should be noted that: the above sequence of the embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A distributed storage control method for computer big data, characterized in that Including: Obtain electrical timing data to be stored in a distributed storage node, where the electrical timing data includes current timing data and voltage timing data; Slide the electrical timing data with a first preset window length to obtain power windows for each data point, and slide the power windows with a second preset window length to obtain multiple power sub-windows, where the first preset window length is greater than the second preset window length; Obtain the authenticity of the electrical data of the power sub-window based on the correlation between the current timing data and voltage timing data of the power sub-window, and the stability degree of the electrical data of the power sub-window; Determine noise data based on the authenticity of the electrical data.

2. The distributed storage control method for computer big data according to claim 1, characterized in that, The process of obtaining the stability degree of the electrical data of the power sub-window includes: Obtain the overall similarity between a first power sub-window and other power sub-windows belonging to the same power window; the first power sub-window is any one of the power sub-windows; Obtain the data difference of the first power sub-window, where the data difference is the difference between the root mean square value and the mean value of the electrical data of the first power sub-window; Obtain the stability degree of the electrical data of the first power sub-window based on the data difference and the overall similarity; the stability degree of the electrical data is inversely proportional to the data difference and directly proportional to the overall similarity.

3. The distributed storage control method for computer big data according to claim 2, characterized in that, The calculation process of the stability degree of the electrical data is as follows: Normalize the overall similarity, perform negative correlation normalization on the data difference, and calculate the product of the normalized overall similarity and the negatively correlated normalized data difference as the stability degree of the electrical data.

4. A distributed storage control method for computer big data according to claim 1, characterized in that, The process of obtaining the correlation between the current timing data and voltage timing data of the power sub-window includes: obtaining the correlation coefficient between the current timing data and voltage timing data of the power sub-window.

5. A distributed storage control method for computer big data according to claim 4, characterized in that, The process of obtaining the authenticity of the electrical data of the power sub-window includes: Obtain the authenticity of the electrical data of the power sub-window based on the stability degree of the electrical data of the power sub-window and the correlation coefficient, where the authenticity of the electrical data is directly proportional to the stability degree of the electrical data and directly proportional to the correlation coefficient.

6. A distributed storage control method for computer big data as described in claim 1, characterized in that, Determining noise data based on the authenticity of the electrical data includes: Compare the authenticity of the electrical data of the power sub-window with a preset authenticity threshold. If it is less than the preset authenticity threshold, then the power sub-window is a noise sub-window.

7. A distributed storage control method for computer big data according to claim 6, characterized in that, The distributed storage control method further includes: Obtain target power sub-windows in the power windows of each data point, where the target power sub-windows are the power sub-windows remaining after screening out the noise sub-windows in the power windows; Calculate the fuzzy entropy value of each data point based on the target power sub-windows in the power windows of each data point; Obtain the entropy difference between the fuzzy entropy values of two adjacent data points; Perform anomaly judgment based on the entropy difference.

8. A distributed storage control method for computer big data according to claim 7, characterized in that, Performing anomaly judgment based on the entropy difference includes: Compare the normalized value of the entropy difference with a preset difference threshold. If the entropy difference is greater than the preset difference threshold, then it is judged that an anomaly has occurred at the latter data point among the two adjacent data points.

9. A distributed storage control method for computer big data according to claim 8, characterized in that, The described distributed storage control method further includes: Marking the data points that appear abnormal, and storing the marked data points and the corresponding electrical timing data in the distributed storage node.

10. A distributed storage control system for computer big data, characterized by comprising: A memory and a processor; The memory is connected to the processor; The memory is used to store program instructions; The processor is used to implement the distributed storage control method of computer big data described in any one of claims 1-9 when the program instructions are executed.

Images