Electric power detection system based on multi-dimensional data

Abstract

The invention relates to the field of data processing, in particular to an electric power detection system based on multi-dimensional data, which comprises a data acquisition unit; the data processing unit is used for determining a data state according to the data reference value and the processing reference value and determining a processing mode according to the data state; the first distribution unit is used for executing a processing mode for determining distribution for the associated combined data and the non-associated data according to the association influence coefficient or the node coefficient according to the reference quantity proportion; the second distribution unit is used for executing a processing mode for distributing the to-be-distributed data according to the node processing reference value; the redundancy analysis unit is used for determining the node state according to the maximum difference degree of the data blocks and the proportion of the number of the similar data blocks; and the redundancy adjusting unit is used for determining an adjusting mode according to the node state and carrying out direct compression or differential compression on the data block. According to the invention, transmission processing of data can be adjusted and controlled in a targeted manner so as to improve data processing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to an electric power detection system based on multi-dimensional data. Background Art

[0002] With the continuous increase of power equipment, the amount of data that the power detection system needs to process is also growing continuously. In addition, due to the limited processing capacity of processing nodes, it is impossible to choose a reasonable way to distribute the data when the data volume surges, resulting in poor data transmission efficiency. Therefore, how to adjust the data transmission processing according to the correlation status and data volume between the actual collected data to improve the data processing efficiency is a technical problem that needs to be urgently solved by technical personnel in this field.

[0003] Chinese patent publication number CN116680423B discloses a management method, device, equipment and medium for multi-source heterogeneous data in the power supply chain, including: collecting multi-source heterogeneous data under the power grid supply chain, and storing the multi-source heterogeneous data according to data type classification; using the encoder of the CLIP algorithm to vectorize data of different data types to obtain a multi-dimensional feature vector; calculating the similarity of the multi-dimensional feature vectors obtained by converting the structured data in the multi-source heterogeneous data, and merging the fields of the multi-dimensional feature vectors whose similarity exceeds a preset threshold; clustering the multi-dimensional feature vectors obtained by converting the unstructured data in the multi-source heterogeneous data, and classifying and storing the clustered multi-dimensional feature vectors. It can be seen that the above technical solution has the following problems: it is impossible to adjust the transmission and processing of the data in a targeted manner according to the association status and data volume between the actual collected data, resulting in poor data processing efficiency. Summary of the invention

[0004] To this end, the present invention provides an electric power detection system based on multidimensional data to overcome the problem in the prior art that the data transmission and processing cannot be specifically adjusted according to the correlation status and data volume between the actual collected data, resulting in poor data processing efficiency.

[0005] To achieve the above object, the present invention provides a power detection system based on multi-dimensional data, comprising:

[0006] A data acquisition unit, comprising a plurality of data acquisition modules for performing data acquisition;

[0007] A data processing unit connected to the data acquisition unit, configured to determine a data state according to a data reference value of the data acquisition unit and a processing reference value of the data processing unit, and determine a processing method according to the data state;

[0008] A first allocation unit, which is connected to the data processing unit, is used to determine a processing method for allocating associated combination data and non-associated data according to an associated influence coefficient or a node coefficient according to a reference quantity ratio;

[0009] A second allocation unit, connected to the data processing unit, for executing a processing method for allocating the data to be allocated according to the node processing reference value;

[0010] a redundancy analysis unit connected to the first allocation unit and the second allocation unit, for dividing the incoming data into a number of data blocks of the same size when performing redundant data analysis on a single processing node, and using a secure hash algorithm to generate a hash value corresponding to each data block, and determining the node status according to the maximum difference between the data blocks and the proportion of the number of similar data blocks;

[0011] The redundancy adjustment unit is connected to the redundancy analysis unit, and is used to determine the adjustment method according to the node state, that is, to adjust the sliding window length or to adjust the reference value of the number of data blocks, and to directly compress or differentially compress the data blocks according to the difference of the data blocks.

[0012] Further, the data processing unit determines the data state according to the data reference value of the data acquisition unit and the processing reference value of the data processing unit;

[0013] The data state includes a first data state in which the data reference value is greater than or equal to a preset data reference value or the processing reference value is less than a preset processing reference value, and a second data state in which the data reference value is less than the preset data reference value and the processing reference value is greater than or equal to the preset processing reference value.

[0014] Further, the data processing unit determines a processing mode according to the data state;

[0015] If the data state is the first data state, the data processing unit determines the processing method to be to determine the associated allocation method according to the reference quantity ratio;

[0016] If the data state is the second data state, the data processing unit determines that the processing method is to allocate the to-be-allocated data according to the node processing reference value.

[0017] Furthermore, the first allocation unit determines the associated allocation method according to the reference quantity ratio;

[0018] If the reference quantity ratio is greater than or equal to the preset reference quantity ratio, the associated allocation method is to allocate according to the associated influence coefficient;

[0019] If the reference quantity ratio is less than the preset reference quantity ratio, the association allocation method is to allocate according to the node coefficient;

[0020] The reference quantity ratio is equal to the ratio of the sum of the quantities of data acquisition modules corresponding to each associated combination to the quantity of all data acquisition modules.

[0021] Further, the first allocation unit performs association analysis on each data acquisition module in sequence according to a preset order, and records the data acquisition module that performs the association analysis as a target data acquisition module, wherein the association analysis includes detecting the association coefficient between the target data acquisition module and other data acquisition modules, and uniformly recording the data acquisition modules whose association coefficients with the target data acquisition module are greater than or equal to the preset association coefficient and the target acquisition module as an association combination, and after the association analysis for the target data acquisition module is completed, continuing the association analysis for the data acquisition modules that are not recorded as the association combination;

[0022] The calculation formula of the correlation coefficient r is:

[0023]

[0024] Where n is the number of data points contained in a single data column, x i and i are the values ​​of the i-th data point in the two data columns respectively, For x i The average value of the corresponding data column, for y i The average value of the corresponding data column, i = 1, 2, 3, ..., n;

[0025] The default order is from large to small.

[0026] Furthermore, the first allocation unit performs association allocation on each associated combination data according to an association preset order, wherein the first allocation unit allocates the associated combination data to a processing node with a minimum association influence coefficient in association allocation for a single associated combination data;

[0027] After the first allocation unit completes the allocation of the associated combined data, the non-associated data are sequentially allocated in an associated manner according to a preset order, and for a single non-associated data, the non-associated data is allocated to a processing node with the smallest associated influence coefficient;

[0028] The associated combination data is the data to be allocated of the data acquisition module corresponding to a single associated combination, and the non-associated data is the data to be allocated of a single data acquisition module that has not been associated with the combination;

[0029] The preset association order is the order from large to small of the combined data reference amount.

[0030] Furthermore, the first allocation unit allocates the associated combined data and the non-associated data according to the node coefficient;

[0031] The first allocation unit allocates each associated combination data to a corresponding processing node with the smallest node coefficient, and allocates each non-associated data to a corresponding processing node with the smallest node coefficient.

[0032] Furthermore, the second allocation unit allocates the data to be allocated according to the node processing reference value, and for a single data to be allocated, selects a processing node with a maximum node processing reference value for allocation.

[0033] Furthermore, when the redundant analysis unit performs redundant data analysis on a single processing node, the incoming data is divided into a number of data blocks of the same size, and a secure hash algorithm is used to generate a hash value corresponding to each data block, and the node status is determined according to the maximum difference between the data blocks and the proportion of the number of similar data blocks;

[0034] The node status includes a first node status in which the maximum difference of data blocks is greater than or equal to the preset maximum difference of data blocks or the ratio of similar data blocks is greater than or equal to the preset ratio of similar data blocks, and a second node status in which the maximum difference of data blocks is less than the preset maximum difference of data blocks and the ratio of similar data blocks is less than the preset ratio of similar data blocks.

[0035] Further, the redundant adjustment unit determines the adjustment mode according to the node status;

[0036] In the first node state, the redundancy adjustment unit determines that the adjustment method is to reduce the sliding window length;

[0037] In the second node state, the redundancy adjustment unit determines that the adjustment method is to increase the reference value of the number of data blocks;

[0038] The reduction value of the sliding window length is negatively correlated with the comprehensive difference;

[0039] The increase in the number of data blocks is positively correlated with the comprehensive difference.

[0040] Compared with the prior art, the beneficial effect of the present invention lies in that, in the technical scheme of the present invention, the data processing unit determines the data state according to the data reference value of the data acquisition unit and the processing reference value of the data processing unit, and effectively reflects the processing capability of the current processing node through the data reference value of the data acquisition unit and the processing reference value of the data processing unit, and adaptively selects different data allocation methods according to the data state, so that the selection of data allocation method is more in line with the actual working scenario, avoiding the problem of poor processing efficiency caused by excessive processing volume of data processed by the processing node, and then improving the processing efficiency of the processing node by reasonably allocating data.

[0041] Furthermore, the first allocation unit in the present invention determines the processing method according to the reference quantity ratio, and effectively reflects the correlation of the data to be allocated through the reference quantity ratio, and then adaptively selects the correlation allocation method, so that the selection of the correlation allocation method is more in line with the actual working scenario, and the data analysis efficiency is guaranteed by combining and allocating highly correlated data. At the same time, the weakly correlated data is allocated to other processing nodes, avoiding the problem of excessive data correlation in a single node leading to serious data loss when the processing node fails, thereby ensuring the security of the data.

[0042] Furthermore, the first allocation unit in the present invention effectively reflects the data throughput of the current processing node and the distance between the data acquisition module and the node through the node coefficient, and then allocates the associated combination data and the non-associated data according to the node coefficient, thereby ensuring that each associated combination data and each non-associated data can be allocated to a node with a stronger data throughput and a closer distance, avoiding the problems of data transmission delays and long waiting time, thereby improving data processing efficiency.

[0043] Furthermore, the second allocation unit in the present invention determines the allocation method according to the node processing reference value of each processing node, and effectively reflects the processing capacity of each processing node through the node processing reference value, thereby avoiding the problem of allocating the data to be allocated to nodes with poor processing capacity, thereby ensuring the rationality of data allocation.

[0044] Furthermore, the redundancy analysis unit in the present invention determines the node status according to the maximum difference of data blocks and the proportion of similar data blocks, and effectively reflects the data redundancy situation of the current processing node through the maximum difference of data blocks and the proportion of similar data blocks, and adaptively selects different adjustment methods according to the node status, so that the selection of adjustment method is more in line with the actual working scenario, avoiding the problem of misjudging normal data as redundant data and inaccurate identification of redundant data due to excessive length of data blocks, thereby improving the accuracy of redundant data identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A unit connection diagram of the electric power detection system based on multi-dimensional data of the present invention;

[0046] Figure 2 A flow chart of determining a data state according to a data reference value of a data acquisition unit and a processing reference value of a data processing unit according to the present invention;

[0047] Figure 3 A flow chart of the present invention for determining a processing method according to a data state;

[0048] Figure 4 This is a flow chart of the present invention for determining the adjustment method according to the node status. DETAILED DESCRIPTION

[0049] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0051] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0052] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0053] See also Figures 1 to 4 As shown, the present invention provides a power detection system based on multi-dimensional data, comprising:

[0054] A data acquisition unit, comprising a plurality of data acquisition modules for performing data acquisition;

[0055] A data processing unit connected to the data acquisition unit, configured to determine a data state according to a data reference value of the data acquisition unit and a processing reference value of the data processing unit, and determine a processing method according to the data state;

[0056] A first allocation unit, which is connected to the data processing unit, is used to determine a processing method for allocating associated combination data and non-associated data according to an associated influence coefficient or a node coefficient according to a reference quantity ratio;

[0057] A second allocation unit, connected to the data processing unit, for executing a processing method for allocating the data to be allocated according to the node processing reference value;

[0058] a redundancy analysis unit connected to the first allocation unit and the second allocation unit, for dividing the incoming data into a number of data blocks of the same size when performing redundant data analysis on a single processing node, and using a secure hash algorithm to generate a hash value corresponding to each data block, and determining the node status according to the maximum difference between the data blocks and the proportion of the number of similar data blocks;

[0059] The redundancy adjustment unit is connected to the redundancy analysis unit, and is used to determine the adjustment method according to the node state, that is, to adjust the sliding window length or to adjust the reference value of the number of data blocks, and to directly compress or differentially compress the data blocks according to the difference of the data blocks.

[0060] The data acquisition unit in the present invention is provided with a continuously cyclic monitoring cycle. At the end of each monitoring cycle, the data processing unit determines the data status once. The duration of the monitoring cycle can be set according to the needs of the user. The greater the user's demand for data allocation and processing efficiency, the shorter the duration of the monitoring cycle. A value of a monitoring cycle is provided, and the monitoring cycle is 1 minute. The data acquisition module of the present invention includes a voltage sensor, a wind speed sensor, a temperature sensor, and a vibration sensor, etc. The data collected by the data acquisition module is the data to be allocated. The data processing unit includes a number of processing nodes with processing capabilities. The specific setting position and setting number of the processing nodes are not specifically limited, and adaptive settings are performed according to the actual application scenario. This is content that is easy for technical personnel in this field to understand and will not be elaborated here.

[0061] In the present invention, direct compression is to compress all the data contents contained in the entire data block. For a single data block, differential compression uses a sliding window technology to find duplicate data between the data block and the reference data block, and compresses the duplicate data content to reduce storage space. The compression method that can be used can be but is not limited to Huffman coding, arithmetic coding or traditional compression algorithm. Users can choose according to actual needs. The reference data block is the historical data block that is closest to the characteristic value of the data block and stored in the redundant analysis unit. If the difference of the data block is within the first preset data block difference range, direct compression is performed on the data block; if the difference of the data block is within the second preset data block difference range, differential compression is performed on the data block; if the difference of the data block is within the third preset data block difference range, there is no need to compress the data block.

[0062] Among them, the values ​​within the first preset data block difference range are all less than or equal to the first preset data block difference, the values ​​within the second preset data block difference range are all greater than the first preset data block difference and less than or equal to the second preset data block difference, and the values ​​within the third preset data block difference range are all greater than the second preset data block difference. The values ​​of the first preset data block difference and the second preset data block difference can be set by the user according to the actual application scenario. The greater the user's demand for accurate processing of redundant data, the greater the values ​​of the first preset data block difference and the second preset data block difference. Provide a value of the first preset data block difference and the second preset data block difference, the first preset data block difference is 30%, and the second preset data block difference is 80%.

[0063] Specifically, the data processing unit determines the data state according to the data reference value of the data acquisition unit and the processing reference value of the data processing unit;

[0064] The data state includes a first data state in which the data reference value is greater than or equal to a preset data reference value or the processing reference value is less than a preset processing reference value, and a second data state in which the data reference value is less than the preset data reference value and the processing reference value is greater than or equal to the preset processing reference value.

[0065] Among them, the data reference value of the data acquisition unit is the sum of the data volume to be allocated collected by each data processing module in a single monitoring cycle, and the processing reference value of the data processing unit is the difference between the maximum data volume that each processing node can process and the data volume being processed by each processing node in a single monitoring cycle. The unit of data volume is GB.

[0066] The values ​​of the preset data reference value and the preset processing reference value can be set by the user according to the actual application scenario. The greater the user's demand for the stability of the processing node, the smaller the value of the preset data reference value, and the larger the value of the preset processing reference value. A preset data reference value and a preset processing reference value are provided. The preset data reference value is 50GB. The method for confirming the preset processing reference value is to detect the processing reference value corresponding to the record that is the same as the current data reference value in the historical monitoring cycle, and record the average value of the processing reference value whose processing speed meets the user's needs as the preset processing reference value. The historical monitoring cycle is all monitoring cycles before the current monitoring cycle.

[0067] Specifically, the data processing unit determines a processing mode according to the data state;

[0068] If the data state is the first data state, the data processing unit determines the processing method to be to determine the associated allocation method according to the reference quantity ratio;

[0069] If the data state is the second data state, the data processing unit determines that the processing method is to allocate the to-be-allocated data according to the node processing reference value.

[0070] Among them, for a single processing node, the node processing reference value is confirmed by recording the difference between the maximum amount of data that the single processing node can process and the amount of data being processed by the processing node in a single monitoring cycle as the node processing reference value of the processing node.

[0071] Specifically, the first allocation unit determines the associated allocation method according to the reference quantity ratio;

[0072] If the reference quantity ratio is greater than or equal to the preset reference quantity ratio, the associated allocation method is to allocate according to the associated influence coefficient;

[0073] If the reference quantity ratio is less than the preset reference quantity ratio, the association allocation method is to allocate according to the node coefficient;

[0074] The reference quantity ratio is equal to the ratio of the sum of the quantities of data acquisition modules corresponding to each associated combination to the quantity of all data acquisition modules.

[0075] The ratio=the sum of the number of data acquisition modules corresponding to each associated combination / the number of all data acquisition modules.

[0076] Specifically, the first allocation unit performs association analysis on each data acquisition module in sequence according to a preset order, and records the data acquisition module that performs the association analysis as a target data acquisition module, wherein the association analysis includes detecting the association coefficient between the target data acquisition module and other data acquisition modules, and uniformly recording the data acquisition modules whose association coefficients with the target data acquisition module are greater than or equal to the preset association coefficient and the target acquisition module as a correlation combination, and after the association analysis for the target data acquisition module is completed, continuing the association analysis for the data acquisition modules that are not recorded as the correlation combination;

[0077] The calculation formula of the correlation coefficient r is:

[0078]

[0079] Where n is the number of data points contained in a single data column, x i and i are the values ​​of the i-th data point in the two data columns respectively, For x i The average value of the corresponding data column, for y i The average value of the corresponding data column, i = 1, 2, 3, ..., n;

[0080] The default order is from large to small.

[0081] The data reference amount is the amount of data to be allocated collected by a single data acquisition module in a single monitoring cycle; the single data to be allocated is taken as the starting point of the starting data, and an interval point is set every 1s in chronological order, and the starting point and all interval points are recorded as time points, and the data is listed as the data value corresponding to each time point, and the data point is the data value corresponding to a single time point; x i The average value of the corresponding data column The calculation formula is: y i The average value of the corresponding data column The calculation formula is:

[0082]

[0083] The value of the preset correlation coefficient can be set according to the actual application scenario. The greater the user's demand for data analysis efficiency, the larger the value of the preset correlation coefficient. It can be understood that the value range of r is [0,1]. The higher the degree of correlation of the data, the closer the value of r is to 1. A value of the preset correlation coefficient is provided, r=0.8.

[0084] Specifically, the first allocation unit performs association allocation on each associated combination data according to an association preset order, wherein the first allocation unit allocates the associated combination data to a processing node with a minimum association influence coefficient in association allocation for a single associated combination data;

[0085] After the first allocation unit completes the allocation of the associated combined data, the non-associated data are sequentially allocated in an associated manner according to a preset order, and for a single non-associated data, the non-associated data is allocated to a processing node with the smallest associated influence coefficient;

[0086] The associated combination data is the data to be allocated of the data acquisition module corresponding to a single associated combination, and the non-associated data is the data to be allocated of a single data acquisition module that has not been associated with the combination;

[0087] The preset association order is the order from large to small of the combined data reference amount.

[0088] The combined data reference quantity is the sum of the data reference quantities of the data acquisition modules corresponding to a single associated combination in a single monitoring cycle.

[0089] The correlation influence coefficient is recorded as Ф. It can be understood that the confirmation method of the correlation influence coefficient corresponding to a single correlation combination and a single processing node is different from that of the correlation influence coefficient corresponding to a single data acquisition module without correlation combination and a single processing node. The calculation formula of the correlation influence coefficient Ф corresponding to a single correlation combination and a single processing node is: The calculation formula of the correlation influence coefficient Ф between a single data acquisition module that is not associated and combined and a single processing node is: Among them, n is the number of data acquisition modules corresponding to a single association combination, m is the number of data acquisition modules corresponding to a single processing node, and the data acquisition modules in a single association combination and the data acquisition modules in a single processing node are sorted in descending order according to the data reference amount. pd is the correlation coefficient between the pth data acquisition module in a single correlation combination and the dth data acquisition module in a single processing node, r d is the correlation coefficient between a single data acquisition module that has not been associated and combined and the dth data acquisition module in a single processing node.

[0090] Specifically, the first allocation unit allocates the associated combined data and the non-associated data according to the node coefficient;

[0091] The first allocation unit allocates each associated combination data to a corresponding processing node with the smallest node coefficient, and allocates each non-associated data to a corresponding processing node with the smallest node coefficient.

[0092] Among them, for a single processing node, the calculation formula of the node coefficient μ is: μ=lnT α1 +lnL α2 , where T is the data throughput of the processing node, the data throughput of the processing node = the amount of data processed by the processing node in a single monitoring cycle / the time required to transmit data, L is the reference distance, it can be understood that the confirmation method of the reference distance for the association combination and the data acquisition module is different, for a single association combination, the reference distance is the average value of the shortest distance from each data acquisition module to the processing node in the association combination, for a single data acquisition module, the reference distance is the shortest distance from the data acquisition module to the processing node, α1 is the weight coefficient corresponding to the data throughput of the processing node, α2 is the weight coefficient corresponding to the reference distance;

[0093] It can be understood that the weight coefficient corresponding to the data throughput of the processing node and the weight coefficient corresponding to the reference distance are set by the user according to the importance of the data throughput of the processing node and the reference distance, where α1+α2=1, and one value of α1 and α2 is provided, α1=0.5, α2=0.5.

[0094] Specifically, the second allocation unit allocates the data to be allocated according to the node processing reference value, and for a single data to be allocated, selects a processing node with the largest node processing reference value for allocation.

[0095] The node processing reference value is the maximum amount of data that a single processing node can process in a single detection cycle.

[0096] Specifically, when the redundant analysis unit performs redundant data analysis on a single processing node, it divides the incoming data into a number of data blocks of the same size, generates a hash value corresponding to each data block using a secure hash algorithm, and determines the node status based on the maximum difference between the data blocks and the proportion of the number of similar data blocks;

[0097] The node status includes a first node status in which the maximum difference of data blocks is greater than or equal to the preset maximum difference of data blocks or the ratio of similar data blocks is greater than or equal to the preset ratio of similar data blocks, and a second node status in which the maximum difference of data blocks is less than the preset maximum difference of data blocks and the ratio of similar data blocks is less than the preset ratio of similar data blocks.

[0098] The incoming data is divided into several data blocks of the same size. For a single data block, a sliding window is used to scan the content of the data block. The sliding window is initially located at the beginning of the data block, and then slides character by character in chronological order to the end of the data block. Each time the sliding window slides, the hash fingerprint of the byte sequence in the window is calculated, and all hash fingerprints of the data block are linearly transformed N times, and the maximum hash value of the hash fingerprint after N linear transformation operations is used as the feature value of the data block. The value of N can be set by the user according to the actual application scenario, and a value of N = 15 is provided.

[0099] For a single data block, the data block difference = the absolute value of the difference between the characteristic value of the data block and the characteristic value of the reference data block / the maximum value between the characteristic value of the data block and the characteristic value of the reference data block; for the data corresponding to a single processing node, the data within a single monitoring cycle is divided into several data blocks of the same size, and the maximum data block difference of the processing node is the maximum value of the data block differences corresponding to the data blocks corresponding to the processing node.

[0100] The ratio of similar data blocks = the reference value of similar data blocks / the reference value of data blocks.

[0101] The reference value of the number of similar data blocks is the number of data blocks whose data block difference is less than or equal to the second preset data block difference, and the reference value of the number of data blocks is the number of data blocks divided for data of a single processing node in a single monitoring cycle;

[0102] The values ​​of the preset maximum difference of data blocks and the ratio of the preset number of similar data blocks can be set according to the actual application scenario. The greater the user's demand for the accuracy of redundant data identification, the greater the values ​​of the preset maximum difference of data blocks and the ratio of the preset number of similar data blocks. Provided are values ​​of the preset maximum difference of data blocks and the ratio of the preset number of similar data blocks, the preset maximum difference of data blocks is 60%, and the preset number of similar data blocks is 50%.

[0103] Specifically, the redundancy adjustment unit determines the adjustment mode according to the node status;

[0104] In the first node state, the redundancy adjustment unit determines that the adjustment method is to reduce the sliding window length;

[0105] In the second node state, the redundancy adjustment unit determines that the adjustment method is to increase the reference value of the number of data blocks;

[0106] The reduction value of the sliding window length is negatively correlated with the comprehensive difference;

[0107] The increase in the number of data blocks is positively correlated with the comprehensive difference.

[0108] The comprehensive difference is confirmed by recording the sum of the differences of the data blocks corresponding to the data blocks divided by the data of a single processing node in a single monitoring cycle as the comprehensive difference. The sliding window length is the number of characters that the sliding window can accommodate. Decreasing the sliding window length is conducive to improving the accuracy of redundant data recognition; increasing the reference value of the number of data blocks can increase the amount of data blocks to avoid the problem of inaccurate redundant data recognition due to too little data blocks.

[0109] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A power detection system based on multidimensional data, characterized in that: include: A data acquisition unit, comprising a plurality of data acquisition modules for performing data acquisition; A data processing unit connected to the data acquisition unit, configured to determine a data state according to a data reference value of the data acquisition unit and a processing reference value of the data processing unit, and determine a processing method according to the data state; A first allocation unit, which is connected to the data processing unit, is used to determine a processing method for allocating associated combination data and non-associated data according to an associated influence coefficient or a node coefficient according to a reference quantity ratio; A second allocation unit, connected to the data processing unit, for executing a processing method for allocating the data to be allocated according to the node processing reference value; a redundancy analysis unit connected to the first allocation unit and the second allocation unit, for dividing the incoming data into a number of data blocks of the same size when performing redundant data analysis on a single processing node, and using a secure hash algorithm to generate a hash value corresponding to each data block, and determining the node status according to the maximum difference between the data blocks and the proportion of the number of similar data blocks; The redundancy adjustment unit is connected to the redundancy analysis unit, and is used to determine the adjustment method according to the node state, that is, to adjust the sliding window length or to adjust the reference value of the number of data blocks, and to directly compress or differentially compress the data blocks according to the difference of the data blocks.

2. The power detection system based on multidimensional data according to claim 1, characterized in that: The data processing unit determines the data state according to the data reference value of the data acquisition unit and the processing reference value of the data processing unit; The data state includes a first data state in which the data reference value is greater than or equal to a preset data reference value or the processing reference value is less than a preset processing reference value, and a second data state in which the data reference value is less than the preset data reference value and the processing reference value is greater than or equal to the preset processing reference value.

3. The power detection system based on multidimensional data according to claim 2, characterized in that: The data processing unit determines a processing mode according to the data state; If the data state is the first data state, the data processing unit determines the processing method to be to determine the associated allocation method according to the reference quantity ratio; If the data state is the second data state, the data processing unit determines that the processing method is to allocate the to-be-allocated data according to the node processing reference value.

4. The power detection system based on multidimensional data according to claim 3, characterized in that: The first allocation unit determines the associated allocation method according to the reference quantity ratio; If the reference quantity ratio is greater than or equal to the preset reference quantity ratio, the associated allocation method is to allocate according to the associated influence coefficient; If the reference quantity ratio is less than the preset reference quantity ratio, the association allocation method is to allocate according to the node coefficient; The reference quantity ratio is equal to the ratio of the sum of the quantities of data acquisition modules corresponding to each associated combination to the quantity of all data acquisition modules.

5. The power detection system based on multidimensional data according to claim 4, characterized in that: The first allocation unit sequentially performs association analysis on each data acquisition module in a preset order, and records the data acquisition module that performs the association analysis as a target data acquisition module, wherein the association analysis includes detecting the association coefficient between the target data acquisition module and other data acquisition modules, and uniformly recording the data acquisition modules whose association coefficients with the target data acquisition module are greater than or equal to the preset association coefficient and the target acquisition module as a correlation combination, and after the association analysis for the target data acquisition module is completed, continuing the association analysis for the data acquisition modules that are not recorded as the correlation combination; The calculation formula of the correlation coefficient r is: Where n is the number of data points contained in a single data column, x i and i are the values ​​of the i-th data point in the two data columns respectively, For x i The average value of the corresponding data column, for y i The average value of the corresponding data column, i = 1, 2, 3, ..., n; The default order is from large to small.

6. The multi-dimensional data-based power detection system according to claim 5, characterized in that: The first allocation unit performs association allocation on each associated combination data according to an association preset order, wherein the first allocation unit allocates the associated combination data to a processing node with a minimum association influence coefficient in association allocation for a single associated combination data; After the first allocation unit completes the allocation of the associated combined data, the non-associated data are sequentially allocated in an associated manner according to a preset order, and for a single non-associated data, the non-associated data is allocated to a processing node with the smallest associated influence coefficient; The associated combination data is the data to be allocated of the data acquisition module corresponding to a single associated combination, and the non-associated data is the data to be allocated of a single data acquisition module that has not been associated with the combination; The preset association order is the order from large to small of the combined data reference amount.

7. The power detection system based on multi-dimensional data according to claim 4, characterized in that: The first allocation unit allocates the associated combined data and the non-associated data according to the node coefficient; The first allocation unit allocates each associated combination data to a corresponding processing node with the smallest node coefficient, and allocates each non-associated data to a corresponding processing node with the smallest node coefficient.

8. The power detection system based on multi-dimensional data according to claim 3, characterized in that: The second allocation unit allocates the data to be allocated according to the node processing reference value, and for a single data to be allocated, selects a processing node with the largest node processing reference value for allocation.

9. The electric power detection system based on multi-dimensional data according to claim 8, characterized in that: When the redundant analysis unit performs redundant data analysis on a single processing node, it divides the incoming data into a number of data blocks of the same size, generates a hash value corresponding to each data block using a secure hash algorithm, and determines the node status based on the maximum difference between the data blocks and the proportion of the number of similar data blocks; The node status includes a first node status in which the maximum difference of data blocks is greater than or equal to the preset maximum difference of data blocks or the ratio of similar data blocks is greater than or equal to the preset ratio of similar data blocks, and a second node status in which the maximum difference of data blocks is less than the preset maximum difference of data blocks and the ratio of similar data blocks is less than the preset ratio of similar data blocks.

10. The electric power detection system based on multi-dimensional data according to claim 9, characterized in that: The redundancy adjustment unit determines an adjustment mode according to a node state; In the first node state, the redundancy adjustment unit determines that the adjustment method is to reduce the sliding window length; In the second node state, the redundancy adjustment unit determines that the adjustment method is to increase the reference value of the number of data blocks; The reduction value of the sliding window length is negatively correlated with the comprehensive difference; The increase in the number of data blocks is positively correlated with the comprehensive difference.

Images