Distributed data center node optimization method and system

Abstract

The present invention relates to the field of node data processing, and in particular to a distributed data center node optimization method and system, the method comprising: collecting data of each node to obtain a data sequence of each node; calculating the influence of each node; obtaining a comprehensive stability coefficient of each node according to the maximum value, minimum value and standard deviation of the data sequence of each node; obtaining each IMF component; calculating the data fluctuation factor and local stability factor of each IMF component sub-segment; obtaining the decomposition variation factor of each IMF component according to the relationship between each IMF component and the corresponding node data sequence; obtaining the local stability coefficient of the node according to the local stability factor and decomposition variation factor of each IMF component; and then obtaining the stability index of each node; obtaining the importance of each node in combination with the influence and stability index of each node, selecting a data center node according to the importance, and completing the optimization processing of the data center node. Thus, efficient selection of data center nodes is achieved.

Description

Technical Field

[0001] The present invention relates to the field of node data processing, and in particular to a distributed data center node optimization method and system. Background Art

[0002] With the rapid development of digital transformation, digital transformation is driving fundamental and rapid changes in business and its operation and maintenance models. At the same time, in order to cope with the challenges faced by data centers and adapt to the trend of technological development, the concept of distributed cloud data centers came into being. Distributed data centers are physically dispersed, logically unified, business-driven, operation-management-system, and business-aware data centers.

[0003] The selection of distributed data centers requires obtaining nodes with higher stability, and distributed data center nodes are no longer limited to solving the efficiency problems of a single data center. Therefore, the setting of distributed data centers needs to meet certain conditions. The selection and optimization of data center nodes is crucial for distributed data management.

[0004] To summarize, the present invention proposes a distributed data center node optimization method and system, which obtains the data sequence of each node, extracts the influence and stability index of each node, and then obtains the importance of each node, completes the adaptive selection of data center nodes according to the importance of each node, and realizes the optimization processing of data center nodes. Summary of the invention

[0005] In order to solve the above technical problems, the purpose of the present invention is to provide a distributed data center node optimization method and system, and the technical solutions adopted are as follows:

[0006] In a first aspect, an embodiment of the present invention provides a distributed data center node optimization method, the method comprising the following steps:

[0007] Obtain the data of each node to obtain the data sequence of each node;

[0008] The influence of each node is obtained according to the correlation coefficient between the data sequences of each node;

[0009] The comprehensive stability coefficient of each node is obtained according to the maximum value, minimum value and standard deviation of the data sequence of each node; the IMF components of the data sequence are obtained, and each IMF component is divided into multiple IMF component sub-segments, and the data fluctuation factor of each IMF component sub-segment is obtained according to the amplitude difference of each IMF component sub-segment, and then the local stability factor of the corresponding IMF component is obtained;

[0010] The decomposition variation factor of each IMF component is obtained according to the frequency domain correlation of each IMF component and the correlation between the corresponding node data sequence; the local stability coefficient of the node is obtained according to the local stability factor and decomposition variation factor of each IMF component; the stability index of each node is obtained according to the comprehensive stability coefficient and local stability coefficient of each node;

[0011] The importance of each node is obtained by combining the influence and stability index of each node, and the top K nodes in importance are selected as data center nodes to complete the optimization processing of data center nodes.

[0012] Furthermore, obtaining the influence of each node according to the Pearson coefficient between the data sequences of each node includes:

[0013] For each node data sequence, the mean of the Pearson correlation coefficients between the node data sequence and all other node data sequences is calculated, and the mean is used as the influence of the node.

[0014] Furthermore, the comprehensive stability coefficient of each node is obtained according to the maximum value, minimum value and standard deviation of the data sequence of each node, including:

[0015] The difference between the maximum and minimum values ​​of the node data sequence is obtained, and the product of the difference and the standard deviation of the node data sequence is calculated. The comprehensive stability coefficient of the node is an exponential function with the negative of the product as the exponent and the natural constant e as the base.

[0016] Furthermore, obtaining the data fluctuation factor of each IMF component subsegment according to the amplitude difference of each IMF component subsegment includes:

[0017] For each IMF component subsegment, the squares of adjacent amplitude differences in the subsegment are calculated, and the average of the sum of the squares of all adjacent amplitude differences in the subsegment is taken as the data fluctuation factor of the IMF component subsegment.

[0018] Furthermore, the acquisition of the local stability factor further includes:

[0019] The sum of the data fluctuation factors of all sub-segments of the IMF component is obtained, and the variance of the data fluctuation factors of each sub-segment of the MF component is calculated, and a power function with the inverse of the sum as the base and the variance as the power is used as the local stability factor of the IMF component.

[0020] Furthermore, the corresponding calculation formula of the decomposition change factor of each IMF component is: ; In the formula, is the short-time energy sequence of IMF component u, is the short-time energy sequence of the previous node data sequence, is the DTW distance, , Respectively IMF components, The frequency sequence after Fourier transform of the IMF components is , Respectively IMF components, The phase sequence after Fourier transform of the IMF components is represents the number of IMF components, is the Pearson correlation coefficient.

[0021] Furthermore, the local stability coefficient of the node is obtained according to the local stability factor of each IMF component and the decomposition variation factor, including:

[0022] The sum of the ratios of the local stability factor and the decomposition variation factor of all IMF components of the node data sequence is taken as the local stability coefficient of the node.

[0023] Furthermore, the stability index of each node is the product of the comprehensive stability coefficient of each node and the local stability coefficient.

[0024] Furthermore, the importance of each node is a weighted sum of the influence of each node and a stability index.

[0025] In a second aspect, an embodiment of the present invention further provides a distributed data center node optimization system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of any one of the above-mentioned methods when executing the computer program.

[0026] The present invention has at least the following beneficial effects:

[0027] The embodiment of the present invention analyzes the influence of each node according to the correlation between the data sequences of each node, takes into account the mutual influence between the nodes, and improves the detection accuracy of the importance of the node; at the same time, in order to avoid noise interference and modal aliasing, the embodiment of the present invention decomposes the data sequence of each node, and constructs a node stability index according to the comprehensive stability coefficient and the local stability coefficient of each IMF component, thereby improving the evaluation accuracy of the stability of the IMF component. At the same time, the stability of the node is detected from the overall and different scale components to ensure the reliability of the stability analysis of each node;

[0028] When analyzing the local stability coefficient, considering that the information consistency of the node data sequence with higher stability before and after decomposition is higher, the embodiment of the present invention constructs a decomposition change factor, and detects the decomposition change degree of each IMF component according to the difference between each IMF component after the node data sequence is decomposed and the node data sequence before decomposition, thereby improving the calculation accuracy of the node local stability coefficient and more comprehensively reflecting the local stability of each node;

[0029] Furthermore, the embodiment of the present invention selects the distributed data center nodes in combination with the importance of each node, which has strong objectivity and can objectively and adaptively select the central node according to the importance of each node in the node distribution, thereby solving the subjectivity of random selection, realizing the optimization processing of the central node, ensuring that the data center node selection result is objective, and improving the correctness of the central node selection process. The present invention has the beneficial effects of strong objectivity, high precision, and high reliability in data center node selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 A flow chart of the steps of a distributed data center node optimization method provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the distributed data center node optimization method and system proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0033] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0034] The specific scheme of the distributed data center node optimization method and system provided by the present invention is described in detail below with reference to the accompanying drawings.

[0035] See also Figure 1, which shows a flow chart of the steps of a distributed data center node optimization method provided by an embodiment of the present invention, the method comprising the following steps:

[0036] First, the data sequence of each node is collected as the basic data for node data analysis.

[0037] Specifically, this embodiment aims to obtain the importance of each data node by analyzing the data collected by each node multiple times, and select the central node according to the importance of each data node, so as to achieve the optimization processing of the central node, reduce the amount of data considered in the data node analysis process, and reduce the data exchange cost, and ensure the reliability of the central node selection. Therefore, this embodiment will first collect data from each data node. For n nodes, m data are collected respectively. It should be noted that n is the number of nodes, which depends on the actual situation, and m is the data collection amount of the node, that is, the total number of data collection moments. The implementer of each data collection moment can set it by himself. This embodiment does not limit it. In this embodiment, the number of nodes is 20, and each node collects 150 data. 20 sets of node data sequences with 150 data can be obtained. The node data sequence is recorded as: , is the data corresponding to node n at data collection time m, is the data sequence of node n. Similarly, obtain the data sequence of each node.

[0038] At this point, the data of each node can be acquired according to the above method of this embodiment, and a corresponding node data analysis matrix can be obtained for analyzing the data of each node.

[0039] Furthermore, the influence of each node is extracted, and the node data series is decomposed by IMF. The stability index of the node is calculated according to the comprehensive stability coefficient of each IMF component and the local stability coefficient. The importance of each node is obtained by combining the node influence and stability index. The importance of each node is evaluated as the basic indicator for the selection of the central node.

[0040] Specifically, this embodiment will extract the correlation information of each node data according to the relationship between each data in the node data analysis matrix, and analyze the stability of each node data as a characteristic indicator for evaluating the importance of each node.

[0041] First, this embodiment takes into account that the stronger the correlation with other nodes, the higher the degree of influence of the corresponding node data. Therefore, this embodiment will analyze the influence of each node. The degree of correlation between any two nodes is detected, and this embodiment takes nodes i and j as an example for detailed description. This embodiment extracts the overall correlation between node data sequences, calculates the correlation coefficient between node i and j data sequences, and uses it as an indicator for overall correlation analysis between nodes. It should be noted that this embodiment uses the Pearson correlation coefficient as the correlation coefficient between sequences. As other implementation methods, implementers can use other existing measurement methods for correlation coefficients between sequences, and this application does not impose any special restrictions on this.

[0042] Repeat the above method to obtain the Pearson correlation coefficient between node i and other nodes, and take the average of the Pearson correlation coefficients between node i and all other nodes as the influence of node i, and analyze the correlation and influence of node i in each node. The greater the influence, the higher the correlation between the node and other nodes, and the greater the influence of the change of node data on the change of data of other nodes.

[0043] The above method of this embodiment is repeated to obtain the influence of each node, which is used to detect the correlation between each node and other nodes.

[0044] Then, considering that the stability of the node is also crucial for the selection of the central node, the nodes with low stability have the problems of low data processing efficiency and long time consumption during data exchange and other data processing processes. At the same time, large data fluctuations will affect the data accuracy. Therefore, in order to improve the reliability of the central node selection, this embodiment further extracts the stability index of each node. The specific process of extracting the stability index is as follows:

[0045] Since data stability includes local stability and overall stability, this embodiment first detects the overall stability of the data of the node data sequence and constructs a comprehensive stability coefficient for comprehensively characterizing the data distribution of each node data sequence collected. The expression of the comprehensive stability coefficient is: ; In the formula, is the comprehensive stability coefficient of node i, e is a natural constant, are the maximum and minimum values ​​in the data sequence of node i, respectively. is the standard deviation of the data sequence of node i. The larger the comprehensive stability coefficient, the smaller the fluctuation of the data distribution of the corresponding node, and the higher the overall stability of the node data sequence;

[0046] Furthermore, this embodiment takes into account that the comprehensive stability coefficient mainly reflects the overall stability of the node data, and cannot accurately detect the distribution of each node data under different scale signal components, which is prone to generalization, resulting in local errors in the determination of node data stability. Therefore, this embodiment will extract the local stability coefficient of each node for detecting the data status of each node under different scale signal components. This embodiment first uses a modal decomposition algorithm to perform IMF decomposition on the data sequence of each node to obtain the IMF components of the node data sequence. This embodiment will detect the local stability of the node data sequence based on the analysis of each IMF component. It should be noted that there are many modal decomposition algorithms, and the implementer can choose them at will. This embodiment does not limit this. This embodiment uses an empirical mode decomposition algorithm. The specific decomposition process is an existing well-known technology and is not within the scope of protection of this embodiment. No relevant details are given here.

[0047] Then analyze the data characteristics of each IMF component. Taking the IMF components as an example, the detailed description is given. Considering that the fluctuation of data in different frequency bands is different, for each IMF component, the amplitude fluctuation of each IMF component is accurately analyzed. For each IMF component, this implementation will refine the IMF component to obtain each sub-segment of the IMF component, and analyze the amplitude change of each IMF component in different frequency bands. The refinement is to divide the IMF component into equal intervals on the horizontal axis, calculate the data fluctuation factor of each sub-segment, and analyze the amplitude fluctuation of each IMF component to construct the local stability factor of the IMF component: ;in, In both formulas, are the p+1th and pth amplitude values ​​of the vth subsegment of the IMF component u, is the number of amplitudes contained in the vth sub-segment of IMF component u, is the data volatility factor of the vth sub-segment of IMF component u, is the variance of the volatility factor of each sub-segment of IMF component u, Q is the number of sub-segments of IMF component u, is the local stability factor of the IMF component u. The larger the local stability factor, the smaller the amplitude fluctuation of the corresponding IMF component, and the better the stability of the IMF component;

[0048] Repeat the above method to obtain the local stability factor of each IMF component.

[0049] Furthermore, this embodiment takes into account that the modal components at each scale obtained after decomposition can effectively avoid the modal aliasing effect, and at the same time avoid the interference and influence of noise. Therefore, the more consistent the information corresponding to each IMF component after decomposition is with the corresponding information of the original data sequence before decomposition, the higher the stability of the node data sequence, that is, the node data sequence is less likely to be affected by external noise. Therefore, this embodiment will analyze the characteristics of each node data sequence before and after decomposition to evaluate the degree of difference between the data before and after decomposition.

[0050] This embodiment constructs the decomposition change factor of the IMF component and detects and analyzes the characteristics of each node data before and after decomposition. The expression of the decomposition change factor is: ; In the formula, is the short-time energy sequence of IMF component u, To decompose the short-time energy sequence of the previous node data sequence, it should be noted that the calculation of the short-time energy sequence is an existing well-known technology, which is not within the protection scope of this embodiment and will not be elaborated here;

[0051] Represents the DTW distance between two sequences. The calculation of DTW distance is a well-known technique. The larger the distance, the greater the short-term energy difference between the IMF component and the node data sequence before decomposition. In other words, the data sequence of the node is less susceptible to external noise, and the corresponding node has higher stability.

[0052] , Respectively represent IMF components, The frequency sequence after Fourier transform of the IMF components is , Respectively represent IMF components, The phase sequence after Fourier transform of the IMF components is represents the number of IMF components, is the Pearson correlation coefficient, and the calculation of the Pearson correlation coefficient is a well-known technology.

[0053] It should be noted that the above formula In the calculation process, the larger the numerator, the higher the correlation between the IMF component and other IMF components, the higher the stability of the side reaction node, and the less obvious the fluctuation of the IMF component of the node;

[0054] Combined with the local stability factor of each IMF component of the node and the decomposition change factor, the stability of each node is tested to construct the node local stability coefficient. The expression of the local stability coefficient is: ; In the formula, is the local stability coefficient of node i, T is the number of IMF components, To avoid the minimum value of zero denominator, the implementer can select it by himself. In this embodiment, it is set to , are the local stability factor and decomposition variation factor of the u-th IMF component respectively.

[0055] It should be noted that the larger the local stability index is, the higher the stability of the corresponding node is, the less susceptible it is to external factors, and the stronger the node reliability is;

[0056] Furthermore, this embodiment will combine the comprehensive stability coefficient and the local stability coefficient of each node to accurately calculate the stability index of each node. The specific expression of the stability index of each node is: ; In the formula, are the local stability coefficient and comprehensive stability coefficient of node i respectively, is the stability index of node i.

[0057] It should be noted that the larger the stability index is, the higher the stability of the corresponding node is, the more suitable it is for a data center node, and the more refined it can ensure the normal and efficient operation of data storage and computing.

[0058] By repeating the above method of this embodiment, the stability index of each node can be obtained, which is used to accurately evaluate the stability of each node.

[0059] Thus, according to the above method of this embodiment, the influence degree and stability coefficient of each node can be obtained, which are used to analyze the data characteristics of each node.

[0060] Finally, this embodiment will determine the comprehensive importance of each node based on the influence and stability index of each node, construct a node importance index, characterize the importance of each node, and use it as a key indicator for central node selection. The node importance expression is specifically: ; In the formula, is a weight coefficient (0,1), which can be set by the implementer. , are the influence degree and stability index of node i respectively, is the importance of node i.

[0061] It should be noted that the higher the importance of the node, the higher the influence of the corresponding node among all nodes, the greater the correlation with other nodes, and the higher the node stability, and the more suitable it is to be a data center node.

[0062] The above method of this embodiment is repeated to obtain the importance of each node, which is used to characterize the importance of each node in the distributed data node and serves as a key indicator for selecting distributed data center nodes.

[0063] Furthermore, data center nodes are selected according to the importance of each node to complete the optimization process of data center nodes.

[0064] According to the above process of this embodiment, the importance of each node can be accurately extracted, and the importance of each node in the data node can be evaluated. The greater the importance, the higher the correlation between the corresponding node and other nodes, and the higher the data stability at the node, the more suitable it is as a data center node. Therefore, this embodiment will sort the nodes from large to small according to the importance, select the first K nodes as the central nodes, and realize the adaptive selection of data center nodes.

[0065] It should be noted that the number of central nodes K is set by the implementer according to actual conditions. In this embodiment, the number of central nodes K=5, that is, the nodes corresponding to the top 5 in importance are selected as data center nodes to complete the adaptive selection optimization of data center nodes and avoid the randomness and limitations of subjective selection.

[0066] At this point, according to the above method of this embodiment, it is possible to optimize the distributed data center nodes, efficiently select the distributed data center nodes, improve the data exchange efficiency, and ensure data stability.

[0067] Based on the same inventive concept as the above method, an embodiment of the present invention also provides a distributed data center node optimization system, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the steps of any one of the above-mentioned distributed data center node optimization methods are implemented.

[0068] In summary, the embodiment of the present invention analyzes the influence of each node according to the correlation between the data sequences of each node, takes into account the mutual influence between the nodes, and improves the detection accuracy of the importance of the node; at the same time, in order to avoid noise interference and modal aliasing, the embodiment of the present invention decomposes the data sequence of each node, and constructs a node stability index according to the comprehensive stability coefficient and the local stability coefficient of each IMF component, thereby improving the evaluation accuracy of the stability of the IMF component, and at the same time, detects the stability of the node from the overall and different scale components, thereby ensuring the reliability of the stability analysis of each node;

[0069] When analyzing the local stability coefficient, considering that the information consistency of the node data sequence with higher stability before and after decomposition is higher, the embodiment of the present invention constructs a decomposition change factor, and detects the decomposition change degree of each IMF component according to the difference between each IMF component after the node data sequence is decomposed and the node data sequence before decomposition, thereby improving the calculation accuracy of the node local stability coefficient and more comprehensively reflecting the local stability of each node;

[0070] Furthermore, the embodiment of the present invention selects distributed data center nodes based on the importance of each node, has strong objectivity, and can objectively and adaptively select central nodes based on the importance of each node in the node distribution, thereby solving the subjectivity of random selection, achieving optimized processing of central nodes, ensuring that the data center node selection results are objective, and improving the correctness of the central node selection process.

[0071] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The above is a description of a specific embodiment of this specification. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0072] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A distributed data center node optimization method, characterized in that: The method comprises the following steps: Obtain the data of each node to obtain the data sequence of each node; The influence of each node is obtained according to the correlation coefficient between the data sequences of each node; The comprehensive stability coefficient of each node is obtained according to the maximum value, minimum value and standard deviation of the data sequence of each node; the IMF components of the data sequence are obtained, and each IMF component is divided into multiple IMF component sub-segments, and the data fluctuation factor of each IMF component sub-segment is obtained according to the amplitude difference of each IMF component sub-segment, and then the local stability factor of the corresponding IMF component is obtained; The decomposition variation factor of each IMF component is obtained according to the frequency domain correlation of each IMF component and the correlation between the corresponding node data sequence; the local stability coefficient of the node is obtained according to the local stability factor and decomposition variation factor of each IMF component; the stability index of each node is obtained according to the comprehensive stability coefficient and local stability coefficient of each node; The importance of each node is obtained by combining the influence and stability index of each node, and the top K nodes in importance ranking are selected as data center nodes to complete the optimization processing of data center nodes; The step of obtaining the influence of each node according to the Pearson coefficient between the data sequences of each node includes: For each node data sequence, calculate the mean of the Pearson correlation coefficients between the node data sequence and all other node data sequences, and use the mean as the influence of the node; The comprehensive stability coefficient of each node obtained according to the maximum value, minimum value and standard deviation of the data sequence of each node includes: Obtain the difference between the maximum and minimum values ​​of the node data sequence, calculate the product of the difference and the standard deviation of the node data sequence, and the comprehensive stability coefficient of the node is an exponential function with the opposite of the product as the exponent and the natural constant e as the base; The step of obtaining the data fluctuation factor of each IMF component subsegment according to the amplitude difference of each IMF component subsegment includes: For each IMF component subsegment, the square of the adjacent amplitude difference in the subsegment is calculated, and the average of the sum of the squares of all adjacent amplitude differences in the subsegment is taken as the data fluctuation factor of the IMF component subsegment; The acquisition of the local stability factor further includes: Obtain the sum of the data fluctuation factors of all sub-segments of the IMF component, and calculate the variance of the data fluctuation factors of each sub-segment of the MF component, and use a power function with the reciprocal of the sum as the base and the variance as the power as the local stability factor of the IMF component; The corresponding calculation formula for the decomposition change factor of each IMF component is: ; In the formula, is the short-time energy sequence of IMF component u, is the short-time energy sequence of the previous node data sequence, is the DTW distance, , Respectively IMF components, The frequency sequence after Fourier transform of the IMF components is , Respectively IMF components, The phase sequence after Fourier transform of the IMF components is represents the number of IMF components, is the Pearson correlation coefficient; The local stability coefficient of the node obtained according to the local stability factor of each IMF component and the decomposition change factor includes: The sum of the ratios of the local stability factor and the decomposition change factor of all IMF components of the node data sequence is taken as the local stability coefficient of the node; The stability index of each node is the product of the comprehensive stability coefficient of each node and the local stability coefficient.

2. The distributed data center node optimization method according to claim 1, characterized in that: The importance of each node is the weighted sum of the influence and stability index of each node.

3. A distributed data center node optimization system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 2 are implemented.

Images